2024-03-30T12:56:28.876 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-30T12:56:28.876 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_mchangir@teuthology --name mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi --description fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} --verbose --archive /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150 -- /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/orig.config.yaml 2024-03-30T12:56:28.923 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T12:56:29.017 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150 branch: wip-mchangir-testing1-main-20240328-072412 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} email: mchangir@redhat.com first_in_suite: false job_id: '7629150' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-mchangir-testing1-main-20240328-072412 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing1-main-20240328-072412 sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 owner: scheduled_mchangir@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 364 sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-mchangir-testing1-main-20240328-072412 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNaqCXgku5wm8rAduSfSpUMrux+P1koXaV2BHr2lfdroQgKNRMUBxi1nIm84HEYi0uXdSPkeSvgHbdEy4/3xDJQ= smithi113.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO8cAcGOMjXJOBLP1+rNVUwlOfwTYgbuBqwTwBM8uzVGbh6LQallKPwYP2ellDlBciZriUua4cEwE3jlHVMtDnM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_uninlining teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-29_13:38:54 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-30T12:56:29.017 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa; will attempt to use it 2024-03-30T12:56:29.018 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks 2024-03-30T12:56:29.018 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-30T12:56:29.020 INFO:teuthology.task.internal:Checking packages... 2024-03-30T12:56:29.045 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7' 2024-03-30T12:56:29.045 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-30T12:56:29.045 INFO:teuthology.packaging:ref: None 2024-03-30T12:56:29.045 INFO:teuthology.packaging:tag: None 2024-03-30T12:56:29.045 INFO:teuthology.packaging:branch: wip-mchangir-testing1-main-20240328-072412 2024-03-30T12:56:29.045 INFO:teuthology.packaging:sha1: ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T12:56:29.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-mchangir-testing1-main-20240328-072412 2024-03-30T12:56:29.186 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2519-gac171039-1jammy 2024-03-30T12:56:29.188 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-30T12:56:29.195 INFO:teuthology.task.internal:no buildpackages task found 2024-03-30T12:56:29.195 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-30T12:56:29.344 INFO:teuthology.task.internal:Saving configuration 2024-03-30T12:56:29.360 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-30T12:56:29.366 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T12:56:29.392 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:21.705614', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID3UgqP8puyVo7FePJraqyKvieHy08r+3oM5KhgQ4vDh'} 2024-03-30T12:56:29.412 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:21.703701', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ912m3pkV+Cejeizh0xV/OUX2PCsty0oopPyyX+q/yL'} 2024-03-30T12:56:29.413 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-30T12:56:29.418 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-30T12:56:29.419 INFO:teuthology.task.internal:roles: ubuntu@smithi113.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-30T12:56:29.419 INFO:teuthology.run_tasks:Running task console_log... 2024-03-30T12:56:29.617 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7facb4d0b3a0>, signals=[15]) 2024-03-30T12:56:29.618 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-30T12:56:29.624 INFO:teuthology.task.internal:Opening connections... 2024-03-30T12:56:29.624 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T12:56:29.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:56:29.707 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T12:56:29.708 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:56:29.785 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-30T12:56:29.792 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-03-30T12:56:29.797 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-03-30T12:56:29.797 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-03-30T12:56:29.845 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-30T12:56:29.845 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="Ubuntu" 2024-03-30T12:56:29.845 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="22.04" 2024-03-30T12:56:29.845 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-30T12:56:29.845 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_CODENAME=jammy 2024-03-30T12:56:29.845 INFO:teuthology.orchestra.run.smithi043.stdout:ID=ubuntu 2024-03-30T12:56:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE=debian 2024-03-30T12:56:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-30T12:56:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-30T12:56:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-30T12:56:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-30T12:56:29.846 INFO:teuthology.orchestra.run.smithi043.stdout:UBUNTU_CODENAME=jammy 2024-03-30T12:56:29.846 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-03-30T12:56:29.875 DEBUG:teuthology.orchestra.run.smithi113:> uname -m 2024-03-30T12:56:29.880 INFO:teuthology.orchestra.run.smithi113.stdout:x86_64 2024-03-30T12:56:29.880 DEBUG:teuthology.orchestra.run.smithi113:> cat /etc/os-release 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:NAME="Ubuntu" 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_ID="22.04" 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:VERSION_CODENAME=jammy 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:ID=ubuntu 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:ID_LIKE=debian 2024-03-30T12:56:29.930 INFO:teuthology.orchestra.run.smithi113.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-30T12:56:29.931 INFO:teuthology.orchestra.run.smithi113.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-30T12:56:29.931 INFO:teuthology.orchestra.run.smithi113.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-30T12:56:29.931 INFO:teuthology.orchestra.run.smithi113.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-30T12:56:29.931 INFO:teuthology.orchestra.run.smithi113.stdout:UBUNTU_CODENAME=jammy 2024-03-30T12:56:29.931 INFO:teuthology.lock.ops:Updating smithi113.front.sepia.ceph.com on lock server 2024-03-30T12:56:29.955 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-30T12:56:29.964 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-30T12:56:30.036 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-30T12:56:30.036 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-03-30T12:56:30.039 DEBUG:teuthology.orchestra.run.smithi113:> test '!' -e /home/ubuntu/cephtest 2024-03-30T12:56:30.044 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-30T12:56:30.049 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-30T12:56:30.049 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-03-30T12:56:30.087 DEBUG:teuthology.orchestra.run.smithi113:> test -z $(ls -A /var/lib/ceph) 2024-03-30T12:56:30.102 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-30T12:56:30.227 INFO:teuthology.run_tasks:Running task kernel... 2024-03-30T12:56:30.368 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-30T12:56:30.368 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-30T12:56:30.369 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T12:56:30.369 DEBUG:teuthology.orchestra.run.smithi113:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-30T12:56:30.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T12:56:30.374 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-03-30T12:56:30.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T12:56:30.375 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-03-30T12:56:30.422 INFO:teuthology.orchestra.run.smithi043.stdout:5.15.0-56-generic 2024-03-30T12:56:30.423 INFO:teuthology.orchestra.run.smithi113.stdout:5.15.0-56-generic 2024-03-30T12:56:30.423 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.15.0-56-generic 2024-03-30T12:56:30.423 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get clean 2024-03-30T12:56:30.423 INFO:teuthology.task.kernel:Running kernel on smithi113: 5.15.0-56-generic 2024-03-30T12:56:30.423 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get clean 2024-03-30T12:56:30.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get update 2024-03-30T12:56:30.550 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get update 2024-03-30T12:56:30.732 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T12:56:30.764 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-30T12:56:30.764 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-30T12:56:30.774 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-30T12:56:30.823 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T12:56:30.890 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-30T12:56:30.897 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-30T12:56:30.982 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-03-30T12:56:31.053 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-03-30T12:56:31.093 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,303 kB] 2024-03-30T12:56:31.184 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-03-30T12:56:31.190 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,519 kB] 2024-03-30T12:56:31.198 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-03-30T12:56:31.205 INFO:teuthology.orchestra.run.smithi113.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-30T12:56:31.206 INFO:teuthology.orchestra.run.smithi113.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-30T12:56:31.207 INFO:teuthology.orchestra.run.smithi113.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-03-30T12:56:31.255 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-30T12:56:31.263 INFO:teuthology.orchestra.run.smithi113.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-03-30T12:56:31.271 INFO:teuthology.orchestra.run.smithi113.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-30T12:56:31.271 INFO:teuthology.orchestra.run.smithi113.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-03-30T12:56:31.294 INFO:teuthology.orchestra.run.smithi113.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-30T12:56:31.303 INFO:teuthology.orchestra.run.smithi113.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-03-30T12:56:31.305 INFO:teuthology.orchestra.run.smithi113.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-03-30T12:56:31.338 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [436 kB] 2024-03-30T12:56:31.351 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [233 kB] 2024-03-30T12:56:31.356 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-30T12:56:31.357 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,616 kB] 2024-03-30T12:56:31.382 INFO:teuthology.orchestra.run.smithi113.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-30T12:56:31.382 INFO:teuthology.orchestra.run.smithi113.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-30T12:56:31.383 INFO:teuthology.orchestra.run.smithi113.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-30T12:56:31.383 INFO:teuthology.orchestra.run.smithi113.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-03-30T12:56:31.386 INFO:teuthology.orchestra.run.smithi113.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-30T12:56:31.387 INFO:teuthology.orchestra.run.smithi113.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-30T12:56:31.387 INFO:teuthology.orchestra.run.smithi113.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-30T12:56:31.395 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,519 kB] 2024-03-30T12:56:31.414 INFO:teuthology.orchestra.run.smithi113.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-30T12:56:31.415 INFO:teuthology.orchestra.run.smithi113.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-03-30T12:56:31.421 INFO:teuthology.orchestra.run.smithi113.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-30T12:56:31.421 INFO:teuthology.orchestra.run.smithi113.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-03-30T12:56:31.433 INFO:teuthology.orchestra.run.smithi113.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-03-30T12:56:31.441 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-30T12:56:31.442 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [271 kB] 2024-03-30T12:56:31.452 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-30T12:56:31.452 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-03-30T12:56:31.454 INFO:teuthology.orchestra.run.smithi113.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-03-30T12:56:31.459 INFO:teuthology.orchestra.run.smithi113.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-30T12:56:31.474 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-30T12:56:31.487 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-03-30T12:56:31.490 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-30T12:56:31.491 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-30T12:56:31.494 INFO:teuthology.orchestra.run.smithi113.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-03-30T12:56:31.495 INFO:teuthology.orchestra.run.smithi113.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-03-30T12:56:31.496 INFO:teuthology.orchestra.run.smithi113.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-03-30T12:56:31.497 INFO:teuthology.orchestra.run.smithi113.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-30T12:56:31.497 INFO:teuthology.orchestra.run.smithi113.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-30T12:56:31.499 INFO:teuthology.orchestra.run.smithi113.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-30T12:56:31.500 INFO:teuthology.orchestra.run.smithi113.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-30T12:56:31.513 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-30T12:56:31.513 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-30T12:56:31.513 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-30T12:56:31.529 INFO:teuthology.orchestra.run.smithi113.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-30T12:56:31.529 INFO:teuthology.orchestra.run.smithi113.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-30T12:56:31.530 INFO:teuthology.orchestra.run.smithi113.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-30T12:56:31.531 INFO:teuthology.orchestra.run.smithi113.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-30T12:56:31.531 INFO:teuthology.orchestra.run.smithi113.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-30T12:56:31.701 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [602 kB] 2024-03-30T12:56:31.741 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [293 kB] 2024-03-30T12:56:31.756 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-30T12:56:31.757 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-30T12:56:31.758 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,644 kB] 2024-03-30T12:56:31.817 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [274 kB] 2024-03-30T12:56:31.824 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-30T12:56:31.989 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [698 kB] 2024-03-30T12:56:32.574 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,060 kB] 2024-03-30T12:56:32.686 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [241 kB] 2024-03-30T12:56:32.705 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-30T12:56:32.707 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [49.6 kB] 2024-03-30T12:56:32.710 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [6,028 B] 2024-03-30T12:56:32.711 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [12.0 kB] 2024-03-30T12:56:32.712 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-30T12:56:32.712 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-30T12:56:32.717 INFO:teuthology.orchestra.run.smithi043.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-30T12:56:32.721 INFO:teuthology.orchestra.run.smithi043.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-30T12:56:32.886 INFO:teuthology.orchestra.run.smithi043.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-30T12:56:32.969 INFO:teuthology.orchestra.run.smithi043.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-30T12:56:33.070 INFO:teuthology.orchestra.run.smithi043.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-30T12:56:33.132 INFO:teuthology.orchestra.run.smithi043.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-30T12:56:33.161 INFO:teuthology.orchestra.run.smithi043.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-30T12:56:36.946 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 12.6 MB in 2s (6,323 kB/s) 2024-03-30T12:56:37.774 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 12.6 MB in 3s (4,956 kB/s) 2024-03-30T12:56:38.430 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-30T12:56:38.448 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-30T12:56:38.503 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-30T12:56:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-03-30T12:56:38.699 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-03-30T12:56:38.849 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T12:56:38.849 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T12:56:38.849 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T12:56:38.849 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T12:56:38.849 INFO:teuthology.orchestra.run.smithi113.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-30T12:56:38.850 INFO:teuthology.orchestra.run.smithi113.stdout: python2.7-minimal 2024-03-30T12:56:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T12:56:38.850 INFO:teuthology.orchestra.run.smithi113.stdout:The following additional packages will be installed: 2024-03-30T12:56:38.851 INFO:teuthology.orchestra.run.smithi113.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-30T12:56:38.851 INFO:teuthology.orchestra.run.smithi113.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-30T12:56:38.851 INFO:teuthology.orchestra.run.smithi113.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-30T12:56:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:Suggested packages: 2024-03-30T12:56:38.852 INFO:teuthology.orchestra.run.smithi113.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-30T12:56:38.852 INFO:teuthology.orchestra.run.smithi113.stdout:Recommended packages: 2024-03-30T12:56:38.852 INFO:teuthology.orchestra.run.smithi113.stdout: thermald 2024-03-30T12:56:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:The following NEW packages will be installed: 2024-03-30T12:56:38.908 INFO:teuthology.orchestra.run.smithi113.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-30T12:56:38.908 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-30T12:56:38.908 INFO:teuthology.orchestra.run.smithi113.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-30T12:56:38.908 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages will be upgraded: 2024-03-30T12:56:38.909 INFO:teuthology.orchestra.run.smithi113.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-30T12:56:39.081 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-03-30T12:56:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:3 upgraded, 5 newly installed, 0 to remove and 311 not upgraded. 2024-03-30T12:56:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:Need to get 113 MB of archives. 2024-03-30T12:56:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-30T12:56:39.087 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-30T12:56:39.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-30T12:56:39.161 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-03-30T12:56:39.362 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-03-30T12:56:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-03-30T12:56:39.543 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T12:56:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T12:56:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T12:56:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T12:56:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-30T12:56:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: python2.7-minimal 2024-03-30T12:56:39.544 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T12:56:39.546 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-03-30T12:56:39.546 INFO:teuthology.orchestra.run.smithi043.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-30T12:56:39.546 INFO:teuthology.orchestra.run.smithi043.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-30T12:56:39.546 INFO:teuthology.orchestra.run.smithi043.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-30T12:56:39.547 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-03-30T12:56:39.547 INFO:teuthology.orchestra.run.smithi043.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-30T12:56:39.547 INFO:teuthology.orchestra.run.smithi043.stdout:Recommended packages: 2024-03-30T12:56:39.548 INFO:teuthology.orchestra.run.smithi043.stdout: thermald 2024-03-30T12:56:39.600 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-03-30T12:56:39.600 INFO:teuthology.orchestra.run.smithi043.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-30T12:56:39.600 INFO:teuthology.orchestra.run.smithi043.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-30T12:56:39.600 INFO:teuthology.orchestra.run.smithi043.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-30T12:56:39.601 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages will be upgraded: 2024-03-30T12:56:39.602 INFO:teuthology.orchestra.run.smithi043.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-30T12:56:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:3 upgraded, 5 newly installed, 0 to remove and 311 not upgraded. 2024-03-30T12:56:39.661 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 113 MB of archives. 2024-03-30T12:56:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-30T12:56:39.662 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-30T12:56:40.182 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-30T12:56:40.324 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-30T12:56:40.473 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-30T12:56:40.817 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-30T12:56:41.444 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-30T12:56:41.444 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-30T12:56:41.444 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-30T12:56:41.621 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-30T12:56:41.671 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-30T12:56:42.235 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 113 MB in 2s (54.9 MB/s) 2024-03-30T12:56:42.401 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-30T12:56:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-30T12:56:42.940 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-30T12:56:43.038 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-30T12:56:44.047 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-30T12:56:44.197 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-30T12:56:44.461 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-30T12:56:44.465 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:56:44.573 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:56:44.586 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 113 MB in 5s (21.4 MB/s) 2024-03-30T12:56:44.791 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-30T12:56:46.782 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-30T12:56:46.804 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:56:46.876 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-30T12:56:46.880 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:56:46.898 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:56:47.122 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:56:47.494 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-30T12:56:47.514 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:56:47.552 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:56:49.366 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-30T12:56:49.387 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:56:49.483 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:56:49.994 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-30T12:56:50.015 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:56:50.069 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:56:54.310 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-30T12:56:54.412 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-30T12:56:54.762 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-30T12:56:54.899 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-30T12:56:55.227 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-30T12:56:55.250 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-30T12:56:55.293 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-30T12:56:57.100 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-30T12:56:57.241 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-30T12:56:57.600 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-30T12:56:57.762 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-30T12:56:58.065 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-30T12:56:58.087 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-30T12:56:58.165 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-30T12:57:01.612 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-30T12:57:01.643 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:57:01.686 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:04.575 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-30T12:57:04.617 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-30T12:57:04.653 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-30T12:57:04.691 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:04.706 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-30T12:57:05.065 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-30T12:57:05.186 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:05.312 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-30T12:57:05.466 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:06.822 INFO:teuthology.orchestra.run.smithi043.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-30T12:57:06.822 INFO:teuthology.orchestra.run.smithi043.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-30T12:57:06.822 INFO:teuthology.orchestra.run.smithi043.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-30T12:57:06.823 INFO:teuthology.orchestra.run.smithi043.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:06.994 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:08.436 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:08.524 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-30T12:57:08.654 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-30T12:57:09.049 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-30T12:57:09.176 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:09.327 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-30T12:57:09.456 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:09.862 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-30T12:57:09.988 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-30T12:57:10.114 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:10.185 INFO:teuthology.orchestra.run.smithi043.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-30T12:57:10.185 INFO:teuthology.orchestra.run.smithi043.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:10.832 INFO:teuthology.orchestra.run.smithi113.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-30T12:57:10.832 INFO:teuthology.orchestra.run.smithi113.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-30T12:57:10.832 INFO:teuthology.orchestra.run.smithi113.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-30T12:57:10.832 INFO:teuthology.orchestra.run.smithi113.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:11.026 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:12.787 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:14.221 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-30T12:57:14.322 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-30T12:57:14.462 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-30T12:57:14.521 INFO:teuthology.orchestra.run.smithi113.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-30T12:57:14.521 INFO:teuthology.orchestra.run.smithi113.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:24.943 INFO:teuthology.orchestra.run.smithi043.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-30T12:57:24.943 INFO:teuthology.orchestra.run.smithi043.stdout:Sourcing file `/etc/default/grub' 2024-03-30T12:57:24.971 INFO:teuthology.orchestra.run.smithi043.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T12:57:25.028 INFO:teuthology.orchestra.run.smithi043.stdout:Generating grub configuration file ... 2024-03-30T12:57:25.380 INFO:teuthology.orchestra.run.smithi043.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-30T12:57:25.411 INFO:teuthology.orchestra.run.smithi043.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:25.806 INFO:teuthology.orchestra.run.smithi043.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T12:57:25.814 INFO:teuthology.orchestra.run.smithi043.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T12:57:25.879 INFO:teuthology.orchestra.run.smithi043.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T12:57:25.884 INFO:teuthology.orchestra.run.smithi043.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T12:57:26.036 INFO:teuthology.orchestra.run.smithi043.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T12:57:26.036 INFO:teuthology.orchestra.run.smithi043.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T12:57:26.036 INFO:teuthology.orchestra.run.smithi043.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T12:57:26.080 INFO:teuthology.orchestra.run.smithi043.stdout:done 2024-03-30T12:57:26.540 DEBUG:teuthology.orchestra.run.smithi043:> dpkg -s linux-image-generic 2024-03-30T12:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout:Package: linux-image-generic 2024-03-30T12:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout:Status: install ok installed 2024-03-30T12:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout:Priority: optional 2024-03-30T12:57:26.565 INFO:teuthology.orchestra.run.smithi043.stdout:Section: kernel 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Installed-Size: 21 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Maintainer: Ubuntu Kernel Team 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: amd64 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Source: linux-meta 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Version: 5.15.0.101.98 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Recommends: thermald 2024-03-30T12:57:26.566 INFO:teuthology.orchestra.run.smithi043.stdout:Description: Generic Linux kernel image 2024-03-30T12:57:26.567 INFO:teuthology.orchestra.run.smithi043.stdout: This package will always depend on the latest generic kernel image 2024-03-30T12:57:26.568 INFO:teuthology.orchestra.run.smithi043.stdout: available. 2024-03-30T12:57:26.568 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-30T12:57:26.568 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-30T12:57:26.568 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-30T12:57:26.568 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-30T12:57:26.569 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-03-30T12:57:26.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get clean 2024-03-30T12:57:26.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get update 2024-03-30T12:57:26.910 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T12:57:26.916 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-30T12:57:26.936 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-30T12:57:27.027 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-30T12:57:28.314 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-03-30T12:57:28.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-30T12:57:28.392 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-03-30T12:57:28.611 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-03-30T12:57:28.611 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-03-30T12:57:28.791 INFO:teuthology.orchestra.run.smithi043.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-30T12:57:28.791 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T12:57:28.791 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T12:57:28.791 INFO:teuthology.orchestra.run.smithi043.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T12:57:28.791 INFO:teuthology.orchestra.run.smithi043.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T12:57:28.792 INFO:teuthology.orchestra.run.smithi043.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T12:57:28.792 INFO:teuthology.orchestra.run.smithi043.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T12:57:28.792 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T12:57:28.834 INFO:teuthology.orchestra.run.smithi043.stdout:0 upgraded, 0 newly installed, 0 to remove and 311 not upgraded. 2024-03-30T12:57:28.836 DEBUG:teuthology.orchestra.run.smithi043:> dpkg -s linux-image-generic 2024-03-30T12:57:28.851 INFO:teuthology.orchestra.run.smithi043.stdout:Package: linux-image-generic 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Status: install ok installed 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Priority: optional 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Section: kernel 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Installed-Size: 21 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Maintainer: Ubuntu Kernel Team 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: amd64 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Source: linux-meta 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Version: 5.15.0.101.98 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Recommends: thermald 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:Description: Generic Linux kernel image 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout: This package will always depend on the latest generic kernel image 2024-03-30T12:57:28.852 INFO:teuthology.orchestra.run.smithi043.stdout: available. 2024-03-30T12:57:28.853 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-03-30T12:57:28.897 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.ibfD5ZeVES 2024-03-30T12:57:28.898 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub/grub.cfg /tmp/tmp.ibfD5ZeVES 2024-03-30T12:57:28.936 INFO:teuthology.orchestra.run.smithi113.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-30T12:57:28.936 INFO:teuthology.orchestra.run.smithi113.stdout:Sourcing file `/etc/default/grub' 2024-03-30T12:57:28.963 INFO:teuthology.orchestra.run.smithi113.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T12:57:28.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.ibfD5ZeVES 2024-03-30T12:57:28.988 INFO:teuthology.orchestra.run.smithi113.stdout:Generating grub configuration file ... 2024-03-30T12:57:29.101 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.ibfD5ZeVES is 10KB 2024-03-30T12:57:29.115 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.ibfD5ZeVES 2024-03-30T12:57:29.120 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-30T12:57:29.174 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T12:57:29.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-30T12:57:29.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-30T12:57:29.249 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-30T12:57:29.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo update-grub 2024-03-30T12:57:29.306 INFO:teuthology.orchestra.run.smithi113.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-30T12:57:29.332 INFO:teuthology.orchestra.run.smithi113.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:29.698 INFO:teuthology.orchestra.run.smithi113.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T12:57:29.702 INFO:teuthology.orchestra.run.smithi113.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T12:57:29.773 INFO:teuthology.orchestra.run.smithi113.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T12:57:29.779 INFO:teuthology.orchestra.run.smithi113.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T12:57:29.945 INFO:teuthology.orchestra.run.smithi113.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T12:57:29.945 INFO:teuthology.orchestra.run.smithi113.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T12:57:29.945 INFO:teuthology.orchestra.run.smithi113.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T12:57:29.992 INFO:teuthology.orchestra.run.smithi113.stdout:done 2024-03-30T12:57:30.466 DEBUG:teuthology.orchestra.run.smithi113:> dpkg -s linux-image-generic 2024-03-30T12:57:30.488 INFO:teuthology.orchestra.run.smithi113.stdout:Package: linux-image-generic 2024-03-30T12:57:30.488 INFO:teuthology.orchestra.run.smithi113.stdout:Status: install ok installed 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Priority: optional 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Section: kernel 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Installed-Size: 21 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Maintainer: Ubuntu Kernel Team 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Architecture: amd64 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Source: linux-meta 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Version: 5.15.0.101.98 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Recommends: thermald 2024-03-30T12:57:30.489 INFO:teuthology.orchestra.run.smithi113.stdout:Description: Generic Linux kernel image 2024-03-30T12:57:30.490 INFO:teuthology.orchestra.run.smithi113.stdout: This package will always depend on the latest generic kernel image 2024-03-30T12:57:30.490 INFO:teuthology.orchestra.run.smithi113.stdout: available. 2024-03-30T12:57:30.490 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-30T12:57:30.490 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-30T12:57:30.490 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-30T12:57:30.491 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-30T12:57:30.491 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi113.front.sepia.ceph.com, path=None, version=distro) 2024-03-30T12:57:30.491 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get clean 2024-03-30T12:57:30.576 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub' 2024-03-30T12:57:30.577 INFO:teuthology.orchestra.run.smithi043.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T12:57:30.580 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-03-30T12:57:30.655 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get update 2024-03-30T12:57:30.776 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T12:57:30.788 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-30T12:57:30.791 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-30T12:57:30.808 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-30T12:57:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-30T12:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:31.192 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T12:57:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T12:57:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T12:57:31.246 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T12:57:31.380 INFO:teuthology.orchestra.run.smithi043.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T12:57:31.380 INFO:teuthology.orchestra.run.smithi043.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T12:57:31.380 INFO:teuthology.orchestra.run.smithi043.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T12:57:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-03-30T12:57:31.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-03-30T12:57:32.130 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-30T12:57:32.148 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-30T12:57:32.207 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-30T12:57:32.419 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-03-30T12:57:32.419 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-03-30T12:57:32.596 INFO:teuthology.orchestra.run.smithi113.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-30T12:57:32.596 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T12:57:32.596 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T12:57:32.596 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T12:57:32.596 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T12:57:32.597 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T12:57:32.597 INFO:teuthology.orchestra.run.smithi113.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T12:57:32.597 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T12:57:32.639 INFO:teuthology.orchestra.run.smithi113.stdout:0 upgraded, 0 newly installed, 0 to remove and 311 not upgraded. 2024-03-30T12:57:32.641 DEBUG:teuthology.orchestra.run.smithi113:> dpkg -s linux-image-generic 2024-03-30T12:57:32.654 INFO:teuthology.orchestra.run.smithi113.stdout:Package: linux-image-generic 2024-03-30T12:57:32.654 INFO:teuthology.orchestra.run.smithi113.stdout:Status: install ok installed 2024-03-30T12:57:32.654 INFO:teuthology.orchestra.run.smithi113.stdout:Priority: optional 2024-03-30T12:57:32.654 INFO:teuthology.orchestra.run.smithi113.stdout:Section: kernel 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Installed-Size: 21 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Maintainer: Ubuntu Kernel Team 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Architecture: amd64 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Source: linux-meta 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Version: 5.15.0.101.98 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Recommends: thermald 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout:Description: Generic Linux kernel image 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout: This package will always depend on the latest generic kernel image 2024-03-30T12:57:32.655 INFO:teuthology.orchestra.run.smithi113.stdout: available. 2024-03-30T12:57:32.656 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-03-30T12:57:32.701 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.HX7Ljehl5z 2024-03-30T12:57:32.701 DEBUG:teuthology.orchestra.run.smithi113:> sudo cp /boot/grub/grub.cfg /tmp/tmp.HX7Ljehl5z 2024-03-30T12:57:32.757 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.HX7Ljehl5z 2024-03-30T12:57:32.881 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.HX7Ljehl5z is 10KB 2024-03-30T12:57:32.894 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.HX7Ljehl5z 2024-03-30T12:57:32.900 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-30T12:57:32.953 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T12:57:32.954 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-30T12:57:32.954 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-30T12:57:33.036 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-30T12:57:33.036 DEBUG:teuthology.orchestra.run.smithi113:> sudo update-grub 2024-03-30T12:57:34.382 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub' 2024-03-30T12:57:34.383 INFO:teuthology.orchestra.run.smithi113.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-30T12:57:34.393 INFO:teuthology.orchestra.run.smithi113.stderr:Generating grub configuration file ... 2024-03-30T12:57:34.652 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-30T12:57:34.659 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-30T12:57:34.899 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-30T12:57:34.904 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-30T12:57:34.947 INFO:teuthology.orchestra.run.smithi113.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-30T12:57:34.952 INFO:teuthology.orchestra.run.smithi113.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-30T12:57:35.069 INFO:teuthology.orchestra.run.smithi113.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-30T12:57:35.069 INFO:teuthology.orchestra.run.smithi113.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-30T12:57:35.069 INFO:teuthology.orchestra.run.smithi113.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-30T12:57:35.109 INFO:teuthology.orchestra.run.smithi113.stderr:done 2024-03-30T12:57:35.111 DEBUG:teuthology.orchestra.run.smithi113:> sudo shutdown -r now 2024-03-30T12:58:01.478 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-30T12:58:01.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-30T12:58:01.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:58:05.116 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-30T12:58:05.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-30T12:58:05.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:58:39.775 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-03-30T12:58:48.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-30T12:58:48.783 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:58:51.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-03-30T12:59:01.490 DEBUG:teuthology.orchestra.remote:timed out 2024-03-30T12:59:03.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-30T12:59:03.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:10.498 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-30T12:59:10.499 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:18.670 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-03-30T12:59:19.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-30T12:59:19.471 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-03-30T12:59:19.471 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-03-30T12:59:19.520 INFO:teuthology.orchestra.run.smithi043.stdout:5.15.0-101-generic 2024-03-30T12:59:19.520 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-30T12:59:19.520 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T12:59:19.521 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-30T12:59:20.521 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-30T12:59:20.521 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-30T12:59:20.596 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-03-30T12:59:20.625 DEBUG:teuthology.parallel:result is None 2024-03-30T12:59:22.270 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.113 2024-03-30T12:59:37.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-30T12:59:37.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T12:59:37.543 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-03-30T12:59:38.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-30T12:59:38.150 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-101-generic"... 2024-03-30T12:59:38.151 DEBUG:teuthology.orchestra.run.smithi113:> uname -r 2024-03-30T12:59:38.198 INFO:teuthology.orchestra.run.smithi113.stdout:5.15.0-101-generic 2024-03-30T12:59:38.199 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-30T12:59:38.199 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-30T12:59:38.199 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-30T12:59:39.199 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-30T12:59:39.200 DEBUG:teuthology.orchestra.run.smithi113:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-30T12:59:39.264 INFO:teuthology.orchestra.run.smithi113.stdout:ttyS1 2024-03-30T12:59:39.299 DEBUG:teuthology.parallel:result is None 2024-03-30T12:59:39.300 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-30T12:59:39.307 INFO:teuthology.task.internal:Creating test directory... 2024-03-30T12:59:39.308 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T12:59:39.310 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-30T12:59:39.316 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-30T12:59:39.466 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-30T12:59:39.613 INFO:teuthology.task.internal:Creating archive directory... 2024-03-30T12:59:39.613 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T12:59:39.616 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-30T12:59:39.632 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-30T12:59:39.638 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-30T12:59:39.638 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T12:59:39.669 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-30T12:59:39.690 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T12:59:39.690 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T12:59:39.698 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T12:59:39.699 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-30T12:59:39.700 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-30T12:59:39.708 INFO:teuthology.task.internal:Configuring sudo... 2024-03-30T12:59:39.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T12:59:39.749 DEBUG:teuthology.orchestra.run.smithi113:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-30T12:59:39.774 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-30T12:59:39.786 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-30T12:59:39.786 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T12:59:39.805 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-30T12:59:39.826 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T12:59:39.856 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T12:59:39.905 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T12:59:39.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T12:59:39.963 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-30T12:59:39.969 DEBUG:teuthology.orchestra.run.smithi113:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-30T12:59:40.018 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T12:59:40.019 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-30T12:59:40.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-03-30T12:59:40.084 DEBUG:teuthology.orchestra.run.smithi113:> sudo service rsyslog restart 2024-03-30T12:59:40.177 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-30T12:59:40.185 INFO:teuthology.task.internal:Starting timer... 2024-03-30T12:59:40.185 INFO:teuthology.run_tasks:Running task pcp... 2024-03-30T12:59:40.266 INFO:teuthology.run_tasks:Running task selinux... 2024-03-30T12:59:40.415 DEBUG:teuthology.task.selinux:Excluding smithi043: OS 'ubuntu' does not support SELinux 2024-03-30T12:59:40.415 DEBUG:teuthology.task.selinux:Excluding smithi113: OS 'ubuntu' does not support SELinux 2024-03-30T12:59:40.415 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-30T12:59:40.415 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-30T12:59:40.415 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-30T12:59:40.415 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-30T12:59:40.530 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-30T12:59:40.552 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-30T12:59:40.554 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi113.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-30T13:06:43.573 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi113.front.sepia.ceph.com')] 2024-03-30T13:06:43.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-30T13:06:43.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T13:06:43.661 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-03-30T13:06:43.739 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-30T13:06:43.739 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-30T13:06:43.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi113.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-30T13:06:43.834 DEBUG:teuthology.orchestra.run.smithi113:> true 2024-03-30T13:06:43.919 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi113.front.sepia.ceph.com' 2024-03-30T13:06:43.919 INFO:teuthology.run_tasks:Running task clock... 2024-03-30T13:06:43.931 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-30T13:06:43.931 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T13:06:43.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:06:43.934 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-30T13:06:43.934 DEBUG:teuthology.orchestra.run.smithi113:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:06:43.958 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-30T13:06:43.958 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Command line: ntpd -gq 2024-03-30T13:06:43.958 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: ---------------------------------------------------- 2024-03-30T13:06:43.959 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: ntp-4 is maintained by Network Time Foundation, 2024-03-30T13:06:43.959 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-30T13:06:43.959 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: corporation. Support and training for ntp-4 are 2024-03-30T13:06:43.959 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: available at https://www.nwtime.org/support 2024-03-30T13:06:43.959 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: ---------------------------------------------------- 2024-03-30T13:06:43.959 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: proto: precision = 0.065 usec (-24) 2024-03-30T13:06:43.959 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: basedate set to 2022-02-04 2024-03-30T13:06:43.960 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: gps base set to 2022-02-06 (week 2196) 2024-03-30T13:06:43.960 INFO:teuthology.orchestra.run.smithi043.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T13:06:43.960 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T13:06:43.960 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: restrict ::: KOD does nothing without LIMITED. 2024-03-30T13:06:43.960 INFO:teuthology.orchestra.run.smithi043.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-30T13:06:43.962 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Listen and drop on 0 v6wildcard [::]:123 2024-03-30T13:06:43.962 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-30T13:06:43.962 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Listen normally on 2 lo 127.0.0.1:123 2024-03-30T13:06:43.962 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Listen normally on 3 ens1f0 172.21.15.43:123 2024-03-30T13:06:43.962 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Listen normally on 4 lo [::1]:123 2024-03-30T13:06:43.962 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1358%4]:123 2024-03-30T13:06:43.962 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:43 ntpd[17062]: Listening on routing socket on fd #22 for interface updates 2024-03-30T13:06:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-30T13:06:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Command line: ntpd -gq 2024-03-30T13:06:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: ---------------------------------------------------- 2024-03-30T13:06:43.990 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: ntp-4 is maintained by Network Time Foundation, 2024-03-30T13:06:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-30T13:06:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: corporation. Support and training for ntp-4 are 2024-03-30T13:06:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: available at https://www.nwtime.org/support 2024-03-30T13:06:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: ---------------------------------------------------- 2024-03-30T13:06:43.991 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: proto: precision = 0.062 usec (-24) 2024-03-30T13:06:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: basedate set to 2022-02-04 2024-03-30T13:06:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: gps base set to 2022-02-06 (week 2196) 2024-03-30T13:06:43.992 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T13:06:43.992 INFO:teuthology.orchestra.run.smithi113.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-30T13:06:43.992 INFO:teuthology.orchestra.run.smithi113.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-30T13:06:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: restrict ::: KOD does nothing without LIMITED. 2024-03-30T13:06:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Listen and drop on 0 v6wildcard [::]:123 2024-03-30T13:06:43.993 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-30T13:06:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Listen normally on 2 lo 127.0.0.1:123 2024-03-30T13:06:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Listen normally on 3 enp3s0f1 172.21.15.113:123 2024-03-30T13:06:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Listen normally on 4 lo [::1]:123 2024-03-30T13:06:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d7c1%5]:123 2024-03-30T13:06:43.994 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:43 ntpd[17187]: Listening on routing socket on fd #22 for interface updates 2024-03-30T13:06:44.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:44.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:44 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:44.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:44.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:44 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:45 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:45 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:45 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:45 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:45 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:45.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:45 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:46 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:46.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:46.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:46 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:47 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:47 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:47 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:47 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:47 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:47 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:48.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:48.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:48 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:48.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:48 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:49.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:49 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:49.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:49 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:49.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:49 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:49.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:49 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:49.959 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:49 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:49.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:49 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:50.960 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:50.961 INFO:teuthology.orchestra.run.smithi043.stdout:30 Mar 13:06:50 ntpd[17062]: ntpd: time slew -0.000379 s 2024-03-30T13:06:50.961 INFO:teuthology.orchestra.run.smithi043.stdout:ntpd: time slew -0.000379s 2024-03-30T13:06:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-30T13:06:50.962 INFO:teuthology.orchestra.run.smithi043.stderr:30 Mar 13:06:50 ntpd[17062]: can't open /var/log/ntpstats/loopstats.20240330: Permission denied 2024-03-30T13:06:50.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:50.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:50.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:50.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:50.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:50.991 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:50.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-30T13:06:50.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240330: Permission denied 2024-03-30T13:06:50.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-30T13:06:50.992 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240330: Permission denied 2024-03-30T13:06:50.992 INFO:teuthology.orchestra.run.smithi113.stdout:30 Mar 13:06:50 ntpd[17187]: ntpd: time slew -0.000378 s 2024-03-30T13:06:50.992 INFO:teuthology.orchestra.run.smithi113.stdout:ntpd: time slew -0.000378s 2024-03-30T13:06:50.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-30T13:06:50.993 INFO:teuthology.orchestra.run.smithi113.stderr:30 Mar 13:06:50 ntpd[17187]: can't open /var/log/ntpstats/loopstats.20240330: Permission denied 2024-03-30T13:06:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T13:06:51.006 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================== 2024-03-30T13:06:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.006 INFO:teuthology.orchestra.run.smithi043.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.048 INFO:teuthology.orchestra.run.smithi113.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T13:06:51.048 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================== 2024-03-30T13:06:51.048 INFO:teuthology.orchestra.run.smithi113.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.049 INFO:teuthology.orchestra.run.smithi113.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.049 INFO:teuthology.orchestra.run.smithi113.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.049 INFO:teuthology.orchestra.run.smithi113.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:06:51.049 INFO:teuthology.run_tasks:Running task install... 2024-03-30T13:06:51.069 DEBUG:teuthology.task.install:project ceph 2024-03-30T13:06:51.069 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}} 2024-03-30T13:06:51.069 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'} 2024-03-30T13:06:51.069 INFO:teuthology.task.install:Using flavor: default 2024-03-30T13:06:51.076 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-30T13:06:51.076 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-30T13:06:51.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-key list | grep Ceph 2024-03-30T13:06:51.078 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-key list | grep Ceph 2024-03-30T13:06:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-30T13:06:51.152 INFO:teuthology.orchestra.run.smithi113.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-30T13:06:51.176 INFO:teuthology.orchestra.run.smithi043.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-30T13:06:51.177 INFO:teuthology.orchestra.run.smithi043.stdout:uid [ unknown] Ceph.com (release key) 2024-03-30T13:06:51.178 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-30T13:06:51.178 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-30T13:06:51.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:06:51.188 INFO:teuthology.orchestra.run.smithi113.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-30T13:06:51.188 INFO:teuthology.orchestra.run.smithi113.stdout:uid [ unknown] Ceph.com (release key) 2024-03-30T13:06:51.188 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-30T13:06:51.188 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-30T13:06:51.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:06:51.338 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default/ 2024-03-30T13:06:51.338 INFO:teuthology.task.install.deb:Package version is 19.0.0-2519-gac171039-1jammy 2024-03-30T13:06:51.406 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default/ 2024-03-30T13:06:51.406 INFO:teuthology.task.install.deb:Package version is 19.0.0-2519-gac171039-1jammy 2024-03-30T13:06:51.464 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:06:51.464 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-30T13:06:51.479 DEBUG:teuthology.orchestra.run.smithi113:> sudo apt-get update 2024-03-30T13:06:51.532 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:06:51.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-30T13:06:51.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo apt-get update 2024-03-30T13:06:51.662 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T13:06:51.663 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-30T13:06:51.673 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-30T13:06:51.693 INFO:teuthology.orchestra.run.smithi113.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-30T13:06:51.729 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-30T13:06:51.738 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-30T13:06:51.758 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-30T13:06:51.768 INFO:teuthology.orchestra.run.smithi113.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy InRelease 2024-03-30T13:06:51.777 INFO:teuthology.orchestra.run.smithi043.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy InRelease 2024-03-30T13:06:51.798 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-30T13:06:51.805 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-30T13:06:51.820 INFO:teuthology.orchestra.run.smithi043.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-30T13:06:51.841 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,415 B] 2024-03-30T13:06:51.841 INFO:teuthology.orchestra.run.smithi113.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-30T13:06:51.847 INFO:teuthology.orchestra.run.smithi043.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-30T13:06:51.868 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-30T13:06:51.880 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-03-30T13:06:51.921 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,415 B] 2024-03-30T13:06:52.010 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 29.2 kB in 0s (84.4 kB/s) 2024-03-30T13:06:52.092 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 29.2 kB in 0s (81.2 kB/s) 2024-03-30T13:06:52.963 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-30T13:06:52.979 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2519-gac171039-1jammy cephadm=19.0.0-2519-gac171039-1jammy ceph-mds=19.0.0-2519-gac171039-1jammy ceph-mgr=19.0.0-2519-gac171039-1jammy ceph-common=19.0.0-2519-gac171039-1jammy ceph-fuse=19.0.0-2519-gac171039-1jammy ceph-test=19.0.0-2519-gac171039-1jammy ceph-volume=19.0.0-2519-gac171039-1jammy radosgw=19.0.0-2519-gac171039-1jammy python3-rados=19.0.0-2519-gac171039-1jammy python3-rgw=19.0.0-2519-gac171039-1jammy python3-cephfs=19.0.0-2519-gac171039-1jammy python3-rbd=19.0.0-2519-gac171039-1jammy libcephfs2=19.0.0-2519-gac171039-1jammy libcephfs-dev=19.0.0-2519-gac171039-1jammy librados2=19.0.0-2519-gac171039-1jammy librbd1=19.0.0-2519-gac171039-1jammy rbd-fuse=19.0.0-2519-gac171039-1jammy python3-cephfs=19.0.0-2519-gac171039-1jammy cephfs-shell=19.0.0-2519-gac171039-1jammy cephfs-top=19.0.0-2519-gac171039-1jammy cephfs-mirror=19.0.0-2519-gac171039-1jammy 2024-03-30T13:06:53.042 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-30T13:06:53.048 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-03-30T13:06:53.065 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2519-gac171039-1jammy cephadm=19.0.0-2519-gac171039-1jammy ceph-mds=19.0.0-2519-gac171039-1jammy ceph-mgr=19.0.0-2519-gac171039-1jammy ceph-common=19.0.0-2519-gac171039-1jammy ceph-fuse=19.0.0-2519-gac171039-1jammy ceph-test=19.0.0-2519-gac171039-1jammy ceph-volume=19.0.0-2519-gac171039-1jammy radosgw=19.0.0-2519-gac171039-1jammy python3-rados=19.0.0-2519-gac171039-1jammy python3-rgw=19.0.0-2519-gac171039-1jammy python3-cephfs=19.0.0-2519-gac171039-1jammy python3-rbd=19.0.0-2519-gac171039-1jammy libcephfs2=19.0.0-2519-gac171039-1jammy libcephfs-dev=19.0.0-2519-gac171039-1jammy librados2=19.0.0-2519-gac171039-1jammy librbd1=19.0.0-2519-gac171039-1jammy rbd-fuse=19.0.0-2519-gac171039-1jammy python3-cephfs=19.0.0-2519-gac171039-1jammy cephfs-shell=19.0.0-2519-gac171039-1jammy cephfs-top=19.0.0-2519-gac171039-1jammy cephfs-mirror=19.0.0-2519-gac171039-1jammy 2024-03-30T13:06:53.129 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-03-30T13:06:53.238 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-03-30T13:06:53.239 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-03-30T13:06:53.329 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-03-30T13:06:53.330 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-03-30T13:06:53.418 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T13:06:53.419 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T13:06:53.419 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T13:06:53.419 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T13:06:53.419 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T13:06:53.419 INFO:teuthology.orchestra.run.smithi113.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T13:06:53.419 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout:The following additional packages will be installed: 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-30T13:06:53.421 INFO:teuthology.orchestra.run.smithi113.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-30T13:06:53.422 INFO:teuthology.orchestra.run.smithi113.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout:Suggested packages: 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-30T13:06:53.423 INFO:teuthology.orchestra.run.smithi113.stdout:Recommended packages: 2024-03-30T13:06:53.424 INFO:teuthology.orchestra.run.smithi113.stdout: btrfs-tools 2024-03-30T13:06:53.520 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T13:06:53.520 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T13:06:53.521 INFO:teuthology.orchestra.run.smithi043.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T13:06:53.521 INFO:teuthology.orchestra.run.smithi043.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T13:06:53.521 INFO:teuthology.orchestra.run.smithi043.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-30T13:06:53.522 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-30T13:06:53.523 INFO:teuthology.orchestra.run.smithi043.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-30T13:06:53.523 INFO:teuthology.orchestra.run.smithi043.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-30T13:06:53.523 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-30T13:06:53.523 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-30T13:06:53.524 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-30T13:06:53.525 INFO:teuthology.orchestra.run.smithi043.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-30T13:06:53.526 INFO:teuthology.orchestra.run.smithi043.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-30T13:06:53.526 INFO:teuthology.orchestra.run.smithi043.stdout:Recommended packages: 2024-03-30T13:06:53.526 INFO:teuthology.orchestra.run.smithi043.stdout: btrfs-tools 2024-03-30T13:06:53.526 INFO:teuthology.orchestra.run.smithi113.stdout:The following NEW packages will be installed: 2024-03-30T13:06:53.526 INFO:teuthology.orchestra.run.smithi113.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-30T13:06:53.527 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-30T13:06:53.527 INFO:teuthology.orchestra.run.smithi113.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-30T13:06:53.528 INFO:teuthology.orchestra.run.smithi113.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-30T13:06:53.529 INFO:teuthology.orchestra.run.smithi113.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-30T13:06:53.530 INFO:teuthology.orchestra.run.smithi113.stdout: socat unzip xmlstarlet zip 2024-03-30T13:06:53.603 INFO:teuthology.orchestra.run.smithi113.stdout:0 upgraded, 124 newly installed, 0 to remove and 298 not upgraded. 2024-03-30T13:06:53.603 INFO:teuthology.orchestra.run.smithi113.stdout:Need to get 179 MB of archives. 2024-03-30T13:06:53.604 INFO:teuthology.orchestra.run.smithi113.stdout:After this operation, 821 MB of additional disk space will be used. 2024-03-30T13:06:53.604 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-30T13:06:53.632 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-30T13:06:53.633 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-03-30T13:06:53.633 INFO:teuthology.orchestra.run.smithi043.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-30T13:06:53.633 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-30T13:06:53.633 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-30T13:06:53.633 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-30T13:06:53.635 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-30T13:06:53.636 INFO:teuthology.orchestra.run.smithi043.stdout: socat unzip xmlstarlet zip 2024-03-30T13:06:53.647 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-30T13:06:53.711 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-30T13:06:53.743 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-30T13:06:53.747 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-30T13:06:53.767 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-30T13:06:53.779 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2519-gac171039-1jammy [3,556 kB] 2024-03-30T13:06:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:0 upgraded, 124 newly installed, 0 to remove and 298 not upgraded. 2024-03-30T13:06:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 179 MB of archives. 2024-03-30T13:06:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 821 MB of additional disk space will be used. 2024-03-30T13:06:53.825 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-30T13:06:53.842 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-30T13:06:53.847 INFO:teuthology.orchestra.run.smithi113.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-30T13:06:53.862 INFO:teuthology.orchestra.run.smithi113.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-30T13:06:53.867 INFO:teuthology.orchestra.run.smithi113.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-30T13:06:53.867 INFO:teuthology.orchestra.run.smithi113.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-30T13:06:53.879 INFO:teuthology.orchestra.run.smithi113.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-30T13:06:53.882 INFO:teuthology.orchestra.run.smithi113.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-30T13:06:53.885 INFO:teuthology.orchestra.run.smithi113.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-30T13:06:53.887 INFO:teuthology.orchestra.run.smithi113.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-30T13:06:53.895 INFO:teuthology.orchestra.run.smithi113.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-30T13:06:53.908 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-30T13:06:53.918 INFO:teuthology.orchestra.run.smithi113.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-30T13:06:53.919 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2519-gac171039-1jammy [3,556 kB] 2024-03-30T13:06:53.921 INFO:teuthology.orchestra.run.smithi113.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-30T13:06:53.923 INFO:teuthology.orchestra.run.smithi113.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-30T13:06:53.924 INFO:teuthology.orchestra.run.smithi113.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-30T13:06:53.924 INFO:teuthology.orchestra.run.smithi113.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-30T13:06:53.925 INFO:teuthology.orchestra.run.smithi113.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-30T13:06:53.926 INFO:teuthology.orchestra.run.smithi113.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-30T13:06:53.926 INFO:teuthology.orchestra.run.smithi113.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-30T13:06:53.927 INFO:teuthology.orchestra.run.smithi113.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-30T13:06:53.952 INFO:teuthology.orchestra.run.smithi113.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-30T13:06:53.953 INFO:teuthology.orchestra.run.smithi113.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-30T13:06:53.953 INFO:teuthology.orchestra.run.smithi113.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-30T13:06:53.953 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-30T13:06:53.956 INFO:teuthology.orchestra.run.smithi113.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-30T13:06:53.957 INFO:teuthology.orchestra.run.smithi113.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-30T13:06:53.982 INFO:teuthology.orchestra.run.smithi113.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-30T13:06:53.982 INFO:teuthology.orchestra.run.smithi113.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-30T13:06:53.983 INFO:teuthology.orchestra.run.smithi113.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-30T13:06:53.984 INFO:teuthology.orchestra.run.smithi113.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-30T13:06:53.984 INFO:teuthology.orchestra.run.smithi113.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-30T13:06:54.006 INFO:teuthology.orchestra.run.smithi113.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-30T13:06:54.007 INFO:teuthology.orchestra.run.smithi113.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-30T13:06:54.007 INFO:teuthology.orchestra.run.smithi113.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-30T13:06:54.014 INFO:teuthology.orchestra.run.smithi113.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-30T13:06:54.015 INFO:teuthology.orchestra.run.smithi113.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-30T13:06:54.036 INFO:teuthology.orchestra.run.smithi113.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-30T13:06:54.037 INFO:teuthology.orchestra.run.smithi113.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-30T13:06:54.039 INFO:teuthology.orchestra.run.smithi113.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-30T13:06:54.043 INFO:teuthology.orchestra.run.smithi113.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-30T13:06:54.044 INFO:teuthology.orchestra.run.smithi113.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-30T13:06:54.064 INFO:teuthology.orchestra.run.smithi113.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-30T13:06:54.065 INFO:teuthology.orchestra.run.smithi113.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-30T13:06:54.070 INFO:teuthology.orchestra.run.smithi113.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-30T13:06:54.071 INFO:teuthology.orchestra.run.smithi113.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-30T13:06:54.073 INFO:teuthology.orchestra.run.smithi113.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-30T13:06:54.091 INFO:teuthology.orchestra.run.smithi113.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-30T13:06:54.156 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-30T13:06:54.178 INFO:teuthology.orchestra.run.smithi113.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-30T13:06:54.179 INFO:teuthology.orchestra.run.smithi113.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-30T13:06:54.182 INFO:teuthology.orchestra.run.smithi113.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-30T13:06:54.182 INFO:teuthology.orchestra.run.smithi113.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-30T13:06:54.184 INFO:teuthology.orchestra.run.smithi113.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-30T13:06:54.216 INFO:teuthology.orchestra.run.smithi113.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-30T13:06:54.220 INFO:teuthology.orchestra.run.smithi113.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-30T13:06:54.220 INFO:teuthology.orchestra.run.smithi113.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-30T13:06:54.250 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-30T13:06:54.254 INFO:teuthology.orchestra.run.smithi113.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-30T13:06:54.254 INFO:teuthology.orchestra.run.smithi113.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-30T13:06:54.264 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-30T13:06:54.271 INFO:teuthology.orchestra.run.smithi113.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-30T13:06:54.272 INFO:teuthology.orchestra.run.smithi113.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-30T13:06:54.272 INFO:teuthology.orchestra.run.smithi113.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-30T13:06:54.273 INFO:teuthology.orchestra.run.smithi113.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-30T13:06:54.274 INFO:teuthology.orchestra.run.smithi113.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-30T13:06:54.274 INFO:teuthology.orchestra.run.smithi113.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-30T13:06:54.279 INFO:teuthology.orchestra.run.smithi113.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-30T13:06:54.298 INFO:teuthology.orchestra.run.smithi113.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-30T13:06:54.299 INFO:teuthology.orchestra.run.smithi113.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-30T13:06:54.300 INFO:teuthology.orchestra.run.smithi113.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-30T13:06:54.315 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-30T13:06:54.326 INFO:teuthology.orchestra.run.smithi113.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-30T13:06:54.328 INFO:teuthology.orchestra.run.smithi113.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-30T13:06:54.329 INFO:teuthology.orchestra.run.smithi113.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-30T13:06:54.330 INFO:teuthology.orchestra.run.smithi113.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-30T13:06:54.332 INFO:teuthology.orchestra.run.smithi113.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-30T13:06:54.339 INFO:teuthology.orchestra.run.smithi113.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-30T13:06:54.340 INFO:teuthology.orchestra.run.smithi113.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-30T13:06:54.360 INFO:teuthology.orchestra.run.smithi113.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-30T13:06:54.360 INFO:teuthology.orchestra.run.smithi113.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-30T13:06:54.361 INFO:teuthology.orchestra.run.smithi113.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-30T13:06:54.386 INFO:teuthology.orchestra.run.smithi113.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-30T13:06:54.389 INFO:teuthology.orchestra.run.smithi113.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-30T13:06:54.390 INFO:teuthology.orchestra.run.smithi113.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-30T13:06:54.402 INFO:teuthology.orchestra.run.smithi113.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-30T13:06:54.402 INFO:teuthology.orchestra.run.smithi113.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-30T13:06:54.403 INFO:teuthology.orchestra.run.smithi113.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-30T13:06:54.406 INFO:teuthology.orchestra.run.smithi113.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-30T13:06:54.419 INFO:teuthology.orchestra.run.smithi113.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-30T13:06:54.489 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-30T13:06:54.499 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-30T13:06:54.526 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-30T13:06:54.534 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-30T13:06:54.699 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-30T13:06:54.943 INFO:teuthology.orchestra.run.smithi113.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2519-gac171039-1jammy [3,226 kB] 2024-03-30T13:06:55.273 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-30T13:06:55.287 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-30T13:06:55.303 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-30T13:06:55.312 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-30T13:06:55.313 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-30T13:06:55.329 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-30T13:06:55.367 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-30T13:06:55.379 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-30T13:06:55.383 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-30T13:06:55.384 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-30T13:06:55.404 INFO:teuthology.orchestra.run.smithi113.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2519-gac171039-1jammy [712 kB] 2024-03-30T13:06:55.434 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2519-gac171039-1jammy [3,226 kB] 2024-03-30T13:06:55.525 INFO:teuthology.orchestra.run.smithi113.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2519-gac171039-1jammy [356 kB] 2024-03-30T13:06:55.554 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-30T13:06:55.576 INFO:teuthology.orchestra.run.smithi113.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2519-gac171039-1jammy [32.8 kB] 2024-03-30T13:06:55.578 INFO:teuthology.orchestra.run.smithi113.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2519-gac171039-1jammy [180 kB] 2024-03-30T13:06:55.591 INFO:teuthology.orchestra.run.smithi113.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2519-gac171039-1jammy [66.7 kB] 2024-03-30T13:06:55.593 INFO:teuthology.orchestra.run.smithi113.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2519-gac171039-1jammy [330 kB] 2024-03-30T13:06:55.637 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-30T13:06:55.648 INFO:teuthology.orchestra.run.smithi113.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2519-gac171039-1jammy [6,660 kB] 2024-03-30T13:06:55.661 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-30T13:06:55.719 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-30T13:06:55.762 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-30T13:06:55.784 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-30T13:06:55.812 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-30T13:06:55.996 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-30T13:06:56.012 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-30T13:06:56.019 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-30T13:06:56.024 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-30T13:06:56.132 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2519-gac171039-1jammy [712 kB] 2024-03-30T13:06:56.202 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2519-gac171039-1jammy [356 kB] 2024-03-30T13:06:56.209 INFO:teuthology.orchestra.run.smithi043.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-30T13:06:56.251 INFO:teuthology.orchestra.run.smithi043.stdout:Get:39 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2519-gac171039-1jammy [32.8 kB] 2024-03-30T13:06:56.252 INFO:teuthology.orchestra.run.smithi043.stdout:Get:40 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2519-gac171039-1jammy [180 kB] 2024-03-30T13:06:56.292 INFO:teuthology.orchestra.run.smithi043.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-30T13:06:56.301 INFO:teuthology.orchestra.run.smithi043.stdout:Get:42 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2519-gac171039-1jammy [66.7 kB] 2024-03-30T13:06:56.302 INFO:teuthology.orchestra.run.smithi043.stdout:Get:43 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2519-gac171039-1jammy [330 kB] 2024-03-30T13:06:56.306 INFO:teuthology.orchestra.run.smithi043.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-30T13:06:56.358 INFO:teuthology.orchestra.run.smithi043.stdout:Get:45 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2519-gac171039-1jammy [6,660 kB] 2024-03-30T13:06:56.371 INFO:teuthology.orchestra.run.smithi113.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2519-gac171039-1jammy [112 kB] 2024-03-30T13:06:56.376 INFO:teuthology.orchestra.run.smithi113.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2519-gac171039-1jammy [466 kB] 2024-03-30T13:06:56.423 INFO:teuthology.orchestra.run.smithi113.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2519-gac171039-1jammy [25.4 MB] 2024-03-30T13:06:56.485 INFO:teuthology.orchestra.run.smithi043.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-30T13:06:56.531 INFO:teuthology.orchestra.run.smithi043.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-30T13:06:56.543 INFO:teuthology.orchestra.run.smithi043.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-30T13:06:56.705 INFO:teuthology.orchestra.run.smithi043.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-30T13:06:56.707 INFO:teuthology.orchestra.run.smithi043.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-30T13:06:56.711 INFO:teuthology.orchestra.run.smithi043.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-30T13:06:56.716 INFO:teuthology.orchestra.run.smithi043.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-30T13:06:56.731 INFO:teuthology.orchestra.run.smithi043.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-30T13:06:56.916 INFO:teuthology.orchestra.run.smithi043.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-30T13:06:57.174 INFO:teuthology.orchestra.run.smithi043.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2519-gac171039-1jammy [112 kB] 2024-03-30T13:06:57.179 INFO:teuthology.orchestra.run.smithi043.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-30T13:06:57.180 INFO:teuthology.orchestra.run.smithi043.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2519-gac171039-1jammy [466 kB] 2024-03-30T13:06:57.254 INFO:teuthology.orchestra.run.smithi043.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2519-gac171039-1jammy [25.4 MB] 2024-03-30T13:06:57.296 INFO:teuthology.orchestra.run.smithi043.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-30T13:06:57.339 INFO:teuthology.orchestra.run.smithi043.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-30T13:06:57.422 INFO:teuthology.orchestra.run.smithi043.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-30T13:06:57.434 INFO:teuthology.orchestra.run.smithi043.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-30T13:06:57.447 INFO:teuthology.orchestra.run.smithi043.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-30T13:06:57.487 INFO:teuthology.orchestra.run.smithi043.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-30T13:06:57.752 INFO:teuthology.orchestra.run.smithi043.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-30T13:06:57.754 INFO:teuthology.orchestra.run.smithi043.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-30T13:06:57.757 INFO:teuthology.orchestra.run.smithi043.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-30T13:06:57.922 INFO:teuthology.orchestra.run.smithi043.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-30T13:06:58.246 INFO:teuthology.orchestra.run.smithi043.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-30T13:06:58.656 INFO:teuthology.orchestra.run.smithi043.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-30T13:06:58.666 INFO:teuthology.orchestra.run.smithi043.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-30T13:06:58.667 INFO:teuthology.orchestra.run.smithi043.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-30T13:06:58.736 INFO:teuthology.orchestra.run.smithi043.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-30T13:06:58.737 INFO:teuthology.orchestra.run.smithi043.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-30T13:06:58.739 INFO:teuthology.orchestra.run.smithi043.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-30T13:06:58.740 INFO:teuthology.orchestra.run.smithi043.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-30T13:06:58.741 INFO:teuthology.orchestra.run.smithi043.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-30T13:06:58.742 INFO:teuthology.orchestra.run.smithi043.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-30T13:06:58.908 INFO:teuthology.orchestra.run.smithi043.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-30T13:06:59.118 INFO:teuthology.orchestra.run.smithi043.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-30T13:06:59.433 INFO:teuthology.orchestra.run.smithi043.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-30T13:06:59.476 INFO:teuthology.orchestra.run.smithi043.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-30T13:06:59.504 INFO:teuthology.orchestra.run.smithi043.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-30T13:06:59.514 INFO:teuthology.orchestra.run.smithi043.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-30T13:06:59.560 INFO:teuthology.orchestra.run.smithi043.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-30T13:06:59.585 INFO:teuthology.orchestra.run.smithi043.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-30T13:06:59.587 INFO:teuthology.orchestra.run.smithi043.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-30T13:06:59.593 INFO:teuthology.orchestra.run.smithi043.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-30T13:06:59.596 INFO:teuthology.orchestra.run.smithi043.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-30T13:06:59.793 INFO:teuthology.orchestra.run.smithi043.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-30T13:07:00.043 INFO:teuthology.orchestra.run.smithi043.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-30T13:07:00.259 INFO:teuthology.orchestra.run.smithi113.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2519-gac171039-1jammy [5,086 kB] 2024-03-30T13:07:00.282 INFO:teuthology.orchestra.run.smithi043.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-30T13:07:00.285 INFO:teuthology.orchestra.run.smithi043.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-30T13:07:00.304 INFO:teuthology.orchestra.run.smithi043.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-30T13:07:00.309 INFO:teuthology.orchestra.run.smithi043.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-30T13:07:00.345 INFO:teuthology.orchestra.run.smithi043.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-30T13:07:00.358 INFO:teuthology.orchestra.run.smithi043.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-30T13:07:00.484 INFO:teuthology.orchestra.run.smithi043.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-30T13:07:00.485 INFO:teuthology.orchestra.run.smithi043.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-30T13:07:00.514 INFO:teuthology.orchestra.run.smithi043.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-30T13:07:00.517 INFO:teuthology.orchestra.run.smithi043.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2519-gac171039-1jammy [5,086 kB] 2024-03-30T13:07:00.683 INFO:teuthology.orchestra.run.smithi043.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-30T13:07:00.773 INFO:teuthology.orchestra.run.smithi113.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2519-gac171039-1jammy [244 kB] 2024-03-30T13:07:00.801 INFO:teuthology.orchestra.run.smithi113.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2519-gac171039-1jammy [124 kB] 2024-03-30T13:07:00.830 INFO:teuthology.orchestra.run.smithi113.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2519-gac171039-1jammy [1,526 kB] 2024-03-30T13:07:00.949 INFO:teuthology.orchestra.run.smithi043.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-30T13:07:00.963 INFO:teuthology.orchestra.run.smithi043.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2519-gac171039-1jammy [244 kB] 2024-03-30T13:07:00.980 INFO:teuthology.orchestra.run.smithi043.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2519-gac171039-1jammy [124 kB] 2024-03-30T13:07:00.993 INFO:teuthology.orchestra.run.smithi113.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2519-gac171039-1jammy [6,259 kB] 2024-03-30T13:07:01.000 INFO:teuthology.orchestra.run.smithi043.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2519-gac171039-1jammy [1,526 kB] 2024-03-30T13:07:01.140 INFO:teuthology.orchestra.run.smithi043.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2519-gac171039-1jammy [6,259 kB] 2024-03-30T13:07:01.748 INFO:teuthology.orchestra.run.smithi113.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2519-gac171039-1jammy [22.8 MB] 2024-03-30T13:07:01.815 INFO:teuthology.orchestra.run.smithi043.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2519-gac171039-1jammy [22.8 MB] 2024-03-30T13:07:06.269 INFO:teuthology.orchestra.run.smithi043.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2519-gac171039-1jammy [14.0 kB] 2024-03-30T13:07:06.270 INFO:teuthology.orchestra.run.smithi043.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2519-gac171039-1jammy [870 kB] 2024-03-30T13:07:06.347 INFO:teuthology.orchestra.run.smithi043.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2519-gac171039-1jammy [2,451 kB] 2024-03-30T13:07:06.597 INFO:teuthology.orchestra.run.smithi113.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2519-gac171039-1jammy [14.0 kB] 2024-03-30T13:07:06.597 INFO:teuthology.orchestra.run.smithi113.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2519-gac171039-1jammy [870 kB] 2024-03-30T13:07:06.727 INFO:teuthology.orchestra.run.smithi113.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2519-gac171039-1jammy [2,451 kB] 2024-03-30T13:07:06.771 INFO:teuthology.orchestra.run.smithi043.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2519-gac171039-1jammy [790 kB] 2024-03-30T13:07:06.884 INFO:teuthology.orchestra.run.smithi043.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2519-gac171039-1jammy [147 kB] 2024-03-30T13:07:06.919 INFO:teuthology.orchestra.run.smithi043.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2519-gac171039-1jammy [2,242 kB] 2024-03-30T13:07:07.282 INFO:teuthology.orchestra.run.smithi113.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2519-gac171039-1jammy [790 kB] 2024-03-30T13:07:07.530 INFO:teuthology.orchestra.run.smithi043.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2519-gac171039-1jammy [8,625 kB] 2024-03-30T13:07:07.533 INFO:teuthology.orchestra.run.smithi113.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2519-gac171039-1jammy [147 kB] 2024-03-30T13:07:07.536 INFO:teuthology.orchestra.run.smithi113.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2519-gac171039-1jammy [2,242 kB] 2024-03-30T13:07:08.240 INFO:teuthology.orchestra.run.smithi113.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2519-gac171039-1jammy [8,625 kB] 2024-03-30T13:07:09.296 INFO:teuthology.orchestra.run.smithi043.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2519-gac171039-1jammy [14.2 kB] 2024-03-30T13:07:09.296 INFO:teuthology.orchestra.run.smithi043.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2519-gac171039-1jammy [50.3 MB] 2024-03-30T13:07:09.872 INFO:teuthology.orchestra.run.smithi113.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2519-gac171039-1jammy [14.2 kB] 2024-03-30T13:07:09.872 INFO:teuthology.orchestra.run.smithi113.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2519-gac171039-1jammy [50.3 MB] 2024-03-30T13:07:19.197 INFO:teuthology.orchestra.run.smithi043.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2519-gac171039-1jammy [121 kB] 2024-03-30T13:07:19.199 INFO:teuthology.orchestra.run.smithi043.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2519-gac171039-1jammy [229 kB] 2024-03-30T13:07:19.204 INFO:teuthology.orchestra.run.smithi043.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2519-gac171039-1jammy [34.0 kB] 2024-03-30T13:07:19.205 INFO:teuthology.orchestra.run.smithi043.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2519-gac171039-1jammy [23.4 kB] 2024-03-30T13:07:19.206 INFO:teuthology.orchestra.run.smithi043.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2519-gac171039-1jammy [38.9 kB] 2024-03-30T13:07:19.207 INFO:teuthology.orchestra.run.smithi043.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2519-gac171039-1jammy [13.2 MB] 2024-03-30T13:07:20.193 INFO:teuthology.orchestra.run.smithi113.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2519-gac171039-1jammy [121 kB] 2024-03-30T13:07:20.196 INFO:teuthology.orchestra.run.smithi113.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2519-gac171039-1jammy [229 kB] 2024-03-30T13:07:20.200 INFO:teuthology.orchestra.run.smithi113.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2519-gac171039-1jammy [34.0 kB] 2024-03-30T13:07:20.201 INFO:teuthology.orchestra.run.smithi113.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2519-gac171039-1jammy [23.4 kB] 2024-03-30T13:07:20.202 INFO:teuthology.orchestra.run.smithi113.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2519-gac171039-1jammy [38.9 kB] 2024-03-30T13:07:20.203 INFO:teuthology.orchestra.run.smithi113.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2519-gac171039-1jammy [13.2 MB] 2024-03-30T13:07:23.865 INFO:teuthology.orchestra.run.smithi043.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2519-gac171039-1jammy [91.2 kB] 2024-03-30T13:07:24.520 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 179 MB in 30s (5,918 kB/s) 2024-03-30T13:07:24.635 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-30T13:07:24.733 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-30T13:07:24.739 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-30T13:07:24.775 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-30T13:07:25.044 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-30T13:07:25.060 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-30T13:07:25.094 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T13:07:25.414 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cmd2. 2024-03-30T13:07:25.440 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-30T13:07:25.480 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-30T13:07:25.833 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-30T13:07:25.860 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-30T13:07:25.942 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T13:07:26.312 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-30T13:07:26.339 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-30T13:07:26.379 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T13:07:26.682 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-30T13:07:26.708 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-30T13:07:26.890 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T13:07:28.037 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-30T13:07:28.063 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T13:07:28.103 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:07:28.242 INFO:teuthology.orchestra.run.smithi113.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing1-main-20240328-072412/ac1710396f9f7c389fa4b407711bc5157a8c0bf7/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2519-gac171039-1jammy [91.2 kB] 2024-03-30T13:07:28.522 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-30T13:07:28.549 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T13:07:28.589 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:07:28.858 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 179 MB in 35s (5,156 kB/s) 2024-03-30T13:07:28.934 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-30T13:07:28.961 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T13:07:29.000 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:07:29.009 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-30T13:07:29.108 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-30T13:07:29.114 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-30T13:07:29.157 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-30T13:07:29.412 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-30T13:07:29.426 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-30T13:07:29.428 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-30T13:07:29.452 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-30T13:07:29.461 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T13:07:29.492 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T13:07:29.765 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librados2. 2024-03-30T13:07:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../010-librados2_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:29.803 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cmd2. 2024-03-30T13:07:29.830 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-30T13:07:29.856 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librados2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:29.870 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-30T13:07:30.232 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-30T13:07:30.258 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-30T13:07:30.325 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librbd1. 2024-03-30T13:07:30.340 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T13:07:30.353 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../011-librbd1_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:30.392 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librbd1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:30.694 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-30T13:07:30.722 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-30T13:07:30.760 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T13:07:30.820 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libcephfs2. 2024-03-30T13:07:30.847 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:30.878 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libcephfs2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:31.046 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-30T13:07:31.073 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-30T13:07:31.113 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T13:07:31.114 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rados. 2024-03-30T13:07:31.130 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:31.164 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rados (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:31.467 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-30T13:07:31.494 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:31.533 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-ceph-argparse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:31.575 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-30T13:07:31.601 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T13:07:31.650 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:07:31.786 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cephfs. 2024-03-30T13:07:31.813 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:31.852 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cephfs (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:32.103 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-30T13:07:32.129 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T13:07:32.139 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-30T13:07:32.166 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:32.170 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:07:32.197 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:32.483 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-prettytable. 2024-03-30T13:07:32.499 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-30T13:07:32.533 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-30T13:07:32.665 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-30T13:07:32.691 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-30T13:07:32.731 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:07:32.936 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rbd. 2024-03-30T13:07:32.963 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:33.003 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rbd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:33.101 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-30T13:07:33.127 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-30T13:07:33.168 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T13:07:33.356 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-30T13:07:33.382 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-30T13:07:33.422 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T13:07:33.487 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librados2. 2024-03-30T13:07:33.513 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../010-librados2_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:33.554 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librados2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:33.716 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-30T13:07:33.744 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-30T13:07:33.783 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-30T13:07:34.007 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librbd1. 2024-03-30T13:07:34.033 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../011-librbd1_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:34.073 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librbd1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:34.094 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-30T13:07:34.122 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-30T13:07:34.161 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-30T13:07:34.481 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua5.1. 2024-03-30T13:07:34.497 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-30T13:07:34.531 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-30T13:07:34.534 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libcephfs2. 2024-03-30T13:07:34.561 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:34.601 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libcephfs2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:34.834 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua-any. 2024-03-30T13:07:34.861 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-30T13:07:34.887 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rados. 2024-03-30T13:07:34.900 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-30T13:07:34.902 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:34.962 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rados (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:35.145 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package zip. 2024-03-30T13:07:35.161 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-30T13:07:35.211 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking zip (3.0-12build2) ... 2024-03-30T13:07:35.265 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-30T13:07:35.291 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:35.323 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-ceph-argparse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:35.515 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package unzip. 2024-03-30T13:07:35.542 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-30T13:07:35.576 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cephfs. 2024-03-30T13:07:35.581 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-30T13:07:35.602 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:35.642 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cephfs (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:35.910 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package luarocks. 2024-03-30T13:07:35.935 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-30T13:07:35.953 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-30T13:07:35.968 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-30T13:07:35.979 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:36.020 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:36.334 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package librgw2. 2024-03-30T13:07:36.348 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-prettytable. 2024-03-30T13:07:36.362 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../027-librgw2_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:36.374 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-30T13:07:36.401 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking librgw2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:36.423 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-30T13:07:36.675 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rbd. 2024-03-30T13:07:36.691 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:36.725 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rbd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:36.953 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rgw. 2024-03-30T13:07:36.970 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:37.012 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:37.062 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-30T13:07:37.088 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-30T13:07:37.128 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T13:07:37.323 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-30T13:07:37.351 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-30T13:07:37.390 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T13:07:37.415 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-30T13:07:37.441 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-30T13:07:37.481 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-30T13:07:37.712 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libradosstriper1. 2024-03-30T13:07:37.736 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:37.776 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libradosstriper1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:37.809 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-30T13:07:37.824 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-30T13:07:37.867 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-30T13:07:38.179 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua5.1. 2024-03-30T13:07:38.194 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-30T13:07:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-common. 2024-03-30T13:07:38.228 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-30T13:07:38.240 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:38.279 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:38.540 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-any. 2024-03-30T13:07:38.555 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-30T13:07:38.607 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-30T13:07:38.851 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package zip. 2024-03-30T13:07:38.877 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-30T13:07:38.917 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking zip (3.0-12build2) ... 2024-03-30T13:07:39.271 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package unzip. 2024-03-30T13:07:39.297 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-30T13:07:39.337 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-30T13:07:39.601 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-base. 2024-03-30T13:07:39.628 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:39.677 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-base (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:39.749 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package luarocks. 2024-03-30T13:07:39.776 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-30T13:07:39.816 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-30T13:07:40.166 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package librgw2. 2024-03-30T13:07:40.192 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../027-librgw2_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:40.223 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking librgw2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:40.247 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-30T13:07:40.274 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-30T13:07:40.313 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-30T13:07:40.591 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cheroot. 2024-03-30T13:07:40.607 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-30T13:07:40.641 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T13:07:40.742 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rgw. 2024-03-30T13:07:40.758 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:40.801 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:40.977 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-30T13:07:41.004 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-30T13:07:41.053 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-30T13:07:41.145 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-30T13:07:41.172 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-30T13:07:41.212 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T13:07:41.297 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-30T13:07:41.324 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-30T13:07:41.380 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-30T13:07:41.532 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libradosstriper1. 2024-03-30T13:07:41.558 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:41.598 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libradosstriper1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:41.683 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-30T13:07:41.710 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-30T13:07:41.750 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-30T13:07:41.985 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-common. 2024-03-30T13:07:41.986 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-tempora. 2024-03-30T13:07:42.002 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-30T13:07:42.014 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:42.036 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-30T13:07:42.051 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:42.339 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-portend. 2024-03-30T13:07:42.366 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-30T13:07:42.406 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-30T13:07:42.708 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-30T13:07:42.735 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-30T13:07:42.775 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-30T13:07:43.236 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-30T13:07:43.253 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-30T13:07:43.286 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-30T13:07:43.598 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-natsort. 2024-03-30T13:07:43.615 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-base. 2024-03-30T13:07:43.625 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-30T13:07:43.642 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:43.665 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-30T13:07:43.699 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-base (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:43.976 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-logutils. 2024-03-30T13:07:44.003 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-30T13:07:44.043 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-30T13:07:44.312 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-30T13:07:44.328 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-30T13:07:44.339 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-30T13:07:44.346 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-30T13:07:44.371 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-30T13:07:44.394 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-30T13:07:44.673 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-mako. 2024-03-30T13:07:44.700 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-30T13:07:44.740 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T13:07:44.747 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cheroot. 2024-03-30T13:07:44.774 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-30T13:07:44.822 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T13:07:45.068 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-30T13:07:45.095 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-30T13:07:45.135 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-30T13:07:45.192 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-30T13:07:45.219 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-30T13:07:45.268 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-30T13:07:45.454 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-30T13:07:45.481 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-30T13:07:45.521 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-30T13:07:45.578 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-30T13:07:45.604 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-30T13:07:45.644 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-30T13:07:45.765 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-webob. 2024-03-30T13:07:45.781 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-30T13:07:45.815 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-30T13:07:45.997 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-30T13:07:46.024 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-30T13:07:46.072 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-30T13:07:46.143 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-waitress. 2024-03-30T13:07:46.170 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-30T13:07:46.213 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T13:07:46.469 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-tempora. 2024-03-30T13:07:46.496 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-30T13:07:46.530 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-tempita. 2024-03-30T13:07:46.544 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-30T13:07:46.557 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-30T13:07:46.596 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T13:07:46.864 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-portend. 2024-03-30T13:07:46.890 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-30T13:07:46.899 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-paste. 2024-03-30T13:07:46.926 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-30T13:07:46.938 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-30T13:07:46.965 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T13:07:47.291 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-30T13:07:47.294 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-30T13:07:47.318 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-30T13:07:47.321 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-30T13:07:47.360 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T13:07:47.366 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-30T13:07:47.672 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-30T13:07:47.699 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-30T13:07:47.738 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-30T13:07:47.744 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-30T13:07:47.771 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-30T13:07:47.819 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-30T13:07:48.016 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-webtest. 2024-03-30T13:07:48.043 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-30T13:07:48.083 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-30T13:07:48.182 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-natsort. 2024-03-30T13:07:48.209 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-30T13:07:48.248 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-30T13:07:48.519 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pecan. 2024-03-30T13:07:48.546 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-30T13:07:48.577 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T13:07:48.710 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-logutils. 2024-03-30T13:07:48.737 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-30T13:07:48.776 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-30T13:07:48.891 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-certifi. 2024-03-30T13:07:48.918 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-30T13:07:48.957 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-30T13:07:49.054 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-30T13:07:49.070 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-30T13:07:49.104 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-30T13:07:49.311 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-idna. 2024-03-30T13:07:49.337 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-30T13:07:49.377 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-30T13:07:49.382 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-mako. 2024-03-30T13:07:49.409 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-30T13:07:49.448 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T13:07:49.680 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-urllib3. 2024-03-30T13:07:49.707 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-30T13:07:49.746 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T13:07:49.810 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-30T13:07:49.837 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-30T13:07:49.877 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-30T13:07:50.074 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-requests. 2024-03-30T13:07:50.101 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-30T13:07:50.141 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T13:07:50.180 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-30T13:07:50.207 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-30T13:07:50.246 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-30T13:07:50.402 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-30T13:07:50.429 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-30T13:07:50.461 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T13:07:50.499 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-webob. 2024-03-30T13:07:50.514 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-30T13:07:50.557 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-30T13:07:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-30T13:07:50.749 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:50.782 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:50.902 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-waitress. 2024-03-30T13:07:50.929 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-30T13:07:50.972 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T13:07:51.179 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-30T13:07:51.206 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:51.245 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:51.280 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-tempita. 2024-03-30T13:07:51.307 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-30T13:07:51.346 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T13:07:51.557 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-30T13:07:51.584 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-30T13:07:51.623 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-30T13:07:51.624 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-paste. 2024-03-30T13:07:51.652 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-30T13:07:51.691 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T13:07:51.901 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-openssl. 2024-03-30T13:07:51.928 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-30T13:07:51.959 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-30T13:07:52.019 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-30T13:07:52.036 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-30T13:07:52.078 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T13:07:52.279 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr. 2024-03-30T13:07:52.306 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:52.345 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:52.347 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-30T13:07:52.374 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-30T13:07:52.406 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-30T13:07:52.684 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-webtest. 2024-03-30T13:07:52.711 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-30T13:07:52.740 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mon. 2024-03-30T13:07:52.763 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-30T13:07:52.768 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:52.813 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mon (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:53.063 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pecan. 2024-03-30T13:07:53.089 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-30T13:07:53.120 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T13:07:53.628 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-osd. 2024-03-30T13:07:53.655 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:53.667 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-certifi. 2024-03-30T13:07:53.697 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-osd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:53.697 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-30T13:07:53.725 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-30T13:07:54.053 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-idna. 2024-03-30T13:07:54.080 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-30T13:07:54.119 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-30T13:07:54.439 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-urllib3. 2024-03-30T13:07:54.466 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-30T13:07:54.505 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T13:07:54.805 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph. 2024-03-30T13:07:54.821 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../068-ceph_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:54.851 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-requests. 2024-03-30T13:07:54.854 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:54.877 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-30T13:07:54.917 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T13:07:55.149 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-fuse. 2024-03-30T13:07:55.177 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:55.204 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-30T13:07:55.215 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:55.230 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-30T13:07:55.270 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T13:07:55.565 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-30T13:07:55.585 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mds. 2024-03-30T13:07:55.591 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:55.613 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:55.631 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:55.652 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mds (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:56.030 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephadm. 2024-03-30T13:07:56.036 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-30T13:07:56.057 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../071-cephadm_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:56.063 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:56.088 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:56.103 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:56.383 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-30T13:07:56.411 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-30T13:07:56.422 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-30T13:07:56.449 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-30T13:07:56.449 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-30T13:07:56.489 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-30T13:07:56.775 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-openssl. 2024-03-30T13:07:56.777 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-30T13:07:56.793 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T13:07:56.803 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-30T13:07:56.827 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T13:07:56.842 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-30T13:07:57.187 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr. 2024-03-30T13:07:57.214 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:57.253 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:57.657 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mon. 2024-03-30T13:07:57.684 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:57.723 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mon (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:57.849 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-babel. 2024-03-30T13:07:57.877 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T13:07:57.915 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T13:07:58.201 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-jinja2. 2024-03-30T13:07:58.218 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-30T13:07:58.285 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T13:07:58.344 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-osd. 2024-03-30T13:07:58.370 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:58.410 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-osd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:58.638 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-30T13:07:58.665 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:58.704 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:59.058 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-30T13:07:59.086 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-30T13:07:59.124 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-30T13:07:59.419 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-routes. 2024-03-30T13:07:59.447 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-30T13:07:59.472 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph. 2024-03-30T13:07:59.477 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T13:07:59.487 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../068-ceph_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:59.521 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:59.757 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-fuse. 2024-03-30T13:07:59.773 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:07:59.805 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-30T13:07:59.807 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:07:59.833 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:07:59.871 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:00.194 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mds. 2024-03-30T13:08:00.222 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:00.260 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mds (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:00.705 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephadm. 2024-03-30T13:08:00.732 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../071-cephadm_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:00.772 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:01.091 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-30T13:08:01.118 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-30T13:08:01.127 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-30T13:08:01.155 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-30T13:08:01.157 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-30T13:08:01.193 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T13:08:01.502 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-30T13:08:01.530 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T13:08:01.569 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T13:08:01.664 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-joblib. 2024-03-30T13:08:01.691 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-30T13:08:01.722 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T13:08:02.042 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-30T13:08:02.067 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-30T13:08:02.186 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-30T13:08:02.497 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-sklearn. 2024-03-30T13:08:02.524 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-30T13:08:02.563 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T13:08:02.666 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-babel. 2024-03-30T13:08:02.693 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-30T13:08:02.732 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T13:08:03.068 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-jinja2. 2024-03-30T13:08:03.095 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-30T13:08:03.135 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T13:08:03.281 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-30T13:08:03.309 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:03.348 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:03.471 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-30T13:08:03.498 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:03.538 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:04.067 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-30T13:08:04.093 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-30T13:08:04.158 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-30T13:08:04.309 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-cachetools. 2024-03-30T13:08:04.337 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-30T13:08:04.375 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-30T13:08:04.461 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-routes. 2024-03-30T13:08:04.488 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-30T13:08:04.527 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T13:08:04.662 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-30T13:08:04.690 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-30T13:08:04.720 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-30T13:08:04.873 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-30T13:08:04.899 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:04.939 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:05.023 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-30T13:08:05.051 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-30T13:08:05.090 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T13:08:05.385 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-rsa. 2024-03-30T13:08:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-30T13:08:05.443 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-30T13:08:05.747 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-google-auth. 2024-03-30T13:08:05.775 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-30T13:08:05.813 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-30T13:08:06.125 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-30T13:08:06.141 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-30T13:08:06.175 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T13:08:06.244 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-30T13:08:06.260 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-30T13:08:06.302 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T13:08:06.436 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-websocket. 2024-03-30T13:08:06.464 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-30T13:08:06.494 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-30T13:08:06.806 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-30T13:08:06.806 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-joblib. 2024-03-30T13:08:06.833 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-30T13:08:06.834 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-30T13:08:06.864 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T13:08:06.872 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T13:08:07.243 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-30T13:08:07.270 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-30T13:08:07.395 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-30T13:08:07.520 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-30T13:08:07.549 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:07.586 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:07.702 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-sklearn. 2024-03-30T13:08:07.729 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-30T13:08:07.764 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T13:08:07.906 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-30T13:08:07.934 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-30T13:08:07.972 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T13:08:08.259 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-30T13:08:08.287 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T13:08:08.325 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T13:08:08.441 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-30T13:08:08.468 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:08.507 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:08.570 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package jq. 2024-03-30T13:08:08.586 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T13:08:08.619 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-30T13:08:08.856 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package socat. 2024-03-30T13:08:08.884 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-30T13:08:08.922 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-30T13:08:09.444 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-cachetools. 2024-03-30T13:08:09.472 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-30T13:08:09.502 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-30T13:08:09.517 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package xmlstarlet. 2024-03-30T13:08:09.546 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-30T13:08:09.584 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-30T13:08:09.780 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-30T13:08:09.807 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-30T13:08:09.846 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-30T13:08:10.005 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-test. 2024-03-30T13:08:10.033 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:10.079 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-test (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:10.167 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-30T13:08:10.194 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-30T13:08:10.225 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T13:08:10.486 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-rsa. 2024-03-30T13:08:10.513 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-30T13:08:10.561 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-30T13:08:10.873 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-google-auth. 2024-03-30T13:08:10.901 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-30T13:08:10.940 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-30T13:08:11.290 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-30T13:08:11.315 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-30T13:08:11.360 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T13:08:11.646 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-websocket. 2024-03-30T13:08:11.674 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-30T13:08:11.713 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-30T13:08:11.999 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-30T13:08:12.026 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-30T13:08:12.057 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T13:08:12.475 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ceph-volume. 2024-03-30T13:08:12.504 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:12.542 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ceph-volume (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:12.705 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-30T13:08:12.721 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:12.755 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:12.938 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-30T13:08:12.966 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:13.004 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-mirror (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:13.074 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-30T13:08:13.102 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-30T13:08:13.141 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T13:08:13.265 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-colorama. 2024-03-30T13:08:13.282 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-30T13:08:13.315 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-30T13:08:13.486 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-30T13:08:13.513 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T13:08:13.534 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-shell. 2024-03-30T13:08:13.551 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:13.552 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T13:08:13.584 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-shell (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:13.813 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package jq. 2024-03-30T13:08:13.829 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-30T13:08:13.857 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package cephfs-top. 2024-03-30T13:08:13.863 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-30T13:08:13.885 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:13.928 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking cephfs-top (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:14.116 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package socat. 2024-03-30T13:08:14.132 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-30T13:08:14.174 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-30T13:08:14.181 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-30T13:08:14.209 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:14.248 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libcephfs-dev (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:14.703 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package xmlstarlet. 2024-03-30T13:08:14.709 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-30T13:08:14.731 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-30T13:08:14.738 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-30T13:08:14.769 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-30T13:08:14.775 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-30T13:08:15.113 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-30T13:08:15.142 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-30T13:08:15.157 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-test. 2024-03-30T13:08:15.180 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-30T13:08:15.185 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:15.215 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-test (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:15.559 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package nvme-cli. 2024-03-30T13:08:15.588 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-30T13:08:15.625 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T13:08:16.022 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package pkg-config. 2024-03-30T13:08:16.051 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-30T13:08:16.088 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-30T13:08:16.333 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-30T13:08:16.350 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-30T13:08:16.392 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-30T13:08:16.770 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-30T13:08:16.798 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-30T13:08:16.828 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-30T13:08:17.131 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pastescript. 2024-03-30T13:08:17.159 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-30T13:08:17.197 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-30T13:08:17.517 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pluggy. 2024-03-30T13:08:17.520 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ceph-volume. 2024-03-30T13:08:17.546 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-30T13:08:17.547 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:17.584 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-30T13:08:17.586 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ceph-volume (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:17.879 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-psutil. 2024-03-30T13:08:17.907 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-30T13:08:17.932 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-30T13:08:17.945 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-30T13:08:17.960 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:17.998 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephfs-mirror (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:18.281 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-py. 2024-03-30T13:08:18.310 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-30T13:08:18.335 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-colorama. 2024-03-30T13:08:18.347 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-30T13:08:18.363 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-30T13:08:18.401 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-30T13:08:18.668 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pygments. 2024-03-30T13:08:18.696 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephfs-shell. 2024-03-30T13:08:18.696 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-30T13:08:18.712 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:18.734 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T13:08:18.754 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephfs-shell (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:18.998 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package cephfs-top. 2024-03-30T13:08:19.026 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2519-gac171039-1jammy_all.deb ... 2024-03-30T13:08:19.056 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking cephfs-top (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:19.164 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-30T13:08:19.192 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-30T13:08:19.221 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-30T13:08:19.301 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-30T13:08:19.317 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:19.359 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libcephfs-dev (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:19.541 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-toml. 2024-03-30T13:08:19.570 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-30T13:08:19.608 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-30T13:08:19.637 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-30T13:08:19.665 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-30T13:08:19.703 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-30T13:08:19.911 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-pytest. 2024-03-30T13:08:19.939 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-30T13:08:19.977 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T13:08:19.999 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-30T13:08:20.016 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-30T13:08:20.058 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-30T13:08:20.330 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package python3-simplejson. 2024-03-30T13:08:20.359 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-30T13:08:20.397 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-30T13:08:20.445 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package nvme-cli. 2024-03-30T13:08:20.474 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-30T13:08:20.512 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T13:08:20.675 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-30T13:08:20.703 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-30T13:08:20.741 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-30T13:08:20.942 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package pkg-config. 2024-03-30T13:08:20.970 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-30T13:08:21.009 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-30T13:08:21.262 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-30T13:08:21.278 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-30T13:08:21.320 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-30T13:08:21.462 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package radosgw. 2024-03-30T13:08:21.491 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../122-radosgw_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:21.528 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking radosgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:21.748 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-30T13:08:21.776 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-30T13:08:21.814 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-30T13:08:22.143 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pastescript. 2024-03-30T13:08:22.170 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-30T13:08:22.209 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-30T13:08:22.504 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pluggy. 2024-03-30T13:08:22.506 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package rbd-fuse. 2024-03-30T13:08:22.523 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:22.532 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-30T13:08:22.562 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-30T13:08:22.564 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking rbd-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:22.857 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-psutil. 2024-03-30T13:08:22.861 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-30T13:08:22.884 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-30T13:08:22.923 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-30T13:08:23.120 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T13:08:23.246 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T13:08:23.260 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-py. 2024-03-30T13:08:23.288 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-30T13:08:23.326 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-30T13:08:23.513 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-30T13:08:23.596 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pygments. 2024-03-30T13:08:23.624 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-30T13:08:23.654 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T13:08:23.823 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-30T13:08:24.093 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-30T13:08:24.120 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-30T13:08:24.159 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-30T13:08:24.288 INFO:teuthology.orchestra.run.smithi043.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-30T13:08:24.348 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:24.478 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-toml. 2024-03-30T13:08:24.506 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-30T13:08:24.545 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-30T13:08:24.781 INFO:teuthology.orchestra.run.smithi043.stdout:Adding system user cephadm....done 2024-03-30T13:08:24.937 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T13:08:24.985 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-pytest. 2024-03-30T13:08:25.014 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-30T13:08:25.158 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T13:08:25.280 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-30T13:08:25.501 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package python3-simplejson. 2024-03-30T13:08:25.529 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-30T13:08:25.548 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-30T13:08:25.568 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-30T13:08:25.682 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-30T13:08:25.862 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-30T13:08:25.878 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-30T13:08:25.920 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-30T13:08:25.959 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-30T13:08:26.243 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T13:08:26.369 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-30T13:08:26.687 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T13:08:26.699 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package radosgw. 2024-03-30T13:08:26.727 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../122-radosgw_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:26.765 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking radosgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:27.064 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-30T13:08:27.333 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-30T13:08:27.608 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-30T13:08:27.613 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package rbd-fuse. 2024-03-30T13:08:27.641 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2519-gac171039-1jammy_amd64.deb ... 2024-03-30T13:08:27.679 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking rbd-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:27.901 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-30T13:08:27.992 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-30T13:08:28.145 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-30T13:08:28.243 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-30T13:08:28.385 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-30T13:08:28.404 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-30T13:08:28.642 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-30T13:08:28.664 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-30T13:08:28.957 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-30T13:08:29.056 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-ceph-argparse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:29.333 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T13:08:29.386 INFO:teuthology.orchestra.run.smithi113.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-30T13:08:29.459 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-30T13:08:29.585 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-30T13:08:29.647 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:30.044 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-30T13:08:30.104 INFO:teuthology.orchestra.run.smithi113.stdout:Adding system user cephadm....done 2024-03-30T13:08:30.170 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-30T13:08:30.269 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-30T13:08:30.430 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-30T13:08:30.546 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-30T13:08:30.760 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-30T13:08:30.794 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-30T13:08:30.805 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-30T13:08:30.850 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T13:08:30.906 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-30T13:08:30.976 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-30T13:08:31.142 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-30T13:08:31.352 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T13:08:31.383 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-30T13:08:31.478 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-30T13:08:31.485 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-30T13:08:31.771 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T13:08:31.787 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-30T13:08:32.047 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-30T13:08:32.147 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-30T13:08:32.365 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up zip (3.0-12build2) ... 2024-03-30T13:08:32.398 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-30T13:08:32.493 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T13:08:32.649 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-30T13:08:32.909 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-30T13:08:33.144 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-30T13:08:33.220 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T13:08:33.404 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-30T13:08:33.487 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T13:08:33.588 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-30T13:08:33.663 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-30T13:08:33.689 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T13:08:33.914 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-ceph-argparse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:34.024 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-30T13:08:34.157 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-30T13:08:34.283 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-30T13:08:34.392 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-30T13:08:34.416 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-30T13:08:34.635 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-30T13:08:34.668 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T13:08:34.761 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-30T13:08:35.020 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-30T13:08:35.061 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T13:08:35.404 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-30T13:08:35.452 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-30T13:08:35.685 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-30T13:08:35.705 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T13:08:35.740 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-30T13:08:35.858 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-30T13:08:36.048 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-30T13:08:36.218 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-30T13:08:36.316 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-30T13:08:36.327 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-30T13:08:36.450 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:36.637 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-30T13:08:36.752 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-idna (3.3-1) ... 2024-03-30T13:08:36.913 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-30T13:08:37.086 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T13:08:37.197 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up zip (3.0-12build2) ... 2024-03-30T13:08:37.323 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-30T13:08:37.975 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-30T13:08:38.209 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-30T13:08:38.267 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-30T13:08:38.335 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-30T13:08:38.462 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-30T13:08:38.535 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:08:38.669 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-30T13:08:38.746 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-30T13:08:38.937 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T13:08:39.038 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-30T13:08:39.130 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-30T13:08:39.139 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T13:08:39.382 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-30T13:08:39.465 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-30T13:08:39.725 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-30T13:08:39.758 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-30T13:08:39.984 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:08:40.026 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-30T13:08:40.110 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-30T13:08:40.344 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-30T13:08:40.420 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-30T13:08:40.671 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-30T13:08:40.687 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-30T13:08:40.930 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-30T13:08:40.939 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-30T13:08:41.056 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:41.232 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-30T13:08:41.349 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-idna (3.3-1) ... 2024-03-30T13:08:41.508 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-30T13:08:41.667 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-30T13:08:41.784 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T13:08:41.910 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-30T13:08:42.203 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T13:08:42.329 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T13:08:42.630 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-30T13:08:42.889 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-30T13:08:42.989 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-30T13:08:43.157 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:08:43.275 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-30T13:08:43.290 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-30T13:08:43.433 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-30T13:08:43.560 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-30T13:08:43.567 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T13:08:43.657 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-30T13:08:43.686 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-30T13:08:43.803 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-30T13:08:43.936 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T13:08:44.070 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T13:08:44.088 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-30T13:08:44.348 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-30T13:08:44.488 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-30T13:08:44.599 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:08:44.708 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-30T13:08:44.756 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-30T13:08:44.907 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-30T13:08:44.993 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-30T13:08:45.175 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:08:45.244 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-30T13:08:45.301 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T13:08:45.495 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-30T13:08:45.652 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T13:08:45.730 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-30T13:08:45.962 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T13:08:46.072 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-30T13:08:46.238 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-30T13:08:46.332 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-30T13:08:46.449 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-30T13:08:46.474 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-30T13:08:46.726 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-30T13:08:46.742 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-30T13:08:46.852 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-30T13:08:47.027 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T13:08:47.170 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-30T13:08:47.295 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-30T13:08:47.479 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-30T13:08:47.713 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-30T13:08:47.730 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-30T13:08:47.856 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-30T13:08:47.982 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-30T13:08:47.998 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T13:08:48.064 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-30T13:08:48.309 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-30T13:08:48.334 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T13:08:48.435 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-30T13:08:48.460 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librados2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:48.586 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:48.720 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-30T13:08:48.828 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-30T13:08:49.079 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-30T13:08:49.123 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libcephfs2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:49.197 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-30T13:08:49.274 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libradosstriper1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:49.408 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-30T13:08:49.456 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-30T13:08:49.566 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-30T13:08:49.760 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librbd1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:49.867 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-30T13:08:49.886 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-modules-core (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:50.028 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:50.176 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-30T13:08:50.264 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T13:08:50.264 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T13:08:50.444 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-30T13:08:50.656 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-30T13:08:50.805 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libcephfs-dev (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:50.940 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-30T13:08:50.973 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rados (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:51.200 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-30T13:08:51.450 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up librgw2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:51.468 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-30T13:08:51.576 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-top (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:51.702 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T13:08:51.845 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-30T13:08:52.130 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-30T13:08:52.457 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-30T13:08:52.558 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librados2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:52.677 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:52.802 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-30T13:08:52.980 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rbd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.106 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up rbd-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.138 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libcephfs2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.233 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-rgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.264 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libradosstriper1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.358 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up python3-cephfs (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.382 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-30T13:08:53.484 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-shell (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.610 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.642 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librbd1 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.768 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-modules-core (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:53.894 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:54.113 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T13:08:54.113 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-30T13:08:54.120 INFO:teuthology.orchestra.run.smithi043.stdout:Adding group ceph....done 2024-03-30T13:08:54.538 INFO:teuthology.orchestra.run.smithi043.stdout:Adding system user ceph....done 2024-03-30T13:08:54.587 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libcephfs-dev (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:54.715 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rados (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:54.715 INFO:teuthology.orchestra.run.smithi043.stdout:Setting system user ceph properties....done 2024-03-30T13:08:54.731 INFO:teuthology.orchestra.run.smithi043.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-30T13:08:54.839 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up librgw2 (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:54.876 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-30T13:08:54.982 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephfs-top (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:55.100 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-30T13:08:55.225 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-30T13:08:55.737 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up cephfs-mirror (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:55.937 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:55.937 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:56.287 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rbd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.364 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-test (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.413 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up rbd-fuse (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.498 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up radosgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.539 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-rgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.665 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up python3-cephfs (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.791 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephfs-shell (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.925 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-common (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:56.992 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:08:56.992 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:08:57.435 INFO:teuthology.orchestra.run.smithi113.stdout:Adding group ceph....done 2024-03-30T13:08:57.458 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-base (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:57.810 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-30T13:08:57.829 INFO:teuthology.orchestra.run.smithi113.stdout:Adding system user ceph....done 2024-03-30T13:08:58.014 INFO:teuthology.orchestra.run.smithi113.stdout:Setting system user ceph properties....done 2024-03-30T13:08:58.031 INFO:teuthology.orchestra.run.smithi113.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-30T13:08:58.172 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-30T13:08:58.260 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mds (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:58.455 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-30T13:08:58.490 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T13:08:58.490 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T13:08:58.918 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up cephfs-mirror (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:58.945 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:59.129 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:59.129 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-30T13:08:59.190 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T13:08:59.190 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T13:08:59.619 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-test (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:59.663 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-osd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:59.720 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up radosgw (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:08:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T13:08:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T13:09:00.161 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:09:00.161 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-30T13:09:00.448 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:00.574 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:00.606 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-base (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:00.919 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-30T13:09:01.020 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mon (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:01.379 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T13:09:01.379 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T13:09:01.391 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mds (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:01.646 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T13:09:01.646 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-30T13:09:01.874 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:02.025 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:02.110 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:02.151 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-mgr-dashboard (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:02.319 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ceph-volume (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:02.328 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T13:09:02.328 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-30T13:09:02.665 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-30T13:09:02.764 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T13:09:02.827 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-osd (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:02.908 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T13:09:03.105 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T13:09:03.105 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-30T13:09:03.612 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:03.713 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:03.839 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mon (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:04.034 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-30T13:09:04.050 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T13:09:04.050 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-30T13:09:04.574 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-cephadm (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:04.688 INFO:teuthology.orchestra.run.smithi043.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T13:09:04.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-30T13:09:04.725 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:04.749 INFO:teuthology.orchestra.run.smithi043.stdout:Reading package lists... 2024-03-30T13:09:04.851 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-mgr-dashboard (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:04.947 INFO:teuthology.orchestra.run.smithi043.stdout:Building dependency tree... 2024-03-30T13:09:04.948 INFO:teuthology.orchestra.run.smithi043.stdout:Reading state information... 2024-03-30T13:09:05.003 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ceph-volume (19.0.0-2519-gac171039-1jammy) ... 2024-03-30T13:09:05.146 INFO:teuthology.orchestra.run.smithi043.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-30T13:09:05.146 INFO:teuthology.orchestra.run.smithi043.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-30T13:09:05.146 INFO:teuthology.orchestra.run.smithi043.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-30T13:09:05.146 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T13:09:05.146 INFO:teuthology.orchestra.run.smithi043.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T13:09:05.146 INFO:teuthology.orchestra.run.smithi043.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T13:09:05.146 INFO:teuthology.orchestra.run.smithi043.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T13:09:05.147 INFO:teuthology.orchestra.run.smithi043.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T13:09:05.147 INFO:teuthology.orchestra.run.smithi043.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T13:09:05.147 INFO:teuthology.orchestra.run.smithi043.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T13:09:05.148 INFO:teuthology.orchestra.run.smithi043.stdout:The following additional packages will be installed: 2024-03-30T13:09:05.148 INFO:teuthology.orchestra.run.smithi043.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-30T13:09:05.148 INFO:teuthology.orchestra.run.smithi043.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-30T13:09:05.148 INFO:teuthology.orchestra.run.smithi043.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-30T13:09:05.148 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-30T13:09:05.148 INFO:teuthology.orchestra.run.smithi043.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T13:09:05.149 INFO:teuthology.orchestra.run.smithi043.stdout:Suggested packages: 2024-03-30T13:09:05.150 INFO:teuthology.orchestra.run.smithi043.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-30T13:09:05.150 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-30T13:09:05.150 INFO:teuthology.orchestra.run.smithi043.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-30T13:09:05.192 INFO:teuthology.orchestra.run.smithi043.stdout:The following NEW packages will be installed: 2024-03-30T13:09:05.192 INFO:teuthology.orchestra.run.smithi043.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-30T13:09:05.193 INFO:teuthology.orchestra.run.smithi043.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-30T13:09:05.193 INFO:teuthology.orchestra.run.smithi043.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-30T13:09:05.193 INFO:teuthology.orchestra.run.smithi043.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-30T13:09:05.193 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-30T13:09:05.193 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-30T13:09:05.194 INFO:teuthology.orchestra.run.smithi043.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T13:09:05.194 INFO:teuthology.orchestra.run.smithi043.stdout:The following packages will be upgraded: 2024-03-30T13:09:05.195 INFO:teuthology.orchestra.run.smithi043.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-30T13:09:05.377 INFO:teuthology.orchestra.run.smithi043.stdout:4 upgraded, 39 newly installed, 0 to remove and 294 not upgraded. 2024-03-30T13:09:05.377 INFO:teuthology.orchestra.run.smithi043.stdout:Need to get 55.7 MB of archives. 2024-03-30T13:09:05.377 INFO:teuthology.orchestra.run.smithi043.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-30T13:09:05.377 INFO:teuthology.orchestra.run.smithi043.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-30T13:09:05.388 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-30T13:09:05.481 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T13:09:05.608 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T13:09:06.011 INFO:teuthology.orchestra.run.smithi043.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-30T13:09:06.027 INFO:teuthology.orchestra.run.smithi043.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-30T13:09:06.153 INFO:teuthology.orchestra.run.smithi043.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-30T13:09:06.154 INFO:teuthology.orchestra.run.smithi043.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-30T13:09:06.155 INFO:teuthology.orchestra.run.smithi043.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-30T13:09:06.184 INFO:teuthology.orchestra.run.smithi043.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-30T13:09:06.185 INFO:teuthology.orchestra.run.smithi043.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-30T13:09:06.189 INFO:teuthology.orchestra.run.smithi043.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-30T13:09:06.191 INFO:teuthology.orchestra.run.smithi043.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-30T13:09:06.192 INFO:teuthology.orchestra.run.smithi043.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-30T13:09:06.358 INFO:teuthology.orchestra.run.smithi043.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-30T13:09:06.641 INFO:teuthology.orchestra.run.smithi043.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-30T13:09:06.651 INFO:teuthology.orchestra.run.smithi043.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-30T13:09:06.656 INFO:teuthology.orchestra.run.smithi043.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-30T13:09:06.781 INFO:teuthology.orchestra.run.smithi043.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-30T13:09:06.870 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-30T13:09:07.529 INFO:teuthology.orchestra.run.smithi113.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T13:09:07.532 DEBUG:teuthology.orchestra.run.smithi113:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-30T13:09:07.586 INFO:teuthology.orchestra.run.smithi113.stdout:Reading package lists... 2024-03-30T13:09:07.783 INFO:teuthology.orchestra.run.smithi113.stdout:Building dependency tree... 2024-03-30T13:09:07.784 INFO:teuthology.orchestra.run.smithi113.stdout:Reading state information... 2024-03-30T13:09:07.822 INFO:teuthology.orchestra.run.smithi043.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-30T13:09:07.827 INFO:teuthology.orchestra.run.smithi043.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-30T13:09:07.871 INFO:teuthology.orchestra.run.smithi043.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-30T13:09:07.877 INFO:teuthology.orchestra.run.smithi043.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-30T13:09:07.878 INFO:teuthology.orchestra.run.smithi043.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-30T13:09:07.888 INFO:teuthology.orchestra.run.smithi043.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages were automatically installed and are no longer required: 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-30T13:09:07.971 INFO:teuthology.orchestra.run.smithi113.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-30T13:09:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-30T13:09:07.972 INFO:teuthology.orchestra.run.smithi113.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-30T13:09:07.972 INFO:teuthology.orchestra.run.smithi113.stdout:The following additional packages will be installed: 2024-03-30T13:09:07.972 INFO:teuthology.orchestra.run.smithi113.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-30T13:09:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-30T13:09:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-30T13:09:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-30T13:09:07.973 INFO:teuthology.orchestra.run.smithi113.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T13:09:07.974 INFO:teuthology.orchestra.run.smithi113.stdout:Suggested packages: 2024-03-30T13:09:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-30T13:09:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-30T13:09:07.974 INFO:teuthology.orchestra.run.smithi113.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-30T13:09:08.014 INFO:teuthology.orchestra.run.smithi113.stdout:The following NEW packages will be installed: 2024-03-30T13:09:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-30T13:09:08.015 INFO:teuthology.orchestra.run.smithi113.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-30T13:09:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-30T13:09:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-30T13:09:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-30T13:09:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-30T13:09:08.016 INFO:teuthology.orchestra.run.smithi113.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-30T13:09:08.017 INFO:teuthology.orchestra.run.smithi113.stdout:The following packages will be upgraded: 2024-03-30T13:09:08.018 INFO:teuthology.orchestra.run.smithi113.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-30T13:09:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-30T13:09:08.076 INFO:teuthology.orchestra.run.smithi113.stdout:4 upgraded, 39 newly installed, 0 to remove and 294 not upgraded. 2024-03-30T13:09:08.076 INFO:teuthology.orchestra.run.smithi113.stdout:Need to get 55.7 MB of archives. 2024-03-30T13:09:08.076 INFO:teuthology.orchestra.run.smithi113.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-30T13:09:08.076 INFO:teuthology.orchestra.run.smithi113.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-30T13:09:08.243 INFO:teuthology.orchestra.run.smithi113.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-30T13:09:08.250 INFO:teuthology.orchestra.run.smithi113.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-30T13:09:08.290 INFO:teuthology.orchestra.run.smithi113.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-30T13:09:08.291 INFO:teuthology.orchestra.run.smithi113.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-30T13:09:08.292 INFO:teuthology.orchestra.run.smithi113.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-30T13:09:08.302 INFO:teuthology.orchestra.run.smithi113.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-30T13:09:08.303 INFO:teuthology.orchestra.run.smithi113.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-30T13:09:08.306 INFO:teuthology.orchestra.run.smithi113.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-30T13:09:08.307 INFO:teuthology.orchestra.run.smithi113.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-30T13:09:08.308 INFO:teuthology.orchestra.run.smithi113.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-30T13:09:08.309 INFO:teuthology.orchestra.run.smithi113.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-30T13:09:08.313 INFO:teuthology.orchestra.run.smithi113.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-30T13:09:08.313 INFO:teuthology.orchestra.run.smithi113.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-30T13:09:08.333 INFO:teuthology.orchestra.run.smithi113.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-30T13:09:08.334 INFO:teuthology.orchestra.run.smithi113.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-30T13:09:08.655 INFO:teuthology.orchestra.run.smithi043.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-30T13:09:08.655 INFO:teuthology.orchestra.run.smithi043.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-30T13:09:08.669 INFO:teuthology.orchestra.run.smithi113.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-30T13:09:08.671 INFO:teuthology.orchestra.run.smithi113.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-30T13:09:08.671 INFO:teuthology.orchestra.run.smithi113.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-30T13:09:08.674 INFO:teuthology.orchestra.run.smithi113.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-30T13:09:08.674 INFO:teuthology.orchestra.run.smithi113.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-30T13:09:08.678 INFO:teuthology.orchestra.run.smithi113.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-30T13:09:08.680 INFO:teuthology.orchestra.run.smithi113.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-30T13:09:08.687 INFO:teuthology.orchestra.run.smithi113.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-30T13:09:08.687 INFO:teuthology.orchestra.run.smithi113.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-30T13:09:08.709 INFO:teuthology.orchestra.run.smithi113.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-30T13:09:08.710 INFO:teuthology.orchestra.run.smithi113.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-30T13:09:08.721 INFO:teuthology.orchestra.run.smithi113.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-30T13:09:08.745 INFO:teuthology.orchestra.run.smithi113.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-30T13:09:08.748 INFO:teuthology.orchestra.run.smithi113.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-30T13:09:08.749 INFO:teuthology.orchestra.run.smithi113.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-30T13:09:08.749 INFO:teuthology.orchestra.run.smithi113.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-30T13:09:08.750 INFO:teuthology.orchestra.run.smithi113.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-30T13:09:08.760 INFO:teuthology.orchestra.run.smithi113.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-30T13:09:08.762 INFO:teuthology.orchestra.run.smithi113.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-30T13:09:08.835 INFO:teuthology.orchestra.run.smithi043.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-30T13:09:08.835 INFO:teuthology.orchestra.run.smithi043.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-30T13:09:08.874 INFO:teuthology.orchestra.run.smithi043.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-30T13:09:08.935 INFO:teuthology.orchestra.run.smithi043.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-30T13:09:08.942 INFO:teuthology.orchestra.run.smithi043.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-30T13:09:08.943 INFO:teuthology.orchestra.run.smithi043.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-30T13:09:08.944 INFO:teuthology.orchestra.run.smithi043.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-30T13:09:08.945 INFO:teuthology.orchestra.run.smithi043.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-30T13:09:09.023 INFO:teuthology.orchestra.run.smithi113.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-30T13:09:09.023 INFO:teuthology.orchestra.run.smithi113.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-30T13:09:09.023 INFO:teuthology.orchestra.run.smithi113.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-30T13:09:09.023 INFO:teuthology.orchestra.run.smithi113.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-30T13:09:09.023 INFO:teuthology.orchestra.run.smithi113.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-30T13:09:09.024 INFO:teuthology.orchestra.run.smithi113.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-30T13:09:09.035 INFO:teuthology.orchestra.run.smithi113.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-30T13:09:09.036 INFO:teuthology.orchestra.run.smithi113.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-30T13:09:09.152 INFO:teuthology.orchestra.run.smithi043.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-30T13:09:09.441 INFO:teuthology.orchestra.run.smithi113.stdout:Preconfiguring packages ... 2024-03-30T13:09:09.555 INFO:teuthology.orchestra.run.smithi043.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-30T13:09:09.606 INFO:teuthology.orchestra.run.smithi113.stdout:Fetched 55.7 MB in 1s (55.1 MB/s) 2024-03-30T13:09:09.922 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-30T13:09:09.928 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T13:09:10.066 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T13:09:10.401 INFO:teuthology.orchestra.run.smithi043.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-30T13:09:10.401 INFO:teuthology.orchestra.run.smithi043.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-30T13:09:10.402 INFO:teuthology.orchestra.run.smithi043.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-30T13:09:10.402 INFO:teuthology.orchestra.run.smithi043.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-30T13:09:10.402 INFO:teuthology.orchestra.run.smithi043.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-30T13:09:10.403 INFO:teuthology.orchestra.run.smithi043.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-30T13:09:10.431 INFO:teuthology.orchestra.run.smithi043.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-30T13:09:10.431 INFO:teuthology.orchestra.run.smithi043.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-30T13:09:10.713 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-30T13:09:10.858 INFO:teuthology.orchestra.run.smithi043.stdout:Preconfiguring packages ... 2024-03-30T13:09:11.032 INFO:teuthology.orchestra.run.smithi043.stdout:Fetched 55.7 MB in 5s (10.7 MB/s) 2024-03-30T13:09:11.349 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-30T13:09:11.355 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T13:09:11.498 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T13:09:11.661 INFO:teuthology.orchestra.run.smithi113.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-30T13:09:11.947 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package flex. 2024-03-30T13:09:12.011 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-30T13:09:12.016 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-30T13:09:12.054 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-30T13:09:12.112 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-30T13:09:12.630 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T13:09:12.802 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-30T13:09:13.032 INFO:teuthology.orchestra.run.smithi043.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-30T13:09:13.213 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T13:09:13.319 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package flex. 2024-03-30T13:09:13.384 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173988 files and directories currently installed.) 2024-03-30T13:09:13.390 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-30T13:09:13.435 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-30T13:09:13.754 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-30T13:09:13.759 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T13:09:13.885 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T13:09:13.910 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T13:09:14.082 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-30T13:09:14.261 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T13:09:14.367 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T13:09:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T13:09:14.702 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T13:09:15.032 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-30T13:09:15.038 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T13:09:15.078 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package bison. 2024-03-30T13:09:15.165 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T13:09:15.183 INFO:teuthology.orchestra.run.smithi113.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-30T13:09:15.189 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-30T13:09:15.239 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T13:09:15.566 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-30T13:09:15.656 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-30T13:09:15.684 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-30T13:09:15.696 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-30T13:09:15.723 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T13:09:16.041 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T13:09:16.050 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package dump. 2024-03-30T13:09:16.067 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-30T13:09:16.101 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking dump (0.4b47-3) ... 2024-03-30T13:09:16.397 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package bison. 2024-03-30T13:09:16.480 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package indent. 2024-03-30T13:09:16.501 INFO:teuthology.orchestra.run.smithi043.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174061 files and directories currently installed.) 2024-03-30T13:09:16.507 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-30T13:09:16.508 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:16.549 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T13:09:16.579 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T13:09:16.942 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-30T13:09:16.970 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-30T13:09:17.008 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T13:09:17.295 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package dump. 2024-03-30T13:09:17.323 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-30T13:09:17.342 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-30T13:09:17.353 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking dump (0.4b47-3) ... 2024-03-30T13:09:17.358 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-30T13:09:17.392 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T13:09:17.665 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package indent. 2024-03-30T13:09:17.694 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:17.721 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-30T13:09:17.732 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T13:09:17.749 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-30T13:09:17.790 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T13:09:18.001 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-30T13:09:18.018 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-30T13:09:18.057 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-30T13:09:18.059 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T13:09:18.085 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-30T13:09:18.124 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T13:09:18.330 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-30T13:09:18.359 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-30T13:09:18.388 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T13:09:18.436 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-30T13:09:18.464 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-30T13:09:18.494 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:18.632 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-30T13:09:18.649 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-30T13:09:18.691 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T13:09:18.721 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-30T13:09:18.738 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-30T13:09:18.780 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:19.019 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-30T13:09:19.033 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libjson-perl. 2024-03-30T13:09:19.048 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-30T13:09:19.060 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-30T13:09:19.086 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:19.099 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-30T13:09:19.330 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-30T13:09:19.347 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-30T13:09:19.388 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:19.453 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-30T13:09:19.480 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-30T13:09:19.519 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T13:09:19.607 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libjson-perl. 2024-03-30T13:09:19.624 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-30T13:09:19.666 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-30T13:09:19.986 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-30T13:09:20.014 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-30T13:09:20.054 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T13:09:21.247 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-30T13:09:21.278 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T13:09:21.314 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:21.591 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-30T13:09:21.607 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T13:09:21.650 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:21.797 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-30T13:09:21.826 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T13:09:21.863 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:22.003 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-30T13:09:22.031 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T13:09:22.069 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:22.108 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-30T13:09:22.124 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-30T13:09:22.157 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:22.466 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-30T13:09:22.469 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-30T13:09:22.494 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-30T13:09:22.497 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T13:09:22.532 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T13:09:22.535 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:22.877 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-30T13:09:22.888 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-30T13:09:22.906 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T13:09:22.917 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-30T13:09:22.955 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T13:09:22.966 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T13:09:23.241 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-30T13:09:23.269 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T13:09:23.280 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-30T13:09:23.299 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T13:09:23.308 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:23.338 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:23.652 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-30T13:09:23.681 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:23.691 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-30T13:09:23.719 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:23.727 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:23.758 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:24.078 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-30T13:09:24.080 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-30T13:09:24.094 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:24.108 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:24.136 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:24.146 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:24.355 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-30T13:09:24.372 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T13:09:24.405 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T13:09:24.449 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-30T13:09:24.474 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:24.516 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:24.744 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-30T13:09:24.772 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-30T13:09:24.810 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T13:09:24.925 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-30T13:09:24.954 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-30T13:09:24.992 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T13:09:25.345 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package modemmanager. 2024-03-30T13:09:25.373 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T13:09:25.380 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-30T13:09:25.409 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-30T13:09:25.497 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:25.497 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T13:09:25.733 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package modemmanager. 2024-03-30T13:09:25.762 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-30T13:09:25.793 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:26.052 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package network-manager. 2024-03-30T13:09:26.080 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T13:09:26.127 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-30T13:09:26.492 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package network-manager. 2024-03-30T13:09:26.521 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-30T13:09:26.560 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-30T13:09:26.768 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package ppp. 2024-03-30T13:09:26.796 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-30T13:09:27.068 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T13:09:27.183 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package ppp. 2024-03-30T13:09:27.212 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-30T13:09:27.505 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T13:09:27.550 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package pptp-linux. 2024-03-30T13:09:27.578 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-30T13:09:27.616 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-30T13:09:27.821 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package pptp-linux. 2024-03-30T13:09:27.850 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-30T13:09:27.861 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-30T13:09:27.877 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-30T13:09:27.888 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-30T13:09:27.911 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-30T13:09:28.149 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-30T13:09:28.177 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-30T13:09:28.180 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-30T13:09:28.208 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-30T13:09:28.224 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-30T13:09:28.247 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-client-common (238) ... 2024-03-30T13:09:28.551 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-30T13:09:28.579 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:28.593 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-30T13:09:28.617 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:28.622 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-30T13:09:28.668 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client-common (238) ... 2024-03-30T13:09:28.997 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-30T13:09:29.026 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:29.064 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:29.198 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-common. 2024-03-30T13:09:29.225 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-30T13:09:29.283 INFO:teuthology.orchestra.run.smithi113.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-30T13:09:29.312 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-common (238) ... 2024-03-30T13:09:29.619 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-common. 2024-03-30T13:09:29.647 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-30T13:09:29.703 INFO:teuthology.orchestra.run.smithi043.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-30T13:09:29.733 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-common (238) ... 2024-03-30T13:09:29.763 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-14. 2024-03-30T13:09:29.791 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:29.832 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:30.201 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-14. 2024-03-30T13:09:30.229 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-30T13:09:30.270 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:31.258 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql. 2024-03-30T13:09:31.275 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-30T13:09:31.316 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql (14+238) ... 2024-03-30T13:09:31.527 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-client. 2024-03-30T13:09:31.544 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-30T13:09:31.585 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-client (14+238) ... 2024-03-30T13:09:31.829 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-30T13:09:31.846 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-30T13:09:31.887 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-30T13:09:32.095 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql. 2024-03-30T13:09:32.114 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-30T13:09:32.154 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql (14+238) ... 2024-03-30T13:09:32.207 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package tcl8.6. 2024-03-30T13:09:32.235 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-30T13:09:32.273 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T13:09:32.389 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-client. 2024-03-30T13:09:32.406 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-30T13:09:32.447 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-client (14+238) ... 2024-03-30T13:09:32.542 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package tcl. 2024-03-30T13:09:32.573 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-30T13:09:32.614 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-30T13:09:32.666 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-30T13:09:32.696 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-30T13:09:32.733 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-30T13:09:32.954 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package wpasupplicant. 2024-03-30T13:09:32.982 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-30T13:09:33.020 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T13:09:33.060 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package tcl8.6. 2024-03-30T13:09:33.090 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-30T13:09:33.127 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T13:09:33.383 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-30T13:09:33.405 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package tcl. 2024-03-30T13:09:33.411 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-30T13:09:33.434 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-30T13:09:33.450 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-30T13:09:33.477 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-30T13:09:33.774 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package wpasupplicant. 2024-03-30T13:09:33.795 INFO:teuthology.orchestra.run.smithi113.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-30T13:09:33.804 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-30T13:09:33.823 INFO:teuthology.orchestra.run.smithi113.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-30T13:09:33.832 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T13:09:33.870 INFO:teuthology.orchestra.run.smithi113.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T13:09:34.141 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-client-common (238) ... 2024-03-30T13:09:34.213 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-30T13:09:34.241 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-30T13:09:34.279 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-30T13:09:34.351 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-30T13:09:34.460 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T13:09:34.561 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:34.657 INFO:teuthology.orchestra.run.smithi043.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-30T13:09:34.686 INFO:teuthology.orchestra.run.smithi043.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-30T13:09:34.687 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T13:09:34.732 INFO:teuthology.orchestra.run.smithi043.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T13:09:34.813 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T13:09:34.939 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:35.037 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client-common (238) ... 2024-03-30T13:09:35.280 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-30T13:09:35.405 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-30T13:09:35.436 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-30T13:09:35.529 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T13:09:35.531 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:35.657 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-30T13:09:35.800 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-30T13:09:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T13:09:35.905 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T13:09:35.926 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:36.406 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T13:09:36.463 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-30T13:09:36.516 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T13:09:36.557 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-30T13:09:36.617 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T13:09:36.718 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-30T13:09:36.861 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T13:09:36.968 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T13:09:36.969 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-30T13:09:36.986 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:37.113 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T13:09:37.239 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T13:09:37.518 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-30T13:09:37.644 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-30T13:09:37.779 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-30T13:09:37.905 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-30T13:09:38.031 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-30T13:09:38.156 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:38.274 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-30T13:09:38.383 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-30T13:09:38.470 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T13:09:38.596 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up dump (0.4b47-3) ... 2024-03-30T13:09:38.689 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-30T13:09:38.756 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:38.882 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T13:09:38.999 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-30T13:09:39.014 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-30T13:09:39.100 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-30T13:09:39.140 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up dump (0.4b47-3) ... 2024-03-30T13:09:39.210 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T13:09:39.233 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-30T13:09:39.283 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:39.301 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-30T13:09:39.387 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T13:09:39.392 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-30T13:09:39.502 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-30T13:09:39.513 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T13:09:39.627 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-30T13:09:39.689 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T13:09:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-30T13:09:39.815 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:39.844 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-30T13:09:39.914 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-30T13:09:39.941 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-30T13:09:40.064 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-30T13:09:40.084 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-client (14+238) ... 2024-03-30T13:09:40.193 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-30T13:09:40.249 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-30T13:09:40.375 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:40.501 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-30T13:09:40.684 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-30T13:09:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:which means they are currently configured by ifupdown: 2024-03-30T13:09:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:- enp3s0f1 2024-03-30T13:09:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-30T13:09:40.684 INFO:teuthology.orchestra.run.smithi113.stdout:remove their configuration from /etc/network/interfaces. 2024-03-30T13:09:40.685 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:40.703 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-client (14+238) ... 2024-03-30T13:09:40.805 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-30T13:09:40.854 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-30T13:09:41.121 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-30T13:09:41.361 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:41.361 INFO:teuthology.orchestra.run.smithi043.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-30T13:09:41.361 INFO:teuthology.orchestra.run.smithi043.stdout:which means they are currently configured by ifupdown: 2024-03-30T13:09:41.361 INFO:teuthology.orchestra.run.smithi043.stdout:- ens1f0 2024-03-30T13:09:41.361 INFO:teuthology.orchestra.run.smithi043.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-30T13:09:41.361 INFO:teuthology.orchestra.run.smithi043.stdout:remove their configuration from /etc/network/interfaces. 2024-03-30T13:09:41.362 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:41.441 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-30T13:09:41.502 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-30T13:09:41.834 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-30T13:09:41.929 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:42.056 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:42.145 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-30T13:09:42.182 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-30T13:09:42.308 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-common (238) ... 2024-03-30T13:09:42.648 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-30T13:09:42.808 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-30T13:09:42.934 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-30T13:09:43.060 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-common (238) ... 2024-03-30T13:09:43.164 INFO:teuthology.orchestra.run.smithi113.stdout:Adding user postgres to group ssl-cert 2024-03-30T13:09:43.362 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:43.362 INFO:teuthology.orchestra.run.smithi113.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-30T13:09:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-30T13:09:43.550 INFO:teuthology.orchestra.run.smithi113.stdout:Removing obsolete dictionary files: 2024-03-30T13:09:44.057 INFO:teuthology.orchestra.run.smithi043.stdout:Adding user postgres to group ssl-cert 2024-03-30T13:09:44.089 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-30T13:09:44.284 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:44.284 INFO:teuthology.orchestra.run.smithi043.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-30T13:09:44.465 INFO:teuthology.orchestra.run.smithi043.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-30T13:09:44.465 INFO:teuthology.orchestra.run.smithi043.stdout:Removing obsolete dictionary files: 2024-03-30T13:09:44.526 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:44.856 INFO:teuthology.orchestra.run.smithi113.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-30T13:09:44.878 INFO:teuthology.orchestra.run.smithi113.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-30T13:09:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-30T13:09:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:This user must also own the server process. 2024-03-30T13:09:44.919 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:The database cluster will be initialized with locale "en_US". 2024-03-30T13:09:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-30T13:09:44.919 INFO:teuthology.orchestra.run.smithi113.stdout:The default text search configuration will be set to "english". 2024-03-30T13:09:44.920 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:Data page checksums are disabled. 2024-03-30T13:09:44.920 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-30T13:09:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:creating subdirectories ... ok 2024-03-30T13:09:44.920 INFO:teuthology.orchestra.run.smithi113.stdout:selecting dynamic shared memory implementation ... posix 2024-03-30T13:09:44.931 INFO:teuthology.orchestra.run.smithi113.stdout:selecting default max_connections ... 100 2024-03-30T13:09:44.949 INFO:teuthology.orchestra.run.smithi113.stdout:selecting default shared_buffers ... 128MB 2024-03-30T13:09:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:selecting default time zone ... Etc/UTC 2024-03-30T13:09:44.960 INFO:teuthology.orchestra.run.smithi113.stdout:creating configuration files ... ok 2024-03-30T13:09:45.008 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-30T13:09:45.072 INFO:teuthology.orchestra.run.smithi113.stdout:running bootstrap script ... ok 2024-03-30T13:09:45.387 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-30T13:09:45.713 INFO:teuthology.orchestra.run.smithi113.stdout:performing post-bootstrap initialization ... ok 2024-03-30T13:09:45.722 INFO:teuthology.orchestra.run.smithi043.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-30T13:09:45.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-30T13:09:45.789 INFO:teuthology.orchestra.run.smithi043.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:This user must also own the server process. 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:The database cluster will be initialized with locale "en_US". 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:The default text search configuration will be set to "english". 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:Data page checksums are disabled. 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:creating subdirectories ... ok 2024-03-30T13:09:45.790 INFO:teuthology.orchestra.run.smithi043.stdout:selecting dynamic shared memory implementation ... posix 2024-03-30T13:09:45.807 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default max_connections ... 100 2024-03-30T13:09:45.827 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default shared_buffers ... 128MB 2024-03-30T13:09:45.834 INFO:teuthology.orchestra.run.smithi043.stdout:selecting default time zone ... Etc/UTC 2024-03-30T13:09:45.835 INFO:teuthology.orchestra.run.smithi043.stdout:creating configuration files ... ok 2024-03-30T13:09:45.947 INFO:teuthology.orchestra.run.smithi043.stdout:running bootstrap script ... ok 2024-03-30T13:09:46.421 INFO:teuthology.orchestra.run.smithi113.stdout:syncing data to disk ... ok 2024-03-30T13:09:46.572 INFO:teuthology.orchestra.run.smithi043.stdout:performing post-bootstrap initialization ... ok 2024-03-30T13:09:46.809 INFO:teuthology.orchestra.run.smithi113.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-30T13:09:47.574 INFO:teuthology.orchestra.run.smithi043.stdout:syncing data to disk ... ok 2024-03-30T13:09:47.976 INFO:teuthology.orchestra.run.smithi043.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-30T13:09:49.541 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:49.675 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-30T13:09:49.801 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up postgresql (14+238) ... 2024-03-30T13:09:49.919 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:50.020 INFO:teuthology.orchestra.run.smithi113.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:50.254 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T13:09:50.254 INFO:teuthology.orchestra.run.smithi113.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T13:09:50.752 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:50.839 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-30T13:09:50.912 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-30T13:09:50.940 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-30T13:09:51.063 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up postgresql (14+238) ... 2024-03-30T13:09:51.214 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-30T13:09:51.234 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T13:09:51.382 INFO:teuthology.orchestra.run.smithi043.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-30T13:09:51.394 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T13:09:51.650 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T13:09:51.650 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-30T13:09:52.201 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-30T13:09:52.294 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-30T13:09:52.612 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-30T13:09:52.706 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-30T13:09:52.781 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-30T13:09:52.814 INFO:teuthology.orchestra.run.smithi113.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-30T13:09:53.318 INFO:teuthology.orchestra.run.smithi113.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T13:09:53.323 DEBUG:teuthology.parallel:result is None 2024-03-30T13:09:54.175 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-30T13:09:54.275 INFO:teuthology.orchestra.run.smithi043.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-30T13:09:54.799 INFO:teuthology.orchestra.run.smithi043.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-30T13:09:54.804 DEBUG:teuthology.parallel:result is None 2024-03-30T13:09:54.804 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:09:55.034 DEBUG:teuthology.orchestra.run.smithi043:> dpkg-query -W -f '${Version}' ceph 2024-03-30T13:09:55.054 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-2519-gac171039-1jammy 2024-03-30T13:09:55.055 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2519-gac171039-1jammy 2024-03-30T13:09:55.055 INFO:teuthology.task.install:The correct ceph version 19.0.0-2519-gac171039-1jammy is installed. 2024-03-30T13:09:55.057 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ac1710396f9f7c389fa4b407711bc5157a8c0bf7 2024-03-30T13:09:55.199 DEBUG:teuthology.orchestra.run.smithi113:> dpkg-query -W -f '${Version}' ceph 2024-03-30T13:09:55.219 INFO:teuthology.orchestra.run.smithi113.stdout:19.0.0-2519-gac171039-1jammy 2024-03-30T13:09:55.219 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2519-gac171039-1jammy 2024-03-30T13:09:55.219 INFO:teuthology.task.install:The correct ceph version 19.0.0-2519-gac171039-1jammy is installed. 2024-03-30T13:09:55.221 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-30T13:09:55.221 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:55.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T13:09:55.237 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:55.237 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-30T13:09:55.275 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-30T13:09:55.275 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:55.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T13:09:55.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T13:09:55.352 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:55.353 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/daemon-helper 2024-03-30T13:09:55.366 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-30T13:09:55.425 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-30T13:09:55.426 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:55.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T13:09:55.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T13:09:55.492 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:55.492 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-30T13:09:55.506 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-30T13:09:55.558 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-30T13:09:55.559 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:55.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T13:09:55.573 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T13:09:55.628 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:55.628 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/usr/bin/stdin-killer 2024-03-30T13:09:55.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-30T13:09:55.702 INFO:teuthology.run_tasks:Running task ceph... 2024-03-30T13:09:55.792 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-30T13:09:55.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2024-03-30T13:09:55.795 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 777 /var/log/ceph 2024-03-30T13:09:55.808 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-30T13:09:55.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T13:09:55.851 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-30T13:09:55.863 INFO:tasks.ceph:Creating extra log directories... 2024-03-30T13:09:55.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T13:09:55.906 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-30T13:09:55.925 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-30T13:09:55.926 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-30T13:09:55.926 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-30T13:09:55.926 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:55.926 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T13:09:55.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:56.042 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:56.042 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-30T13:09:56.047 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-30T13:09:56.129 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-30T13:09:56.129 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-30T13:09:56.129 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150', 'branch': 'wip-mchangir-testing1-main-20240328-072412', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7629150', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-testing1-main-20240328-072412'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-mchangir-testing1-main-20240328-072412', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}}, 'owner': 'scheduled_mchangir@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 364, 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-mchangir-testing1-main-20240328-072412', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNaqCXgku5wm8rAduSfSpUMrux+P1koXaV2BHr2lfdroQgKNRMUBxi1nIm84HEYi0uXdSPkeSvgHbdEy4/3xDJQ=', 'smithi113.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO8cAcGOMjXJOBLP1+rNVUwlOfwTYgbuBqwTwBM8uzVGbh6LQallKPwYP2ellDlBciZriUua4cEwE3jlHVMtDnM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'ac1710396f9f7c389fa4b407711bc5157a8c0bf7', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_uninlining']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-29_13:38:54', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-30T13:09:56.129 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T13:09:56.132 DEBUG:teuthology.orchestra.run.smithi113:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-30T13:09:56.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T13:09:56.282 DEBUG:teuthology.orchestra.run.smithi113:> sudo install -d -m0777 -- /var/run/ceph 2024-03-30T13:09:56.299 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:56.299 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T13:09:56.341 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T13:09:56.341 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-03-30T13:09:56.389 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T13:09:56.390 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:56.390 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-03-30T13:09:56.390 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:56.390 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-30 13:03:23.885232062 +0000 2024-03-30T13:09:56.390 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-30 13:03:23.609234528 +0000 2024-03-30T13:09:56.390 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-30 13:03:23.609234528 +0000 2024-03-30T13:09:56.390 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-03-30T13:09:56.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T13:09:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-30T13:09:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-30T13:09:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00032432 s, 1.6 MB/s 2024-03-30T13:09:56.449 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T13:09:56.498 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-03-30T13:09:56.545 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T13:09:56.545 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:56.545 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-03-30T13:09:56.545 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:56.545 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-30 13:03:24.109230063 +0000 2024-03-30T13:09:56.546 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-30 13:03:24.105230097 +0000 2024-03-30T13:09:56.546 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-30 13:03:24.105230097 +0000 2024-03-30T13:09:56.546 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-03-30T13:09:56.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T13:09:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-30T13:09:56.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-30T13:09:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000249414 s, 2.1 MB/s 2024-03-30T13:09:56.601 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T13:09:56.650 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-03-30T13:09:56.699 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T13:09:56.699 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:56.699 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-30T13:09:56.699 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:56.699 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-30 13:03:24.877223202 +0000 2024-03-30T13:09:56.699 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-30 13:03:24.621225489 +0000 2024-03-30T13:09:56.699 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-30 13:03:24.621225489 +0000 2024-03-30T13:09:56.700 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-03-30T13:09:56.700 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T13:09:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-30T13:09:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-30T13:09:56.752 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00029942 s, 1.7 MB/s 2024-03-30T13:09:56.753 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T13:09:56.802 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-03-30T13:09:56.849 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T13:09:56.850 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:56.850 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 940 Links: 1 2024-03-30T13:09:56.850 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:56.851 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-30 13:03:25.377218737 +0000 2024-03-30T13:09:56.851 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-30 13:03:25.109221130 +0000 2024-03-30T13:09:56.851 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-30 13:03:25.109221130 +0000 2024-03-30T13:09:56.851 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2024-03-30T13:09:56.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T13:09:56.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-30T13:09:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-30T13:09:56.904 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000294507 s, 1.7 MB/s 2024-03-30T13:09:56.905 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T13:09:56.954 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:56.954 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:56.955 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/scratch_devs of=/dev/stdout 2024-03-30T13:09:56.960 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-30T13:09:56.960 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_1 2024-03-30T13:09:57.008 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-30T13:09:57.008 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:57.008 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-03-30T13:09:57.008 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:57.009 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-30 13:03:22.925440796 +0000 2024-03-30T13:09:57.009 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-30 13:03:22.649404893 +0000 2024-03-30T13:09:57.009 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-30 13:03:22.649404893 +0000 2024-03-30T13:09:57.009 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-03-30T13:09:57.009 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-30T13:09:57.061 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-30T13:09:57.062 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-30T13:09:57.062 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000363617 s, 1.4 MB/s 2024-03-30T13:09:57.063 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-30T13:09:57.112 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_2 2024-03-30T13:09:57.159 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-30T13:09:57.160 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:57.160 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 899 Links: 1 2024-03-30T13:09:57.160 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:57.160 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-30 13:03:23.405501621 +0000 2024-03-30T13:09:57.160 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-30 13:03:23.153469850 +0000 2024-03-30T13:09:57.160 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-30 13:03:23.153469850 +0000 2024-03-30T13:09:57.160 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-03-30T13:09:57.161 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-30T13:09:57.213 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-30T13:09:57.214 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-30T13:09:57.214 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000354951 s, 1.4 MB/s 2024-03-30T13:09:57.215 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-30T13:09:57.268 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_3 2024-03-30T13:09:57.315 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-30T13:09:57.315 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:57.315 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-30T13:09:57.316 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:57.316 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-30 13:03:23.905564661 +0000 2024-03-30T13:09:57.316 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-30 13:03:23.653532889 +0000 2024-03-30T13:09:57.316 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-30 13:03:23.653532889 +0000 2024-03-30T13:09:57.316 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-03-30T13:09:57.316 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-30T13:09:57.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-30T13:09:57.369 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-30T13:09:57.369 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.000451472 s, 1.1 MB/s 2024-03-30T13:09:57.371 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-30T13:09:57.421 DEBUG:teuthology.orchestra.run.smithi113:> stat /dev/vg_nvme/lv_4 2024-03-30T13:09:57.467 INFO:teuthology.orchestra.run.smithi113.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-30T13:09:57.467 INFO:teuthology.orchestra.run.smithi113.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-30T13:09:57.467 INFO:teuthology.orchestra.run.smithi113.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-03-30T13:09:57.467 INFO:teuthology.orchestra.run.smithi113.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-30T13:09:57.467 INFO:teuthology.orchestra.run.smithi113.stdout:Access: 2024-03-30 13:03:24.385623262 +0000 2024-03-30T13:09:57.467 INFO:teuthology.orchestra.run.smithi113.stdout:Modify: 2024-03-30 13:03:24.129592266 +0000 2024-03-30T13:09:57.467 INFO:teuthology.orchestra.run.smithi113.stdout:Change: 2024-03-30 13:03:24.129592266 +0000 2024-03-30T13:09:57.468 INFO:teuthology.orchestra.run.smithi113.stdout: Birth: - 2024-03-30T13:09:57.468 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-30T13:09:57.521 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records in 2024-03-30T13:09:57.521 INFO:teuthology.orchestra.run.smithi113.stderr:1+0 records out 2024-03-30T13:09:57.521 INFO:teuthology.orchestra.run.smithi113.stderr:512 bytes copied, 0.00035796 s, 1.4 MB/s 2024-03-30T13:09:57.522 DEBUG:teuthology.orchestra.run.smithi113:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-30T13:09:57.572 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:57.572 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T13:09:57.572 INFO:tasks.ceph:Generating config... 2024-03-30T13:09:57.574 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-30T13:09:57.574 INFO:tasks.ceph:[client] debug client = 20 2024-03-30T13:09:57.574 INFO:tasks.ceph:[client] debug ms = 1 2024-03-30T13:09:57.574 INFO:tasks.ceph:[client] pid file = 2024-03-30T13:09:57.574 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-30T13:09:57.574 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-30T13:09:57.574 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-30T13:09:57.574 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-30T13:09:57.574 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-30T13:09:57.574 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-30T13:09:57.574 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-30T13:09:57.574 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mds] mds_bal_replicate_threshold = 1 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-30T13:09:57.575 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-30T13:09:57.575 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-30T13:09:57.575 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-30T13:09:57.576 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-30T13:09:57.576 INFO:tasks.ceph:Setting up mon.a... 2024-03-30T13:09:57.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-30T13:09:57.612 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.keyring 2024-03-30T13:09:57.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-30T13:09:57.698 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T13:09:57.758 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.43'), ('mon.b', '172.21.15.113'), ('mon.c', '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]')] 2024-03-30T13:09:57.758 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.43,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-30T13:09:57.762 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:57.762 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T13:09:57.805 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.43 --add b 172.21.15.113 --addv c '[v2:172.21.15.113:3301,v1:172.21.15.113:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:57.892 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:57.892 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: generated fsid ace86ea7-ebbf-4c39-b9cf-d22b6fea8142 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 0 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:fsid ace86ea7-ebbf-4c39-b9cf-d22b6fea8142 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:last_changed 2024-03-30T13:09:57.890479+0000 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-03-30T13:09:57.890479+0000 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:min_mon_release 17 (quincy) 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:election_strategy: 1 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:1: [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] mon.b 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:2: [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] mon.c 2024-03-30T13:09:57.893 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-30T13:09:57.894 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-30T13:09:57.941 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ace86ea7-ebbf-4c39-b9cf-d22b6fea8142... 2024-03-30T13:09:57.942 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T13:09:57.986 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-30T13:09:58.015 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2024-03-30T13:09:58.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2024-03-30T13:09:58.017 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2024-03-30T13:09:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.43,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = ace86ea7-ebbf-4c39-b9cf-d22b6fea8142 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2024-03-30T13:09:58.019 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug trim objects = true 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2024-03-30T13:09:58.020 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 20 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 20 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 10 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2024-03-30T13:09:58.021 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: debug finisher = 20 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.022 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T13:09:58.023 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: mon down mkfs grace = 300 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: mon op complaint time = 120 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2024-03-30T13:09:58.024 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: client mount timeout = 600 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = "" 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-03-30T13:09:58.025 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout:[mds] 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds = 20 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds balancer = 20 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug frag = True 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug scatterstat = True 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: mds op complaint time = 180 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: mds verify scatter = True 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: mds_bal_replicate_threshold = 1 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-03-30T13:09:58.026 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-03-30T13:09:58.028 INFO:teuthology.orchestra.run.smithi113.stdout:[global] 2024-03-30T13:09:58.028 INFO:teuthology.orchestra.run.smithi113.stdout: chdir = "" 2024-03-30T13:09:58.028 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-30T13:09:58.028 INFO:teuthology.orchestra.run.smithi113.stdout: auth supported = cephx 2024-03-30T13:09:58.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.028 INFO:teuthology.orchestra.run.smithi113.stdout: filestore xattr use omap = true 2024-03-30T13:09:58.028 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: mon clock drift allowed = 1.000 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd crush chooseleaf type = 0 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: auth debug = true 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on old message = true 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: ms die on bug = true 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: mon pg warn max object skew = 0 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: # disable pg_autoscaler by default for new pools 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-30T13:09:58.029 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default size = 2 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow primary affinity = true 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd allow pg remap = true 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on legacy crush tunables = false 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on crush straw calc version zero = false 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on no sortbitwise = false 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on osd down out interval zero = false 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon warn on too few osds = false 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: mon_allow_pool_size_one = true 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.030 INFO:teuthology.orchestra.run.smithi113.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: osd default data pool replay window = 5 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: mon allow pool delete = true 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: mon cluster log file level = debug 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: debug asserts on shutdown = true 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: mon health detail to clog = false 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: mon host = "172.21.15.43,172.21.15.113,[v2:172.21.15.113:3301,v1:172.21.15.113:6790]" 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: fsid = ace86ea7-ebbf-4c39-b9cf-d22b6fea8142 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout:[osd] 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: osd journal size = 100 2024-03-30T13:09:58.031 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub load threshold = 5.0 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd scrub max interval = 600 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd mclock profile = high_recovery_ops 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd recover clone overlap = true 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd recovery max chunk = 1048576 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug shutdown = true 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug op order = true 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug verify stray on activate = true 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug trim objects = true 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd open classes on start = true 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug pg log writeout = true 2024-03-30T13:09:58.032 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd deep scrub update digest min age = 30 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd map max advance = 10 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: journal zero on create = true 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: filestore ondisk finisher threads = 3 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: filestore apply finisher threads = 3 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: bdev debug aio = true 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: osd debug misdirected ops = true 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore block size = 96636764160 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: bluestore fsck on mount = True 2024-03-30T13:09:58.033 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluefs = 20 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug bluestore = 20 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug osd = 20 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug rocksdb = 10 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd backfillfull_ratio = 0.85 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd full ratio = 0.9 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd nearfull ratio = 0.8 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: osd failsafe full ratio = 0.95 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: osd objectstore = bluestore 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr] 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-30T13:09:58.034 INFO:teuthology.orchestra.run.smithi113.stdout: debug mgr = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: mgr/telemetry/nag = false 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug finisher = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout:[mon] 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug mon = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug paxos = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: debug auth = 20 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon data avail warn = 5 2024-03-30T13:09:58.035 INFO:teuthology.orchestra.run.smithi113.stdout: mon mgr mkfs grace = 240 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min pgs per osd = 4 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd reporter subtree level = osd 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon osd prime pg temp = true 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon reweight min bytes per osd = 10 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: auth mon ticket ttl = 660 # 11m 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: auth service ticket ttl = 240 # 4m 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: # don't complain about insecure global_id in the test suite 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: # 1m isn't quite enough 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: mon_down_mkfs_grace = 2m 2024-03-30T13:09:58.036 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: mon_warn_on_filestore_osds = false 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: mon down mkfs grace = 300 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: mon op complaint time = 120 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout:[client] 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: rgw cache enabled = true 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable ops log = true 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: rgw enable usage log = true 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: client mount timeout = 600 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: debug client = 20 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: pid file = "" 2024-03-30T13:09:58.037 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.a] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.b] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mon.c] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout:[mds] 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds = 20 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout: debug mds balancer = 20 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout: debug ms = 1 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug frag = True 2024-03-30T13:09:58.038 INFO:teuthology.orchestra.run.smithi113.stdout: mds debug scatterstat = True 2024-03-30T13:09:58.039 INFO:teuthology.orchestra.run.smithi113.stdout: mds op complaint time = 180 2024-03-30T13:09:58.039 INFO:teuthology.orchestra.run.smithi113.stdout: mds verify scatter = True 2024-03-30T13:09:58.039 INFO:teuthology.orchestra.run.smithi113.stdout: mds_bal_replicate_threshold = 1 2024-03-30T13:09:58.039 INFO:teuthology.orchestra.run.smithi113.stdout: osd op complaint time = 180 2024-03-30T13:09:58.039 INFO:teuthology.orchestra.run.smithi113.stdout: rados mon op timeout = 900 2024-03-30T13:09:58.039 INFO:teuthology.orchestra.run.smithi113.stdout: rados osd op timeout = 900 2024-03-30T13:09:58.039 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-30T13:09:58.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-30T13:09:58.122 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-30T13:09:58.123 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:58.123 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-30T13:09:58.128 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:58.128 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-30T13:09:58.173 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:09:58.174 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:58.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T13:09:58.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T13:09:58.236 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:09:58.236 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:58.285 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:09:58.285 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:58.285 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-30T13:09:58.285 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-30T13:09:58.308 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:09:58.309 DEBUG:teuthology.orchestra.run.smithi113:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:09:58.355 INFO:tasks.ceph:Setting up mon nodes... 2024-03-30T13:09:58.356 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-30T13:09:58.356 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T13:09:58.399 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-30T13:09:58.401 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T13:09:58.444 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-30T13:09:58.460 INFO:tasks.ceph:Setting up mds nodes... 2024-03-30T13:09:58.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T13:09:58.501 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-30T13:09:58.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-30T13:09:58.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T13:09:58.643 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-30T13:09:58.645 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-30T13:09:58.700 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T13:09:58.745 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-30T13:09:58.748 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-30T13:09:58.808 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T13:09:58.890 INFO:teuthology.orchestra.run.smithi113.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-30T13:09:58.893 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-30T13:09:58.945 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-30T13:09:58.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-30T13:09:58.982 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-30T13:09:58.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-30T13:09:59.070 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-30T13:09:59.083 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-30T13:09:59.117 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-30T13:09:59.129 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-30T13:09:59.202 INFO:teuthology.orchestra.run.smithi113.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-30T13:09:59.212 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-30T13:09:59.212 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi113.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-30T13:09:59.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-30T13:09:59.227 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:59.227 INFO:tasks.ceph:role: osd.0 2024-03-30T13:09:59.228 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:09:59.228 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:09:59.284 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:09:59.308 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-03-30T13:09:59.309 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-03-30T13:09:59.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-30T13:09:59.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-30T13:09:59.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:09:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:09:59.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-30T13:09:59.470 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:59.471 INFO:tasks.ceph:role: osd.1 2024-03-30T13:09:59.471 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:09:59.471 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T13:09:59.524 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:09:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:09:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:09:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:09:59.524 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:09:59.524 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:09:59.524 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:09:59.525 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:09:59.525 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:09:59.525 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:09:59.548 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-03-30T13:09:59.550 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-03-30T13:09:59.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-30T13:09:59.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-30T13:09:59.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:09:59.663 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:09:59.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-30T13:09:59.716 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:59.716 INFO:tasks.ceph:role: osd.2 2024-03-30T13:09:59.716 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:09:59.717 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T13:09:59.776 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:09:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:09:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:09:59.776 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:09:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:09:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:09:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:09:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:09:59.777 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:09:59.777 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:09:59.800 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-03-30T13:09:59.801 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-03-30T13:09:59.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-30T13:09:59.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-30T13:09:59.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:09:59.914 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:09:59.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-30T13:09:59.967 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-30T13:09:59.968 INFO:tasks.ceph:role: osd.3 2024-03-30T13:09:59.968 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:09:59.968 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T13:10:00.022 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:10:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:10:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:10:00.022 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:10:00.022 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:10:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:10:00.023 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:10:00.023 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:10:00.023 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:10:00.023 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:10:00.046 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-03-30T13:10:00.048 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-03-30T13:10:00.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-30T13:10:00.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-30T13:10:00.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:00.162 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:10:00.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:00.260+0000 7ff455c637c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-30T13:10:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:00.264+0000 7ff455c637c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-30T13:10:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:00.264+0000 7ff455c637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T13:10:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:00.264+0000 7ff455c637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T13:10:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:00.264+0000 7ff455c637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-30T13:10:00.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:00.264+0000 7ff455c637c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-30T13:10:03.240 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-30T13:10:03.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:03.332+0000 7f91389a77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-30T13:10:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:03.332+0000 7f91389a77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-30T13:10:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:03.332+0000 7f91389a77c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T13:10:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:03.332+0000 7f91389a77c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T13:10:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:03.332+0000 7f91389a77c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-30T13:10:03.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:03.332+0000 7f91389a77c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-30T13:10:06.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-30T13:10:06.366 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:06.448+0000 7f2dc0fca7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-30T13:10:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:06.448+0000 7f2dc0fca7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-30T13:10:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:06.448+0000 7f2dc0fca7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T13:10:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:06.448+0000 7f2dc0fca7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T13:10:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:06.448+0000 7f2dc0fca7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-30T13:10:06.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:06.448+0000 7f2dc0fca7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-30T13:10:09.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-30T13:10:09.498 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:09.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:09.576+0000 7f63e81177c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-30T13:10:09.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:09.576+0000 7f63e81177c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-30T13:10:09.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:09.576+0000 7f63e81177c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T13:10:09.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:09.576+0000 7f63e81177c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T13:10:09.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:09.576+0000 7f63e81177c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-30T13:10:09.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:09.576+0000 7f63e81177c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-30T13:10:12.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-30T13:10:12.614 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-30T13:10:12.629 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:10:12.629 INFO:tasks.ceph:role: osd.4 2024-03-30T13:10:12.629 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:10:12.629 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-30T13:10:12.690 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:10:12.690 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:10:12.690 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:10:12.690 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:10:12.690 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:10:12.690 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:10:12.690 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:10:12.691 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:10:12.691 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:10:12.691 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:10:13.186 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-30T13:10:13.188 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-30T13:10:13.189 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-30T13:10:13.304 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-30T13:10:13.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:13.316 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:10:13.317 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-30T13:10:13.370 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:10:13.370 INFO:tasks.ceph:role: osd.5 2024-03-30T13:10:13.370 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:10:13.371 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:10:13.436 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:10:13.437 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:10:13.437 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:10:13.934 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-30T13:10:13.937 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-30T13:10:13.937 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-30T13:10:14.049 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-30T13:10:14.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:14.061 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:10:14.061 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-30T13:10:14.114 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:10:14.115 INFO:tasks.ceph:role: osd.6 2024-03-30T13:10:14.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:10:14.115 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-30T13:10:14.180 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:10:14.180 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:10:14.180 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:10:14.180 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:10:14.180 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:10:14.180 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:10:14.181 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:10:14.181 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:10:14.181 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:10:14.181 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:10:14.680 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-30T13:10:14.682 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-30T13:10:14.682 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-30T13:10:14.792 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-30T13:10:14.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:14.804 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:10:14.804 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-30T13:10:14.859 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-30T13:10:14.859 INFO:tasks.ceph:role: osd.7 2024-03-30T13:10:14.859 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:10:14.859 DEBUG:teuthology.orchestra.run.smithi113:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-30T13:10:14.923 INFO:teuthology.orchestra.run.smithi113.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-30T13:10:14.923 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-30T13:10:14.923 INFO:teuthology.orchestra.run.smithi113.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-30T13:10:14.923 INFO:teuthology.orchestra.run.smithi113.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-30T13:10:14.923 INFO:teuthology.orchestra.run.smithi113.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-30T13:10:14.923 INFO:teuthology.orchestra.run.smithi113.stdout: = sunit=0 swidth=0 blks 2024-03-30T13:10:14.923 INFO:teuthology.orchestra.run.smithi113.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-30T13:10:14.924 INFO:teuthology.orchestra.run.smithi113.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-30T13:10:14.924 INFO:teuthology.orchestra.run.smithi113.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-30T13:10:14.924 INFO:teuthology.orchestra.run.smithi113.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-30T13:10:15.428 INFO:teuthology.orchestra.run.smithi113.stdout:Discarding blocks...Done. 2024-03-30T13:10:15.430 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi113.front.sepia.ceph.com -o noatime 2024-03-30T13:10:15.430 DEBUG:teuthology.orchestra.run.smithi113:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-30T13:10:15.553 DEBUG:teuthology.orchestra.run.smithi113:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-30T13:10:15.564 INFO:teuthology.orchestra.run.smithi113.stderr:sudo: /sbin/restorecon: command not found 2024-03-30T13:10:15.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:10:15.565 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:15.675 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:15.670+0000 7fa00d0237c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-30T13:10:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:15.674+0000 7fa00d0237c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-30T13:10:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:15.674+0000 7fa00d0237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T13:10:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:15.674+0000 7fa00d0237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T13:10:15.679 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:15.674+0000 7fa00d0237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-30T13:10:15.682 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:15.674+0000 7fa00d0237c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-30T13:10:18.659 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-30T13:10:18.673 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:18.754 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:18.746+0000 7f4c8d53e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-30T13:10:18.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:18.750+0000 7f4c8d53e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-30T13:10:18.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:18.750+0000 7f4c8d53e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T13:10:18.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:18.750+0000 7f4c8d53e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T13:10:18.755 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:18.750+0000 7f4c8d53e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-30T13:10:18.756 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:18.750+0000 7f4c8d53e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-30T13:10:21.722 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-30T13:10:21.776 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:21.857 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:21.850+0000 7f7a4bec77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-30T13:10:21.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:21.850+0000 7f7a4bec77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-30T13:10:21.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:21.850+0000 7f7a4bec77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T13:10:21.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:21.850+0000 7f7a4bec77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T13:10:21.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:21.850+0000 7f7a4bec77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-30T13:10:21.858 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:21.850+0000 7f7a4bec77c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-30T13:10:24.835 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-30T13:10:24.892 DEBUG:teuthology.orchestra.run.smithi113:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:24.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:24.970+0000 7f9e83b067c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-30T13:10:24.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:24.970+0000 7f9e83b067c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-30T13:10:24.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:24.970+0000 7f9e83b067c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T13:10:24.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:24.970+0000 7f9e83b067c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T13:10:24.974 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:24.970+0000 7f9e83b067c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-30T13:10:24.975 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:24.974+0000 7f9e83b067c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-30T13:10:26.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:26.134 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:27.959 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-30T13:10:27.972 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-30T13:10:27.972 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:27.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-30T13:10:27.987 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:27.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-30T13:10:28.042 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:28.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-30T13:10:28.096 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:28.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-30T13:10:28.153 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:28.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-30T13:10:28.210 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:28.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-30T13:10:28.265 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:28.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-30T13:10:28.321 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.321 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-30T13:10:28.336 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.336 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-30T13:10:28.391 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.392 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-30T13:10:28.448 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.448 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-30T13:10:28.504 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.504 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-30T13:10:28.559 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.559 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-30T13:10:28.615 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.615 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-30T13:10:28.671 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:28.671 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-30T13:10:28.676 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:28.677 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-30T13:10:28.726 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.726 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-30T13:10:28.731 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:28.731 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-30T13:10:28.780 INFO:tasks.ceph:Adding keys to all mons... 2024-03-30T13:10:28.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T13:10:28.782 DEBUG:teuthology.orchestra.run.smithi113:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-30T13:10:28.826 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2024-03-30T13:10:28.826 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhmpMenFxAA0TnJp3fiTI61U2P/GDktCQ== 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhmqjrKHRAAjgqPklYwNqmXlXxyG2hCDg== 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhmv5Q0JhAAFNDkQ/t0fNOi08HbPPZ51g== 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAoDwhmodjkDxAAnARCBa5r+k4VpsrBHZJPdA== 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQArDwhmh68CFBAAwSBpkmAtLrFsUUPDlfLsPg== 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAuDwhmWpP5GhAAOxvQoZdxhmbslCPu3aB+sw== 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAxDwhmKGaFIhAAggPT08CCwHt3qW8OCLR7Xw== 2024-03-30T13:10:28.827 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhmD5t8GhAAhvu7f7K2maRFCWA/n1i6bQ== 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhmszFnLBAAQW0A5lFQf9NCopOQYF0KgA== 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhm7Y4NNRAAUfV2sqtMJLlNrUjGSOeMPQ== 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQA3DwhmcUBAKBAAnRIdvtxsWe8tCKU74LjnGQ== 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQA6DwhmWILuLBAA1SK/P0b7xFhvb1Nw6Etn2w== 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQA9Dwhm+c8JMxAACTA0hZnr45rL7JeRXc8EwA== 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2024-03-30T13:10:28.828 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBADwhmDuL9ORAA/qZhjTe4/hHKQe+VNHvjxg== 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhmDh1uOhAAyQ23sd4h6j2HVM6D/hX5WA== 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout:[client.2] 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAnDwhmCPsZBBAARwmR4u1Gdq/f7UsCnCHyEg== 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAnDwhm7Sz7BhAAV0khY4znoozzYORr7aJISQ== 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout:[client.3] 2024-03-30T13:10:28.829 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAnDwhmqHP/CxAAyKG1FC4XCpBgsEHA/lKHYg== 2024-03-30T13:10:28.834 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.y] 2024-03-30T13:10:28.834 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAmDwhmpMenFxAA0TnJp3fiTI61U2P/GDktCQ== 2024-03-30T13:10:28.834 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.a] 2024-03-30T13:10:28.834 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAmDwhmqjrKHRAAjgqPklYwNqmXlXxyG2hCDg== 2024-03-30T13:10:28.834 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.c] 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAmDwhmv5Q0JhAAFNDkQ/t0fNOi08HbPPZ51g== 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.0] 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAoDwhmodjkDxAAnARCBa5r+k4VpsrBHZJPdA== 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.1] 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQArDwhmh68CFBAAwSBpkmAtLrFsUUPDlfLsPg== 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.2] 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAuDwhmWpP5GhAAOxvQoZdxhmbslCPu3aB+sw== 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.3] 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAxDwhmKGaFIhAAggPT08CCwHt3qW8OCLR7Xw== 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout:[mgr.x] 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAmDwhmD5t8GhAAhvu7f7K2maRFCWA/n1i6bQ== 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.b] 2024-03-30T13:10:28.835 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAmDwhmszFnLBAAQW0A5lFQf9NCopOQYF0KgA== 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout:[mds.d] 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAmDwhm7Y4NNRAAUfV2sqtMJLlNrUjGSOeMPQ== 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.4] 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQA3DwhmcUBAKBAAnRIdvtxsWe8tCKU74LjnGQ== 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.5] 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQA6DwhmWILuLBAA1SK/P0b7xFhvb1Nw6Etn2w== 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.6] 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQA9Dwhm+c8JMxAACTA0hZnr45rL7JeRXc8EwA== 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout:[osd.7] 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQBADwhmDuL9ORAA/qZhjTe4/hHKQe+VNHvjxg== 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout:[client.0] 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAmDwhmDh1uOhAAyQ23sd4h6j2HVM6D/hX5WA== 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout:[client.2] 2024-03-30T13:10:28.836 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAnDwhmCPsZBBAARwmR4u1Gdq/f7UsCnCHyEg== 2024-03-30T13:10:28.837 INFO:teuthology.orchestra.run.smithi113.stdout:[client.1] 2024-03-30T13:10:28.837 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAnDwhm7Sz7BhAAV0khY4znoozzYORr7aJISQ== 2024-03-30T13:10:28.837 INFO:teuthology.orchestra.run.smithi113.stdout:[client.3] 2024-03-30T13:10:28.837 INFO:teuthology.orchestra.run.smithi113.stdout: key = AQAnDwhmqHP/CxAAyKG1FC4XCpBgsEHA/lKHYg== 2024-03-30T13:10:28.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:10:28.875 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:10:28.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:28.932 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:28.978 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:29.022 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:29.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.071 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.154 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.234 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.318 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:10:29.402 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-30T13:10:29.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:29.490 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:29.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:29.555 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-30T13:10:29.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.641 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.726 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.894 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-30T13:10:29.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:29.976 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:30.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:30.066 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:30.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:30.118 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:30.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:30.202 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-30T13:10:30.276 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-30T13:10:30.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-30T13:10:30.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T13:10:30.397 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-30T13:10:30.408 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-30T13:10:30.422 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T13:10:30.512 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-30T13:10:30.526 DEBUG:teuthology.orchestra.run.smithi113:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-30T13:10:30.577 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-30T13:10:30.663 DEBUG:teuthology.orchestra.run.smithi113:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-30T13:10:30.714 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:30.716 DEBUG:teuthology.orchestra.run.smithi113:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-30T13:10:30.760 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-30T13:10:30.760 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-30T13:10:30.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-30T13:10:30.766 INFO:tasks.ceph.mon.a:Started 2024-03-30T13:10:30.767 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-30T13:10:30.767 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-30T13:10:30.809 INFO:tasks.ceph.mon.b:Started 2024-03-30T13:10:30.810 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-30T13:10:30.810 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-30T13:10:30.813 INFO:tasks.ceph.mon.c:Started 2024-03-30T13:10:30.813 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-30T13:10:30.813 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-30T13:10:30.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-30T13:10:30.817 INFO:tasks.ceph.mgr.y:Started 2024-03-30T13:10:30.817 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-30T13:10:30.818 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-30T13:10:30.821 INFO:tasks.ceph.mgr.x:Started 2024-03-30T13:10:30.821 INFO:tasks.ceph:Setting crush tunables to default 2024-03-30T13:10:30.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd crush tunables default 2024-03-30T13:10:30.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff622bb8640 1 Processor -- start 2024-03-30T13:10:30.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff622bb8640 1 -- start start 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff622bb8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff622bb8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff61c06c850 con 0x7ff61c15e4c0 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff622bb8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff61c06cb30 con 0x7ff61c0b9e30 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff622bb8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff61c0ba290 con 0x7ff61c0cf6c0 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff620954640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff61c15e4c0 0x7ff61c06c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:42180/0 (socket says 172.21.15.43:42180) 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff620954640 1 -- 172.21.15.43:0/2960257544 learned_addr learned my addr 172.21.15.43:0/2960257544 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff621155640 1 --2- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff621155640 1 -- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 msgr2=0x7ff61c169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff621155640 1 -- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 msgr2=0x7ff61c169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff621155640 1 --2- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T13:10:30.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:30.932+0000 7ff621155640 1 --2- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-30T13:10:31.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.136+0000 7ff621155640 1 --2- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.136+0000 7ff621155640 1 -- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 msgr2=0x7ff61c169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-30T13:10:31.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.136+0000 7ff621155640 1 -- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 msgr2=0x7ff61c169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-30T13:10:31.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.136+0000 7ff621155640 1 --2- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-30T13:10:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.136+0000 7ff621155640 1 --2- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-30T13:10:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.168+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2434853704 0 0) 0x7ff61c06c850 con 0x7ff61c15e4c0 2024-03-30T13:10:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.168+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff604003530 con 0x7ff61c15e4c0 2024-03-30T13:10:31.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.168+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1208117391 0 0) 0x7ff61c06cb30 con 0x7ff61c0b9e30 2024-03-30T13:10:31.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.168+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff61c06c850 con 0x7ff61c0b9e30 2024-03-30T13:10:31.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 477019530 0 0) 0x7ff61c06c850 con 0x7ff61c0b9e30 2024-03-30T13:10:31.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 msgr2=0x7ff61c169030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:10:31.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 --2- 172.21.15.43:0/2960257544 >> v2:172.21.15.43:3300/0 conn(0x7ff61c0cf6c0 0x7ff61c169030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 >> v1:172.21.15.43:6789/0 conn(0x7ff61c15e4c0 legacy=0x7ff61c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c06ce10 con 0x7ff61c0b9e30 2024-03-30T13:10:31.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ff610001680 con 0x7ff61c0b9e30 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 >> v1:172.21.15.113:6789/0 conn(0x7ff61c0b9e30 legacy=0x7ff61c0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff604003ba0 0x7ff604003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604004810 0x7ff604004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604005220 0x7ff60400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff60400c120 con 0x7ff604004810 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff60400c2d0 con 0x7ff604003ba0 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff60400c4d0 con 0x7ff604005220 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61bfff640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604004810 0x7ff604004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff621155640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604005220 0x7ff60400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff620954640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff604003ba0 0x7ff604003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61bfff640 1 -- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604005220 msgr2=0x7ff60400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61bfff640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604005220 0x7ff60400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61bfff640 1 -- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff604003ba0 msgr2=0x7ff604003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61bfff640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff604003ba0 0x7ff604003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff61bfff640 1 -- 172.21.15.43:0/2960257544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c06c850 con 0x7ff604004810 2024-03-30T13:10:31.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.172+0000 7ff621155640 1 -- 172.21.15.43:0/2960257544 reap_dead start 2024-03-30T13:10:31.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff61bfff640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604004810 0x7ff604004c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff610007eb0 tx=0x7ff61001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c349da003dea5cce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100b5070 con 0x7ff604004810 2024-03-30T13:10:31.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6100a8d50 con 0x7ff604004810 2024-03-30T13:10:31.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff61b7fe640 1 -- 172.21.15.43:0/2960257544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610007b00 con 0x7ff604004810 2024-03-30T13:10:31.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff622bb8640 1 -- 172.21.15.43:0/2960257544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604004810 msgr2=0x7ff604004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff622bb8640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604004810 0x7ff604004c90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff610007eb0 tx=0x7ff61001f110 comp rx=0 tx=0).stop 2024-03-30T13:10:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff622bb8640 1 -- 172.21.15.43:0/2960257544 shutdown_connections 2024-03-30T13:10:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff622bb8640 1 --2- 172.21.15.43:0/2960257544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604004810 0x7ff604004c90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.180+0000 7ff622bb8640 1 -- 172.21.15.43:0/2960257544 >> 172.21.15.43:0/2960257544 conn(0x7ff61c0b6410 msgr2=0x7ff61c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.184+0000 7ff622bb8640 1 -- 172.21.15.43:0/2960257544 shutdown_connections 2024-03-30T13:10:31.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.184+0000 7ff622bb8640 1 -- 172.21.15.43:0/2960257544 wait complete. 2024-03-30T13:10:31.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.188+0000 7ff622bb8640 1 Processor -- start 2024-03-30T13:10:31.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.188+0000 7ff622bb8640 1 -- start start 2024-03-30T13:10:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604004810 0x7ff61c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff61c159ed0 0x7ff61c0cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61c159690 0x7ff61c0d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c0cf340 con 0x7ff61c159690 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff61c0cf1c0 con 0x7ff61c159ed0 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff61c0cf4c0 con 0x7ff604004810 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61c159690 0x7ff61c0d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61c159690 0x7ff61c0d2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60444/0 (socket says 172.21.15.43:60444) 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 -- 172.21.15.43:0/54031764 learned_addr learned my addr 172.21.15.43:0/54031764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff61bfff640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff61c159ed0 0x7ff61c0cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff620954640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604004810 0x7ff61c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 -- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604004810 msgr2=0x7ff61c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604004810 0x7ff61c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 -- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff61c159ed0 msgr2=0x7ff61c0cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff61c159ed0 0x7ff61c0cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 -- 172.21.15.43:0/54031764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61c15a750 con 0x7ff61c159690 2024-03-30T13:10:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff621155640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61c159690 0x7ff61c0d2bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff614009610 tx=0x7ff614007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff6197fa640 1 -- 172.21.15.43:0/54031764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614016660 con 0x7ff61c159690 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff6197fa640 1 -- 172.21.15.43:0/54031764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61401c070 con 0x7ff61c159690 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff6197fa640 1 -- 172.21.15.43:0/54031764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614016a60 con 0x7ff61c159690 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff61c161370 con 0x7ff61c159690 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61c0cda50 con 0x7ff61c159690 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff6197fa640 1 -- 172.21.15.43:0/54031764 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff614016c00 con 0x7ff61c159690 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff6197fa640 1 -- 172.21.15.43:0/54031764 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff614021cc0 con 0x7ff61c159690 2024-03-30T13:10:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.192+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61c068900 con 0x7ff61c159690 2024-03-30T13:10:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.208+0000 7ff6197fa640 1 -- 172.21.15.43:0/54031764 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff61c15a750 con 0x7ff61c159690 2024-03-30T13:10:31.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.276+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff61c0c7a00 con 0x7ff61c159690 2024-03-30T13:10:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.276+0000 7ff6197fa640 1 -- 172.21.15.43:0/54031764 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ff614021460 con 0x7ff61c159690 2024-03-30T13:10:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61c159690 msgr2=0x7ff61c0d2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61c159690 0x7ff61c0d2bc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff614009610 tx=0x7ff614007700 comp rx=0 tx=0).stop 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 shutdown_connections 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff61c159690 0x7ff61c0d2bc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff61c159ed0 0x7ff61c0cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 --2- 172.21.15.43:0/54031764 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff604004810 0x7ff61c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 >> 172.21.15.43:0/54031764 conn(0x7ff61c0b6410 msgr2=0x7ff61c0d4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 shutdown_connections 2024-03-30T13:10:31.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:31.280+0000 7ff622bb8640 1 -- 172.21.15.43:0/54031764 wait complete. 2024-03-30T13:10:31.308 INFO:tasks.ceph:check_enable_crimson: False 2024-03-30T13:10:31.308 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-30T13:10:31.309 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:31.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-30T13:10:31.321 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:31.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-30T13:10:31.376 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:31.376 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-30T13:10:31.409 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:31.404+0000 7fa7e3a4e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T13:10:31.420 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:10:31.420 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-30T13:10:31.460 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:31.457+0000 7fd36e30d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-30T13:10:31.476 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:31.476 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-30T13:10:31.489 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:31.489 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-30T13:10:31.543 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:31.543 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-30T13:10:31.599 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:10:31.599 DEBUG:teuthology.orchestra.run.smithi113:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-30T13:10:31.655 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new f1e4c298-2105-4de8-a9a5-4b8e41a07a47 0 2024-03-30T13:10:31.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.817+0000 7fd2a93a6640 1 Processor -- start 2024-03-30T13:10:31.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.817+0000 7fd2a93a6640 1 -- start start 2024-03-30T13:10:31.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.817+0000 7fd2a93a6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2a415b190 0x7fd2a40ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.817+0000 7fd2a93a6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a40b3820 con 0x7fd2a40afc00 2024-03-30T13:10:31.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.817+0000 7fd2a93a6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2a406ef80 con 0x7fd2a406fd40 2024-03-30T13:10:31.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.817+0000 7fd2a93a6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a40b4110 con 0x7fd2a415b190 2024-03-30T13:10:31.825 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.817+0000 7fd2a2ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2a415b190 0x7fd2a40ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a2ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd2a415b190 0x7fd2a40ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42074/0 (socket says 172.21.15.113:42074) 2024-03-30T13:10:31.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a2ffd640 1 -- 172.21.15.113:0/3221307694 learned_addr learned my addr 172.21.15.113:0/3221307694 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:31.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a2ffd640 1 -- 172.21.15.113:0/3221307694 >> v1:172.21.15.113:6789/0 conn(0x7fd2a406fd40 legacy=0x7fd2a40b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a2ffd640 1 -- 172.21.15.113:0/3221307694 >> v1:172.21.15.43:6789/0 conn(0x7fd2a40afc00 legacy=0x7fd2a406f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:31.826 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a2ffd640 1 -- 172.21.15.113:0/3221307694 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a40b3ac0 con 0x7fd2a415b190 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a2ffd640 1 --2- 172.21.15.113:0/3221307694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a415b190 0x7fd2a40ce220 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40b0420 tx=0x7fd29c01c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=418cef3d912aea1d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a1ffb640 1 -- 172.21.15.113:0/3221307694 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0a5800 con 0x7fd2a415b190 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a1ffb640 1 -- 172.21.15.113:0/3221307694 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd29c0a59a0 con 0x7fd2a415b190 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a1ffb640 1 -- 172.21.15.113:0/3221307694 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0a5d70 con 0x7fd2a415b190 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a93a6640 1 -- 172.21.15.113:0/3221307694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a415b190 msgr2=0x7fd2a40ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a93a6640 1 --2- 172.21.15.113:0/3221307694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a415b190 0x7fd2a40ce220 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40b0420 tx=0x7fd29c01c9a0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a93a6640 1 -- 172.21.15.113:0/3221307694 shutdown_connections 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a93a6640 1 --2- 172.21.15.113:0/3221307694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a415b190 0x7fd2a40ce220 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.828 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a93a6640 1 -- 172.21.15.113:0/3221307694 >> 172.21.15.113:0/3221307694 conn(0x7fd2a40d0c90 msgr2=0x7fd2a40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a93a6640 1 -- 172.21.15.113:0/3221307694 shutdown_connections 2024-03-30T13:10:31.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.821+0000 7fd2a93a6640 1 -- 172.21.15.113:0/3221307694 wait complete. 2024-03-30T13:10:31.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 Processor -- start 2024-03-30T13:10:31.829 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 -- start start 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2a406fd40 0x7fd2a40cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a40afc00 0x7fd2a413b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2a415b190 0x7fd2a4150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a41692b0 con 0x7fd2a406fd40 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a4169130 con 0x7fd2a40afc00 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a4169430 con 0x7fd2a415b190 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a40afc00 0x7fd2a413b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2a415b190 0x7fd2a4150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2a415b190 0x7fd2a4150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:42540/0 (socket says 172.21.15.113:42540) 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a27fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a40afc00 0x7fd2a413b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42076/0 (socket says 172.21.15.113:42076) 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 -- 172.21.15.113:0/1556512179 learned_addr learned my addr 172.21.15.113:0/1556512179 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a2ffd640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2a406fd40 0x7fd2a40cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 -- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a40afc00 msgr2=0x7fd2a413b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a40afc00 0x7fd2a413b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 -- 172.21.15.113:0/1556512179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2a406fd40 msgr2=0x7fd2a40cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2a406fd40 0x7fd2a40cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 -- 172.21.15.113:0/1556512179 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a414fba0 con 0x7fd2a415b190 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a27fc640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a40afc00 0x7fd2a413b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:31.832 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a37fe640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2a415b190 0x7fd2a4150c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c400 tx=0x7fd29400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:31.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd28b7fe640 1 -- 172.21.15.113:0/1556512179 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294017070 con 0x7fd2a415b190 2024-03-30T13:10:31.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd28b7fe640 1 -- 172.21.15.113:0/1556512179 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2940052d0 con 0x7fd2a415b190 2024-03-30T13:10:31.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a414fe80 con 0x7fd2a415b190 2024-03-30T13:10:31.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd28b7fe640 1 -- 172.21.15.113:0/1556512179 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294003800 con 0x7fd2a415b190 2024-03-30T13:10:31.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a4150440 con 0x7fd2a415b190 2024-03-30T13:10:31.833 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd28b7fe640 1 -- 172.21.15.113:0/1556512179 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd29401a070 con 0x7fd2a415b190 2024-03-30T13:10:31.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd28b7fe640 1 -- 172.21.15.113:0/1556512179 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fd29401c440 con 0x7fd2a415b190 2024-03-30T13:10:31.834 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.825+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a4068900 con 0x7fd2a415b190 2024-03-30T13:10:31.841 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.837+0000 7fd28b7fe640 1 -- 172.21.15.113:0/1556512179 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd294011590 con 0x7fd2a415b190 2024-03-30T13:10:31.909 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.905+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f1e4c298-2105-4de8-a9a5-4b8e41a07a47", "id": 0} v 0) -- 0x7fd2a40758a0 con 0x7fd2a415b190 2024-03-30T13:10:31.916 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:31.908+0000 7fa7e3a4e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T13:10:31.916 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd28b7fe640 1 -- 172.21.15.113:0/1556512179 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f1e4c298-2105-4de8-a9a5-4b8e41a07a47", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fd294011770 con 0x7fd2a415b190 2024-03-30T13:10:31.916 INFO:teuthology.orchestra.run.smithi113.stdout:0 2024-03-30T13:10:31.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2a415b190 msgr2=0x7fd2a4150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:31.917 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2a415b190 0x7fd2a4150c40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c400 tx=0x7fd29400c8c0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 shutdown_connections 2024-03-30T13:10:31.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2a415b190 0x7fd2a4150c40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd2a40afc00 0x7fd2a413b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 --2- 172.21.15.113:0/1556512179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2a406fd40 0x7fd2a40cff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:31.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 >> 172.21.15.113:0/1556512179 conn(0x7fd2a40d0c90 msgr2=0x7fd2a40b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:31.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 shutdown_connections 2024-03-30T13:10:31.918 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:31.913+0000 7fd2a93a6640 1 -- 172.21.15.113:0/1556512179 wait complete. 2024-03-30T13:10:31.934 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 46a54234-af69-46ba-8c50-436769f3e1f1 1 2024-03-30T13:10:31.954 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:31.949+0000 7fd36e30d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-30T13:10:32.054 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.049+0000 7f6c8fc52640 1 Processor -- start 2024-03-30T13:10:32.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.049+0000 7f6c8fc52640 1 -- start start 2024-03-30T13:10:32.055 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.049+0000 7f6c8fc52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c880b46f0 0x7f6c8806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c880b4ba0 con 0x7f6c8806ff20 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c8815ec60 con 0x7f6c8815afc0 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6c88070c30 con 0x7f6c880b46f0 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8d1ed640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c880b46f0 0x7f6c8806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8d9ee640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f6c8815afc0 0x7f6c880b3fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51664/0 (socket says 172.21.15.113:51664) 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8d9ee640 1 -- 172.21.15.113:0/2006857835 learned_addr learned my addr 172.21.15.113:0/2006857835 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8c9ec640 1 -- 172.21.15.113:0/2006857835 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2094937212 0 0) 0x7f6c8815ec60 con 0x7f6c8815afc0 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8d1ed640 1 -- 172.21.15.113:0/2006857835 >> v1:172.21.15.113:6789/0 conn(0x7f6c8815afc0 legacy=0x7f6c880b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8d1ed640 1 -- 172.21.15.113:0/2006857835 >> v1:172.21.15.43:6789/0 conn(0x7f6c8806ff20 legacy=0x7f6c880ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.056 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8d1ed640 1 -- 172.21.15.113:0/2006857835 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8815ef40 con 0x7f6c880b46f0 2024-03-30T13:10:32.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8d1ed640 1 --2- 172.21.15.113:0/2006857835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c880b46f0 0x7f6c8806f9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c780097f0 tx=0x7f6c7801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2dc1fcb57ea1e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8c9ec640 1 -- 172.21.15.113:0/2006857835 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c780b2070 con 0x7f6c880b46f0 2024-03-30T13:10:32.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8c9ec640 1 -- 172.21.15.113:0/2006857835 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c780ad640 con 0x7f6c880b46f0 2024-03-30T13:10:32.058 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8c9ec640 1 -- 172.21.15.113:0/2006857835 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c780ada10 con 0x7f6c880b46f0 2024-03-30T13:10:32.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- 172.21.15.113:0/2006857835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c880b46f0 msgr2=0x7f6c8806f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 --2- 172.21.15.113:0/2006857835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c880b46f0 0x7f6c8806f9e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c780097f0 tx=0x7f6c7801c4a0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- 172.21.15.113:0/2006857835 shutdown_connections 2024-03-30T13:10:32.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 --2- 172.21.15.113:0/2006857835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c880b46f0 0x7f6c8806f9e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- 172.21.15.113:0/2006857835 >> 172.21.15.113:0/2006857835 conn(0x7f6c880d1070 msgr2=0x7f6c880d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- 172.21.15.113:0/2006857835 shutdown_connections 2024-03-30T13:10:32.059 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.053+0000 7f6c8fc52640 1 -- 172.21.15.113:0/2006857835 wait complete. 2024-03-30T13:10:32.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 Processor -- start 2024-03-30T13:10:32.060 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 -- start start 2024-03-30T13:10:32.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8806ff20 0x7f6c8814e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.062 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c880b46f0 0x7f6c8815df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8815afc0 0x7f6c8814fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8815b520 con 0x7f6c880b46f0 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8815b3a0 con 0x7f6c8815afc0 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6c88169010 con 0x7f6c8806ff20 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8806ff20 0x7f6c8814e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8806ff20 0x7f6c8814e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:42558/0 (socket says 172.21.15.113:42558) 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 -- 172.21.15.113:0/3084489442 learned_addr learned my addr 172.21.15.113:0/3084489442 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8e1ef640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8815afc0 0x7f6c8814fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d1ed640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c880b46f0 0x7f6c8815df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 -- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8815afc0 msgr2=0x7f6c8814fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8815afc0 0x7f6c8814fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 -- 172.21.15.113:0/3084489442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c880b46f0 msgr2=0x7f6c8815df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c880b46f0 0x7f6c8815df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.063 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 -- 172.21.15.113:0/3084489442 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8815b740 con 0x7f6c8806ff20 2024-03-30T13:10:32.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8d9ee640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8806ff20 0x7f6c8814e9a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c4b0 tx=0x7f6c8000c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.064 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c7e7fc640 1 -- 172.21.15.113:0/3084489442 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80018070 con 0x7f6c8806ff20 2024-03-30T13:10:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c7e7fc640 1 -- 172.21.15.113:0/3084489442 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c80007500 con 0x7f6c8806ff20 2024-03-30T13:10:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8815ba20 con 0x7f6c8806ff20 2024-03-30T13:10:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c7e7fc640 1 -- 172.21.15.113:0/3084489442 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80004a30 con 0x7f6c8806ff20 2024-03-30T13:10:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c88164410 con 0x7f6c8806ff20 2024-03-30T13:10:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c7e7fc640 1 -- 172.21.15.113:0/3084489442 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6c8001b070 con 0x7f6c8806ff20 2024-03-30T13:10:32.065 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c7e7fc640 1 -- 172.21.15.113:0/3084489442 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f6c800127b0 con 0x7f6c8806ff20 2024-03-30T13:10:32.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.057+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c88164220 con 0x7f6c8806ff20 2024-03-30T13:10:32.066 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.061+0000 7f6c7e7fc640 1 -- 172.21.15.113:0/3084489442 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6c8001d440 con 0x7f6c8806ff20 2024-03-30T13:10:32.124 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:32.116+0000 7fa7e3a4e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T13:10:32.142 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.137+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "46a54234-af69-46ba-8c50-436769f3e1f1", "id": 1} v 0) -- 0x7f6c88068900 con 0x7f6c8806ff20 2024-03-30T13:10:32.150 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.145+0000 7f6c7e7fc640 1 -- 172.21.15.113:0/3084489442 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "46a54234-af69-46ba-8c50-436769f3e1f1", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f6c8001d710 con 0x7f6c8806ff20 2024-03-30T13:10:32.151 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-03-30T13:10:32.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8806ff20 msgr2=0x7f6c8814e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.152 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8806ff20 0x7f6c8814e9a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6c8000c4b0 tx=0x7f6c8000c970 comp rx=0 tx=0).stop 2024-03-30T13:10:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 shutdown_connections 2024-03-30T13:10:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6c8815afc0 0x7f6c8814fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c880b46f0 0x7f6c8815df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 --2- 172.21.15.113:0/3084489442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6c8806ff20 0x7f6c8814e9a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 >> 172.21.15.113:0/3084489442 conn(0x7f6c880d1070 msgr2=0x7f6c880d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 shutdown_connections 2024-03-30T13:10:32.153 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.149+0000 7f6c8fc52640 1 -- 172.21.15.113:0/3084489442 wait complete. 2024-03-30T13:10:32.169 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new aa1ce7b0-6694-4d32-aba7-8cdb92c40906 2 2024-03-30T13:10:32.177 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:32.173+0000 7fd36e30d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-30T13:10:32.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55d043d640 1 Processor -- start 2024-03-30T13:10:32.290 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55d043d640 1 -- start start 2024-03-30T13:10:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55d043d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c815f880 0x7f55c80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55d043d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c8070aa0 con 0x7f55c806f090 2024-03-30T13:10:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55d043d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55c80ce4d0 con 0x7f55c8070640 2024-03-30T13:10:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55d043d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f55c80cc7f0 con 0x7f55c815f880 2024-03-30T13:10:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55cd9d8640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f55c8070640 0x7f55c806e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51670/0 (socket says 172.21.15.113:51670) 2024-03-30T13:10:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55ce1d9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c815f880 0x7f55c80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.291 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.285+0000 7f55cd9d8640 1 -- 172.21.15.113:0/2203825689 learned_addr learned my addr 172.21.15.113:0/2203825689 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-30T13:10:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55ce1d9640 1 -- 172.21.15.113:0/2203825689 >> v1:172.21.15.113:6789/0 conn(0x7f55c8070640 legacy=0x7f55c806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55cd1d7640 1 -- 172.21.15.113:0/2203825689 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565911854 0 0) 0x7f55c80ce4d0 con 0x7f55c8070640 2024-03-30T13:10:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55ce1d9640 1 -- 172.21.15.113:0/2203825689 >> v1:172.21.15.43:6789/0 conn(0x7f55c806f090 legacy=0x7f55c80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.292 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55ce1d9640 1 -- 172.21.15.113:0/2203825689 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c80ccad0 con 0x7f55c815f880 2024-03-30T13:10:32.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55ce1d9640 1 --2- 172.21.15.113:0/2203825689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c815f880 0x7f55c80b4ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55c40097f0 tx=0x7f55c401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f135337ff2c69c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55cd1d7640 1 -- 172.21.15.113:0/2203825689 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40b0070 con 0x7f55c815f880 2024-03-30T13:10:32.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55cd1d7640 1 -- 172.21.15.113:0/2203825689 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c40ab600 con 0x7f55c815f880 2024-03-30T13:10:32.294 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55cd1d7640 1 -- 172.21.15.113:0/2203825689 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c40ab9d0 con 0x7f55c815f880 2024-03-30T13:10:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55d043d640 1 -- 172.21.15.113:0/2203825689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c815f880 msgr2=0x7f55c80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55d043d640 1 --2- 172.21.15.113:0/2203825689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c815f880 0x7f55c80b4ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55c40097f0 tx=0x7f55c401c380 comp rx=0 tx=0).stop 2024-03-30T13:10:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55d043d640 1 -- 172.21.15.113:0/2203825689 shutdown_connections 2024-03-30T13:10:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55d043d640 1 --2- 172.21.15.113:0/2203825689 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c815f880 0x7f55c80b4ff0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.295 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55d043d640 1 -- 172.21.15.113:0/2203825689 >> 172.21.15.113:0/2203825689 conn(0x7f55c80b3cf0 msgr2=0x7f55c80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55d043d640 1 -- 172.21.15.113:0/2203825689 shutdown_connections 2024-03-30T13:10:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.289+0000 7f55d043d640 1 -- 172.21.15.113:0/2203825689 wait complete. 2024-03-30T13:10:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 Processor -- start 2024-03-30T13:10:32.296 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 -- start start 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c806f090 0x7f55c8155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c8070640 0x7f55c81560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c815f880 0x7f55c8156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f55c80cf650 con 0x7f55c806f090 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f55c80cf4d0 con 0x7f55c815f880 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f55c8160850 con 0x7f55c8070640 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c815f880 0x7f55c8156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c815f880 0x7f55c8156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42090/0 (socket says 172.21.15.113:42090) 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55cd9d8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c8070640 0x7f55c81560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.299 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 -- 172.21.15.113:0/2764510359 learned_addr learned my addr 172.21.15.113:0/2764510359 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce1d9640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c806f090 0x7f55c8155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 -- 172.21.15.113:0/2764510359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c8070640 msgr2=0x7f55c81560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c8070640 0x7f55c81560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 -- 172.21.15.113:0/2764510359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c806f090 msgr2=0x7f55c8155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c806f090 0x7f55c8155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 -- 172.21.15.113:0/2764510359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c80cbff0 con 0x7f55c815f880 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55ce9da640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c815f880 0x7f55c8156630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f55c000be60 tx=0x7f55c000bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55beffd640 1 -- 172.21.15.113:0/2764510359 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0019070 con 0x7f55c815f880 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55beffd640 1 -- 172.21.15.113:0/2764510359 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55c0014440 con 0x7f55c815f880 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c81665d0 con 0x7f55c815f880 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55beffd640 1 -- 172.21.15.113:0/2764510359 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0014870 con 0x7f55c815f880 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c80cb7f0 con 0x7f55c815f880 2024-03-30T13:10:32.300 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55beffd640 1 -- 172.21.15.113:0/2764510359 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f55c001c070 con 0x7f55c815f880 2024-03-30T13:10:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55beffd640 1 -- 172.21.15.113:0/2764510359 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f55c0004580 con 0x7f55c815f880 2024-03-30T13:10:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.293+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c8068900 con 0x7f55c815f880 2024-03-30T13:10:32.301 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.297+0000 7f55beffd640 1 -- 172.21.15.113:0/2764510359 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f55c0004160 con 0x7f55c815f880 2024-03-30T13:10:32.370 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.365+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "aa1ce7b0-6694-4d32-aba7-8cdb92c40906", "id": 2} v 0) -- 0x7f55c8155e90 con 0x7f55c815f880 2024-03-30T13:10:32.373 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55beffd640 1 -- 172.21.15.113:0/2764510359 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "aa1ce7b0-6694-4d32-aba7-8cdb92c40906", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f55c0014b00 con 0x7f55c815f880 2024-03-30T13:10:32.373 INFO:teuthology.orchestra.run.smithi113.stdout:2 2024-03-30T13:10:32.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c815f880 msgr2=0x7f55c8156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.374 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c815f880 0x7f55c8156630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f55c000be60 tx=0x7f55c000bf50 comp rx=0 tx=0).stop 2024-03-30T13:10:32.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 shutdown_connections 2024-03-30T13:10:32.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55c815f880 0x7f55c8156630 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55c8070640 0x7f55c81560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 --2- 172.21.15.113:0/2764510359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55c806f090 0x7f55c8155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 >> 172.21.15.113:0/2764510359 conn(0x7f55c80b3cf0 msgr2=0x7f55c80b7b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 shutdown_connections 2024-03-30T13:10:32.375 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.369+0000 7f55d043d640 1 -- 172.21.15.113:0/2764510359 wait complete. 2024-03-30T13:10:32.390 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 517f3d3e-31e4-45dc-a6a7-d2f7d779f73d 3 2024-03-30T13:10:32.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb25567640 1 Processor -- start 2024-03-30T13:10:32.501 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb25567640 1 -- start start 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb25567640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb2015f880 0x7fcb200b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb25567640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb200ce4d0 con 0x7fcb2006f090 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb25567640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb200cc7f0 con 0x7fcb20070640 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb25567640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcb20070ad0 con 0x7fcb2015f880 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb1effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb2015f880 0x7fcb200b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb1effd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb2015f880 0x7fcb200b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42102/0 (socket says 172.21.15.113:42102) 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb1effd640 1 -- 172.21.15.113:0/1222441549 learned_addr learned my addr 172.21.15.113:0/1222441549 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.503 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb1effd640 1 -- 172.21.15.113:0/1222441549 >> v1:172.21.15.113:6789/0 conn(0x7fcb20070640 legacy=0x7fcb2006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb1effd640 1 -- 172.21.15.113:0/1222441549 >> v1:172.21.15.43:6789/0 conn(0x7fcb2006f090 legacy=0x7fcb200ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.504 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.497+0000 7fcb1effd640 1 -- 172.21.15.113:0/1222441549 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb200ccad0 con 0x7fcb2015f880 2024-03-30T13:10:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb1effd640 1 --2- 172.21.15.113:0/1222441549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2015f880 0x7fcb200b4ff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb200b4810 tx=0x7fcb1401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9841561973487 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb1dffb640 1 -- 172.21.15.113:0/1222441549 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb140b0070 con 0x7fcb2015f880 2024-03-30T13:10:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb1dffb640 1 -- 172.21.15.113:0/1222441549 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb140ab600 con 0x7fcb2015f880 2024-03-30T13:10:32.505 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb1dffb640 1 -- 172.21.15.113:0/1222441549 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb140ab9d0 con 0x7fcb2015f880 2024-03-30T13:10:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 -- 172.21.15.113:0/1222441549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2015f880 msgr2=0x7fcb200b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.506 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 --2- 172.21.15.113:0/1222441549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2015f880 0x7fcb200b4ff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb200b4810 tx=0x7fcb1401c3e0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.507 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 -- 172.21.15.113:0/1222441549 shutdown_connections 2024-03-30T13:10:32.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 --2- 172.21.15.113:0/1222441549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb2015f880 0x7fcb200b4ff0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 -- 172.21.15.113:0/1222441549 >> 172.21.15.113:0/1222441549 conn(0x7fcb200b3cf0 msgr2=0x7fcb200b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 -- 172.21.15.113:0/1222441549 shutdown_connections 2024-03-30T13:10:32.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 -- 172.21.15.113:0/1222441549 wait complete. 2024-03-30T13:10:32.509 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 Processor -- start 2024-03-30T13:10:32.510 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.501+0000 7fcb25567640 1 -- start start 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2006f090 0x7fcb2015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb20070640 0x7fcb2015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb2015f880 0x7fcb20155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb200cf650 con 0x7fcb2006f090 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcb200cf4d0 con 0x7fcb20070640 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcb20160850 con 0x7fcb2015f880 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb2015f880 0x7fcb20155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1e7fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb20070640 0x7fcb2015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1f7fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb2015f880 0x7fcb20155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:42586/0 (socket says 172.21.15.113:42586) 2024-03-30T13:10:32.512 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1f7fe640 1 -- 172.21.15.113:0/2641242826 learned_addr learned my addr 172.21.15.113:0/2641242826 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1effd640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2006f090 0x7fcb2015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1e7fc640 1 -- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb2015f880 msgr2=0x7fcb20155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1e7fc640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb2015f880 0x7fcb20155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1e7fc640 1 -- 172.21.15.113:0/2641242826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2006f090 msgr2=0x7fcb2015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1e7fc640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2006f090 0x7fcb2015d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1e7fc640 1 -- 172.21.15.113:0/2641242826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb200c9d60 con 0x7fcb20070640 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1f7fe640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb2015f880 0x7fcb20155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb1e7fc640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb20070640 0x7fcb2015d5a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c3f0 tx=0x7fcb1000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb077fe640 1 -- 172.21.15.113:0/2641242826 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100052d0 con 0x7fcb20070640 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb077fe640 1 -- 172.21.15.113:0/2641242826 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb10005470 con 0x7fcb20070640 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb200ca040 con 0x7fcb20070640 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb077fe640 1 -- 172.21.15.113:0/2641242826 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb10007260 con 0x7fcb20070640 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb200ca600 con 0x7fcb20070640 2024-03-30T13:10:32.513 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb077fe640 1 -- 172.21.15.113:0/2641242826 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb10007400 con 0x7fcb20070640 2024-03-30T13:10:32.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb077fe640 1 -- 172.21.15.113:0/2641242826 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fcb10003db0 con 0x7fcb20070640 2024-03-30T13:10:32.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb20068900 con 0x7fcb20070640 2024-03-30T13:10:32.514 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.505+0000 7fcb077fe640 1 -- 172.21.15.113:0/2641242826 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcb10003920 con 0x7fcb20070640 2024-03-30T13:10:32.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.573+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "517f3d3e-31e4-45dc-a6a7-d2f7d779f73d", "id": 3} v 0) -- 0x7fcb200d1790 con 0x7fcb20070640 2024-03-30T13:10:32.584 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb077fe640 1 -- 172.21.15.113:0/2641242826 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "517f3d3e-31e4-45dc-a6a7-d2f7d779f73d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fcb100067e0 con 0x7fcb20070640 2024-03-30T13:10:32.585 INFO:teuthology.orchestra.run.smithi113.stdout:3 2024-03-30T13:10:32.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb20070640 msgr2=0x7fcb2015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb20070640 0x7fcb2015d5a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c3f0 tx=0x7fcb1000c8b0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 shutdown_connections 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb2015f880 0x7fcb20155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb20070640 0x7fcb2015d5a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 --2- 172.21.15.113:0/2641242826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb2006f090 0x7fcb2015d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 >> 172.21.15.113:0/2641242826 conn(0x7fcb200b3cf0 msgr2=0x7fcb200b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 shutdown_connections 2024-03-30T13:10:32.587 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.581+0000 7fcb25567640 1 -- 172.21.15.113:0/2641242826 wait complete. 2024-03-30T13:10:32.600 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 985ea672-67df-4c19-b4f6-fba3aced4f30 4 2024-03-30T13:10:32.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.713+0000 7f3fbc87a640 1 Processor -- start 2024-03-30T13:10:32.719 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.713+0000 7f3fbc87a640 1 -- start start 2024-03-30T13:10:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.713+0000 7f3fbc87a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb81627c0 0x7f3fb806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.713+0000 7f3fbc87a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3fb8070790 0x7f3fb806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.713+0000 7f3fbc87a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3fb80cda80 0x7f3fb806ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.713+0000 7f3fbc87a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3fb80b4c50 con 0x7f3fb8070790 2024-03-30T13:10:32.720 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fbc87a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3fb80b4080 con 0x7f3fb80cda80 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fbc87a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb80b4320 con 0x7f3fb81627c0 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3fb80cda80 0x7f3fb806ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb81627c0 0x7f3fb806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3fb80cda80 0x7f3fb806ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42124/0 (socket says 172.21.15.113:42124) 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 -- 172.21.15.113:0/375067117 learned_addr learned my addr 172.21.15.113:0/375067117 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3faffff640 1 --2- 172.21.15.113:0/375067117 >> v2:172.21.15.43:3300/0 conn(0x7f3fb8070790 0x7f3fb806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 -- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb81627c0 msgr2=0x7f3fb806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 --2- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb81627c0 0x7f3fb806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 -- 172.21.15.113:0/375067117 >> v2:172.21.15.43:3300/0 conn(0x7f3fb8070790 msgr2=0x7f3fb806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 --2- 172.21.15.113:0/375067117 >> v2:172.21.15.43:3300/0 conn(0x7f3fb8070790 0x7f3fb806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 -- 172.21.15.113:0/375067117 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb80b4600 con 0x7f3fb80cda80 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb659c640 1 --2- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb81627c0 0x7f3fb806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:32.721 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3faffff640 1 --2- 172.21.15.113:0/375067117 >> v2:172.21.15.43:3300/0 conn(0x7f3fb8070790 0x7f3fb806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:32.722 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:32.716+0000 7fa7e3a4e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T13:10:32.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb6d9d640 1 --2- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb80cda80 0x7f3fb806ff20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80739a0 tx=0x7f3fb001c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3aae5d70341463 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb5d9b640 1 -- 172.21.15.113:0/375067117 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00a5750 con 0x7f3fb80cda80 2024-03-30T13:10:32.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb5d9b640 1 -- 172.21.15.113:0/375067117 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb00a58f0 con 0x7f3fb80cda80 2024-03-30T13:10:32.723 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.717+0000 7f3fb5d9b640 1 -- 172.21.15.113:0/375067117 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00a5cc0 con 0x7f3fb80cda80 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb80cda80 msgr2=0x7f3fb806ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb80cda80 0x7f3fb806ff20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80739a0 tx=0x7f3fb001c930 comp rx=0 tx=0).stop 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- 172.21.15.113:0/375067117 shutdown_connections 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb80cda80 0x7f3fb806ff20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/375067117 >> v2:172.21.15.43:3300/0 conn(0x7f3fb8070790 0x7f3fb806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/375067117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb81627c0 0x7f3fb806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- 172.21.15.113:0/375067117 >> 172.21.15.113:0/375067117 conn(0x7f3fb80d1070 msgr2=0x7f3fb80d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.724 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- 172.21.15.113:0/375067117 shutdown_connections 2024-03-30T13:10:32.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- 172.21.15.113:0/375067117 wait complete. 2024-03-30T13:10:32.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 Processor -- start 2024-03-30T13:10:32.725 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- start start 2024-03-30T13:10:32.727 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb8070790 0x7f3fb80b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb80cda80 0x7f3fb8162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb81627c0 0x7f3fb8163370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb80b5a50 con 0x7f3fb80cda80 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb80b58d0 con 0x7f3fb81627c0 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb80b5bd0 con 0x7f3fb8070790 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb8070790 0x7f3fb80b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb6d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb81627c0 0x7f3fb8163370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb8070790 0x7f3fb80b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:42596/0 (socket says 172.21.15.113:42596) 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb659c640 1 -- 172.21.15.113:0/2211783406 learned_addr learned my addr 172.21.15.113:0/2211783406 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3faffff640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb80cda80 0x7f3fb8162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb6d9d640 1 -- 172.21.15.113:0/2211783406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb8070790 msgr2=0x7f3fb80b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb6d9d640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb8070790 0x7f3fb80b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb6d9d640 1 -- 172.21.15.113:0/2211783406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb80cda80 msgr2=0x7f3fb8162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.728 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb6d9d640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb80cda80 0x7f3fb8162e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb6d9d640 1 -- 172.21.15.113:0/2211783406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb806cc80 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fb6d9d640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb81627c0 0x7f3fb8163370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00ab000 tx=0x7f3fb00196d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3faeffd640 1 -- 172.21.15.113:0/2211783406 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00bc020 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3faeffd640 1 -- 172.21.15.113:0/2211783406 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb00b8670 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3faeffd640 1 -- 172.21.15.113:0/2211783406 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00b8a40 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.721+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb814fff0 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.725+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb81505b0 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.725+0000 7f3faeffd640 1 -- 172.21.15.113:0/2211783406 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3fb00bf020 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.725+0000 7f3faeffd640 1 -- 172.21.15.113:0/2211783406 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f3fb00c1b80 con 0x7f3fb81627c0 2024-03-30T13:10:32.729 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.725+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb8068900 con 0x7f3fb81627c0 2024-03-30T13:10:32.730 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.725+0000 7f3faeffd640 1 -- 172.21.15.113:0/2211783406 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3fb00c6030 con 0x7f3fb81627c0 2024-03-30T13:10:32.748 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:32.745+0000 7fd36e30d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-30T13:10:32.777 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:32.772+0000 7fa7e3a4e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T13:10:32.797 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.793+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "985ea672-67df-4c19-b4f6-fba3aced4f30", "id": 4} v 0) -- 0x7f3fb80cd860 con 0x7f3fb81627c0 2024-03-30T13:10:32.801 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3faeffd640 1 -- 172.21.15.113:0/2211783406 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "985ea672-67df-4c19-b4f6-fba3aced4f30", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f3fb00165e0 con 0x7f3fb81627c0 2024-03-30T13:10:32.801 INFO:teuthology.orchestra.run.smithi113.stdout:4 2024-03-30T13:10:32.802 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:32.797+0000 7fd36e30d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb81627c0 msgr2=0x7f3fb8163370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb81627c0 0x7f3fb8163370 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00ab000 tx=0x7f3fb00196d0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 shutdown_connections 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb81627c0 0x7f3fb8163370 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb80cda80 0x7f3fb8162e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 --2- 172.21.15.113:0/2211783406 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb8070790 0x7f3fb80b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 >> 172.21.15.113:0/2211783406 conn(0x7f3fb80d1070 msgr2=0x7f3fb816c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 shutdown_connections 2024-03-30T13:10:32.803 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.797+0000 7f3fbc87a640 1 -- 172.21.15.113:0/2211783406 wait complete. 2024-03-30T13:10:32.817 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 56e8bbbc-9c57-402b-8b46-0eec1ec86b2c 5 2024-03-30T13:10:32.833 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:32.828+0000 7fa7e3a4e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T13:10:32.858 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:32.853+0000 7fd36e30d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-30T13:10:32.913 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:32.908+0000 7fa7e3a4e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T13:10:32.934 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.929+0000 7fab339b9640 1 Processor -- start 2024-03-30T13:10:32.935 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.929+0000 7fab339b9640 1 -- start start 2024-03-30T13:10:32.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.929+0000 7fab339b9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab2c15afc0 0x7fab2c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.929+0000 7fab339b9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab2c0700f0 0x7fab2c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.929+0000 7fab339b9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab2c0b4ba0 con 0x7fab2c0b4520 2024-03-30T13:10:32.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.929+0000 7fab339b9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fab2c070c30 con 0x7fab2c15afc0 2024-03-30T13:10:32.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.929+0000 7fab339b9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fab2c0b4960 con 0x7fab2c0700f0 2024-03-30T13:10:32.936 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab2c0700f0 0x7fab2c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab2c0700f0 0x7fab2c0ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42152/0 (socket says 172.21.15.113:42152) 2024-03-30T13:10:32.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 -- 172.21.15.113:0/1792206052 learned_addr learned my addr 172.21.15.113:0/1792206052 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31755640 1 --2- 172.21.15.113:0/1792206052 >> v2:172.21.15.43:3300/0 conn(0x7fab2c15afc0 0x7fab2c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 -- 172.21.15.113:0/1792206052 >> v2:172.21.15.43:3300/0 conn(0x7fab2c15afc0 msgr2=0x7fab2c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:32.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 --2- 172.21.15.113:0/1792206052 >> v2:172.21.15.43:3300/0 conn(0x7fab2c15afc0 0x7fab2c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 -- 172.21.15.113:0/1792206052 >> v1:172.21.15.113:6789/0 conn(0x7fab2c0b4520 legacy=0x7fab2c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.937 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 -- 172.21.15.113:0/1792206052 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c15ec60 con 0x7fab2c0700f0 2024-03-30T13:10:32.938 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fd36e30d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-30T13:10:32.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab31f56640 1 --2- 172.21.15.113:0/1792206052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c0ce7b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab2c06e500 tx=0x7fab2401c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a79808b53e69dc6b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab23fff640 1 -- 172.21.15.113:0/1792206052 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240a5880 con 0x7fab2c0700f0 2024-03-30T13:10:32.939 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab23fff640 1 -- 172.21.15.113:0/1792206052 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab240a5a20 con 0x7fab2c0700f0 2024-03-30T13:10:32.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.933+0000 7fab23fff640 1 -- 172.21.15.113:0/1792206052 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240a5df0 con 0x7fab2c0700f0 2024-03-30T13:10:32.940 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- 172.21.15.113:0/1792206052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 msgr2=0x7fab2c0ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 --2- 172.21.15.113:0/1792206052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c0ce7b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab2c06e500 tx=0x7fab2401c9c0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- 172.21.15.113:0/1792206052 shutdown_connections 2024-03-30T13:10:32.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 --2- 172.21.15.113:0/1792206052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c0ce7b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 --2- 172.21.15.113:0/1792206052 >> v2:172.21.15.43:3300/0 conn(0x7fab2c15afc0 0x7fab2c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- 172.21.15.113:0/1792206052 >> 172.21.15.113:0/1792206052 conn(0x7fab2c0d1070 msgr2=0x7fab2c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:32.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- 172.21.15.113:0/1792206052 shutdown_connections 2024-03-30T13:10:32.941 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- 172.21.15.113:0/1792206052 wait complete. 2024-03-30T13:10:32.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 Processor -- start 2024-03-30T13:10:32.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- start start 2024-03-30T13:10:32.944 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0b4520 0x7fab2c150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c15afc0 0x7fab2c15d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c15fa20 con 0x7fab2c15afc0 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c15b3a0 con 0x7fab2c0700f0 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fab2c161bb0 con 0x7fab2c0b4520 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab30f54640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0b4520 0x7fab2c150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42154/0 (socket says 172.21.15.113:42154) 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 -- 172.21.15.113:0/3448307167 learned_addr learned my addr 172.21.15.113:0/3448307167 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31f56640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c15afc0 0x7fab2c15d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:32.945 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 -- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0b4520 msgr2=0x7fab2c150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0b4520 0x7fab2c150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 -- 172.21.15.113:0/3448307167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c15afc0 msgr2=0x7fab2c15d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c15afc0 0x7fab2c15d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 -- 172.21.15.113:0/3448307167 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c161e90 con 0x7fab2c0700f0 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab30f54640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0b4520 0x7fab2c150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab31755640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c151530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab2800d620 tx=0x7fab2800dae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab21ffb640 1 -- 172.21.15.113:0/3448307167 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800b620 con 0x7fab2c0700f0 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab21ffb640 1 -- 172.21.15.113:0/3448307167 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab28019070 con 0x7fab2c0700f0 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c162170 con 0x7fab2c0700f0 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab21ffb640 1 -- 172.21.15.113:0/3448307167 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2800ba20 con 0x7fab2c0700f0 2024-03-30T13:10:32.946 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c15fdb0 con 0x7fab2c0700f0 2024-03-30T13:10:32.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab21ffb640 1 -- 172.21.15.113:0/3448307167 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab2800bbc0 con 0x7fab2c0700f0 2024-03-30T13:10:32.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.937+0000 7fab21ffb640 1 -- 172.21.15.113:0/3448307167 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fab2801e9e0 con 0x7fab2c0700f0 2024-03-30T13:10:32.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.941+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c068900 con 0x7fab2c0700f0 2024-03-30T13:10:32.947 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:32.941+0000 7fab21ffb640 1 -- 172.21.15.113:0/3448307167 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fab2801ec80 con 0x7fab2c0700f0 2024-03-30T13:10:32.967 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:32.960+0000 7fa7e3a4e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T13:10:32.992 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:32.989+0000 7fd36e30d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-30T13:10:33.012 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.009+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "56e8bbbc-9c57-402b-8b46-0eec1ec86b2c", "id": 5} v 0) -- 0x7fab2c075930 con 0x7fab2c0700f0 2024-03-30T13:10:33.016 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab21ffb640 1 -- 172.21.15.113:0/3448307167 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "56e8bbbc-9c57-402b-8b46-0eec1ec86b2c", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fab2801e460 con 0x7fab2c0700f0 2024-03-30T13:10:33.017 INFO:teuthology.orchestra.run.smithi113.stdout:5 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 msgr2=0x7fab2c151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c151530 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab2800d620 tx=0x7fab2800dae0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 shutdown_connections 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab2c15afc0 0x7fab2c15d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab2c0b4520 0x7fab2c150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 --2- 172.21.15.113:0/3448307167 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab2c0700f0 0x7fab2c151530 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 >> 172.21.15.113:0/3448307167 conn(0x7fab2c0d1070 msgr2=0x7fab2c0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 shutdown_connections 2024-03-30T13:10:33.018 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.013+0000 7fab339b9640 1 -- 172.21.15.113:0/3448307167 wait complete. 2024-03-30T13:10:33.032 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea 6 2024-03-30T13:10:33.048 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:33.040+0000 7fa7e3a4e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T13:10:33.071 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:33.065+0000 7fd36e30d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-30T13:10:33.103 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:33.096+0000 7fa7e3a4e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T13:10:33.126 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:33.121+0000 7fd36e30d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-30T13:10:33.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.185+0000 7f37e7c6d640 1 Processor -- start 2024-03-30T13:10:33.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.185+0000 7f37e7c6d640 1 -- start start 2024-03-30T13:10:33.192 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f37e015c160 0x7f37e006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e0160c80 con 0x7f37e00ccaa0 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f37e01607d0 con 0x7f37e015c160 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f37e00cd680 con 0x7f37e015fe20 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:42620/0 (socket says 172.21.15.113:42620) 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 -- 172.21.15.113:0/905922728 learned_addr learned my addr 172.21.15.113:0/905922728 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 -- 172.21.15.113:0/905922728 >> v2:172.21.15.43:3300/0 conn(0x7f37e015c160 msgr2=0x7f37e006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5a09640 1 --2- 172.21.15.113:0/905922728 >> v2:172.21.15.43:3300/0 conn(0x7f37e015c160 0x7f37e006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 --2- 172.21.15.113:0/905922728 >> v2:172.21.15.43:3300/0 conn(0x7f37e015c160 0x7f37e006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 -- 172.21.15.113:0/905922728 >> v1:172.21.15.43:6789/0 conn(0x7f37e00ccaa0 legacy=0x7f37e00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 -- 172.21.15.113:0/905922728 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e00cd960 con 0x7f37e015fe20 2024-03-30T13:10:33.193 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5a09640 1 --2- 172.21.15.113:0/905922728 >> v2:172.21.15.43:3300/0 conn(0x7f37e015c160 0x7f37e006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:33.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e5208640 1 --2- 172.21.15.113:0/905922728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0160290 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009870 tx=0x7f37d001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb573d81abe68f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e4a07640 1 -- 172.21.15.113:0/905922728 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d00aa610 con 0x7f37e015fe20 2024-03-30T13:10:33.194 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e4a07640 1 -- 172.21.15.113:0/905922728 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d00af070 con 0x7f37e015fe20 2024-03-30T13:10:33.195 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e4a07640 1 -- 172.21.15.113:0/905922728 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d00aa9d0 con 0x7f37e015fe20 2024-03-30T13:10:33.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- 172.21.15.113:0/905922728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 msgr2=0x7f37e0160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 --2- 172.21.15.113:0/905922728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0160290 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009870 tx=0x7f37d001c4f0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- 172.21.15.113:0/905922728 shutdown_connections 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 --2- 172.21.15.113:0/905922728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0160290 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 --2- 172.21.15.113:0/905922728 >> v2:172.21.15.43:3300/0 conn(0x7f37e015c160 0x7f37e006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- 172.21.15.113:0/905922728 >> 172.21.15.113:0/905922728 conn(0x7f37e00d0c90 msgr2=0x7f37e00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- 172.21.15.113:0/905922728 shutdown_connections 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.189+0000 7f37e7c6d640 1 -- 172.21.15.113:0/905922728 wait complete. 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 Processor -- start 2024-03-30T13:10:33.197 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 -- start start 2024-03-30T13:10:33.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00ccaa0 0x7f37e01384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.199 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37e015c160 0x7f37e0138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00cf420 con 0x7f37e015c160 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f37e00cf2a0 con 0x7f37e00ccaa0 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f37e015d550 con 0x7f37e015fe20 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e5a09640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00ccaa0 0x7f37e01384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.113:42626/0 (socket says 172.21.15.113:42626) 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 -- 172.21.15.113:0/3140877182 learned_addr learned my addr 172.21.15.113:0/3140877182 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e5208640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37e015c160 0x7f37e0138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 -- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00ccaa0 msgr2=0x7f37e01384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00ccaa0 0x7f37e01384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 -- 172.21.15.113:0/3140877182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37e015c160 msgr2=0x7f37e0138b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.200 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37e015c160 0x7f37e0138b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 -- 172.21.15.113:0/3140877182 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e01394e0 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e5a09640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00ccaa0 0x7f37e01384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e620a640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0141960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37d800b970 tx=0x7f37d800be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37d67fc640 1 -- 172.21.15.113:0/3140877182 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8019070 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37d67fc640 1 -- 172.21.15.113:0/3140877182 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d800cb40 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e01397c0 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37d67fc640 1 -- 172.21.15.113:0/3140877182 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8014440 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e0139d80 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37d67fc640 1 -- 172.21.15.113:0/3140877182 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f37d801c070 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37d67fc640 1 -- 172.21.15.113:0/3140877182 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f37d801d9a0 con 0x7f37e015fe20 2024-03-30T13:10:33.201 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.193+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e0068900 con 0x7f37e015fe20 2024-03-30T13:10:33.203 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.197+0000 7f37d67fc640 1 -- 172.21.15.113:0/3140877182 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f37d801dc80 con 0x7f37e015fe20 2024-03-30T13:10:33.260 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:33.252+0000 7fa7e3a4e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T13:10:33.271 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.265+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea", "id": 6} v 0) -- 0x7f37e006e040 con 0x7f37e015fe20 2024-03-30T13:10:33.280 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37d67fc640 1 -- 172.21.15.113:0/3140877182 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f37d801d3b0 con 0x7f37e015fe20 2024-03-30T13:10:33.280 INFO:teuthology.orchestra.run.smithi113.stdout:6 2024-03-30T13:10:33.281 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 msgr2=0x7f37e0141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0141960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37d800b970 tx=0x7f37d800be30 comp rx=0 tx=0).stop 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 shutdown_connections 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f37e015fe20 0x7f37e0141960 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37e015c160 0x7f37e0138b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 --2- 172.21.15.113:0/3140877182 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f37e00ccaa0 0x7f37e01384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 >> 172.21.15.113:0/3140877182 conn(0x7f37e00d0c90 msgr2=0x7f37e00b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 shutdown_connections 2024-03-30T13:10:33.282 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.277+0000 7f37e7c6d640 1 -- 172.21.15.113:0/3140877182 wait complete. 2024-03-30T13:10:33.284 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:33.281+0000 7fd36e30d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-30T13:10:33.296 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --cluster ceph osd new 33b78f88-19b5-4778-9760-3137db597d09 7 2024-03-30T13:10:33.325 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:33.320+0000 7fa7e3a4e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T13:10:33.348 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:33.345+0000 7fd36e30d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-30T13:10:33.402 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47cdd11640 1 Processor -- start 2024-03-30T13:10:33.403 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47cdd11640 1 -- start start 2024-03-30T13:10:33.415 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47cdd11640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f47c806f090 0x7f47c80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47cdd11640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47c8070640 0x7f47c80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47cdd11640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c80ce4d0 con 0x7f47c815f880 2024-03-30T13:10:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47cdd11640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f47c8070a50 con 0x7f47c8070640 2024-03-30T13:10:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47cdd11640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f47c80cc7f0 con 0x7f47c806f090 2024-03-30T13:10:33.416 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47c77fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f47c815f880 0x7f47c806e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.113:51698/0 (socket says 172.21.15.113:51698) 2024-03-30T13:10:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47c77fe640 1 -- 172.21.15.113:0/2443142542 learned_addr learned my addr 172.21.15.113:0/2443142542 (peer_addr_for_me v1:172.21.15.113:0/0) 2024-03-30T13:10:33.417 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.397+0000 7f47c6ffd640 1 --2- 172.21.15.113:0/2443142542 >> v2:172.21.15.113:3300/0 conn(0x7f47c806f090 0x7f47c80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c7fff640 1 --2- 172.21.15.113:0/2443142542 >> v2:172.21.15.43:3300/0 conn(0x7f47c8070640 0x7f47c80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c67fc640 1 -- 172.21.15.113:0/2443142542 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2306127546 0 0) 0x7f47c80ce4d0 con 0x7f47c815f880 2024-03-30T13:10:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c67fc640 1 -- 172.21.15.113:0/2443142542 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47b0003530 con 0x7f47c815f880 2024-03-30T13:10:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c6ffd640 1 -- 172.21.15.113:0/2443142542 >> v2:172.21.15.43:3300/0 conn(0x7f47c8070640 msgr2=0x7f47c80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.418 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c6ffd640 1 --2- 172.21.15.113:0/2443142542 >> v2:172.21.15.43:3300/0 conn(0x7f47c8070640 0x7f47c80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c6ffd640 1 -- 172.21.15.113:0/2443142542 >> v1:172.21.15.113:6789/0 conn(0x7f47c815f880 legacy=0x7f47c806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c6ffd640 1 -- 172.21.15.113:0/2443142542 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c80ccad0 con 0x7f47c806f090 2024-03-30T13:10:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c7fff640 1 --2- 172.21.15.113:0/2443142542 >> v2:172.21.15.43:3300/0 conn(0x7f47c8070640 0x7f47c80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c6ffd640 1 --2- 172.21.15.113:0/2443142542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c806f090 0x7f47c80b4ff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f47c80b4810 tx=0x7f47bc01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42aabb1d231b385d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c67fc640 1 -- 172.21.15.113:0/2443142542 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b0070 con 0x7f47c806f090 2024-03-30T13:10:33.419 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c67fc640 1 -- 172.21.15.113:0/2443142542 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc0ab600 con 0x7f47c806f090 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47c67fc640 1 -- 172.21.15.113:0/2443142542 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0ab9d0 con 0x7f47c806f090 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 -- 172.21.15.113:0/2443142542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c806f090 msgr2=0x7f47c80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 --2- 172.21.15.113:0/2443142542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c806f090 0x7f47c80b4ff0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f47c80b4810 tx=0x7f47bc01c360 comp rx=0 tx=0).stop 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 -- 172.21.15.113:0/2443142542 shutdown_connections 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 --2- 172.21.15.113:0/2443142542 >> v2:172.21.15.43:3300/0 conn(0x7f47c8070640 0x7f47c80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 --2- 172.21.15.113:0/2443142542 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c806f090 0x7f47c80b4ff0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 -- 172.21.15.113:0/2443142542 >> 172.21.15.113:0/2443142542 conn(0x7f47c80b3cf0 msgr2=0x7f47c80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.420 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 -- 172.21.15.113:0/2443142542 shutdown_connections 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.401+0000 7f47cdd11640 1 -- 172.21.15.113:0/2443142542 wait complete. 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 Processor -- start 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 -- start start 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c806f090 0x7f47c81578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c8070640 0x7f47c80ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815f880 0x7f47c81580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81609d0 con 0x7f47c806f090 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8160850 con 0x7f47c8070640 2024-03-30T13:10:33.421 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f47c81691e0 con 0x7f47c815f880 2024-03-30T13:10:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c8070640 0x7f47c80ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c8070640 0x7f47c80ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.113:42172/0 (socket says 172.21.15.113:42172) 2024-03-30T13:10:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 -- 172.21.15.113:0/2801701573 learned_addr learned my addr 172.21.15.113:0/2801701573 (peer_addr_for_me v2:172.21.15.113:0/0) 2024-03-30T13:10:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c7fff640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815f880 0x7f47c81580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c77fe640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c806f090 0x7f47c81578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.422 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 -- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815f880 msgr2=0x7f47c81580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815f880 0x7f47c81580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 -- 172.21.15.113:0/2801701573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c806f090 msgr2=0x7f47c81578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c806f090 0x7f47c81578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 -- 172.21.15.113:0/2801701573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c8157e30 con 0x7f47c8070640 2024-03-30T13:10:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c77fe640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c806f090 0x7f47c81578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:33.423 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47c6ffd640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c8070640 0x7f47c80ca820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0ae040 tx=0x7f47bc019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47affff640 1 -- 172.21.15.113:0/2801701573 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b0040 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47affff640 1 -- 172.21.15.113:0/2801701573 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc0b9070 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c80cb170 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47affff640 1 -- 172.21.15.113:0/2801701573 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc006e20 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c80cb730 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47affff640 1 -- 172.21.15.113:0/2801701573 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f47bc0b4670 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47affff640 1 -- 172.21.15.113:0/2801701573 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f47bc0bd3c0 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.405+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c8068900 con 0x7f47c8070640 2024-03-30T13:10:33.424 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.409+0000 7f47affff640 1 -- 172.21.15.113:0/2801701573 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f47bc016070 con 0x7f47c8070640 2024-03-30T13:10:33.479 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.477+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd new", "uuid": "33b78f88-19b5-4778-9760-3137db597d09", "id": 7} v 0) -- 0x7f47c80d1790 con 0x7f47c8070640 2024-03-30T13:10:33.488 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47affff640 1 -- 172.21.15.113:0/2801701573 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33b78f88-19b5-4778-9760-3137db597d09", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f47bc0b4aa0 con 0x7f47c8070640 2024-03-30T13:10:33.488 INFO:teuthology.orchestra.run.smithi113.stdout:7 2024-03-30T13:10:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c8070640 msgr2=0x7f47c80ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.489 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c8070640 0x7f47c80ca820 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0ae040 tx=0x7f47bc019100 comp rx=0 tx=0).stop 2024-03-30T13:10:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 shutdown_connections 2024-03-30T13:10:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f47c815f880 0x7f47c81580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f47c8070640 0x7f47c80ca820 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 --2- 172.21.15.113:0/2801701573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47c806f090 0x7f47c81578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 >> 172.21.15.113:0/2801701573 conn(0x7f47c80b3cf0 msgr2=0x7f47c80b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 shutdown_connections 2024-03-30T13:10:33.490 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:10:33.485+0000 7f47cdd11640 1 -- 172.21.15.113:0/2801701573 wait complete. 2024-03-30T13:10:33.504 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-30T13:10:33.504 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-30T13:10:33.507 INFO:tasks.ceph.osd.0:Started 2024-03-30T13:10:33.507 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-30T13:10:33.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-30T13:10:33.511 INFO:tasks.ceph.osd.1:Started 2024-03-30T13:10:33.511 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-30T13:10:33.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-30T13:10:33.513 INFO:tasks.ceph.osd.2:Started 2024-03-30T13:10:33.513 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-30T13:10:33.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-30T13:10:33.515 INFO:tasks.ceph.osd.3:Started 2024-03-30T13:10:33.515 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-30T13:10:33.515 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-30T13:10:33.517 INFO:tasks.ceph.osd.4:Started 2024-03-30T13:10:33.517 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-30T13:10:33.518 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-30T13:10:33.519 INFO:tasks.ceph.osd.5:Started 2024-03-30T13:10:33.519 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-30T13:10:33.519 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-30T13:10:33.521 INFO:tasks.ceph.osd.6:Started 2024-03-30T13:10:33.521 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-30T13:10:33.521 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-30T13:10:33.523 INFO:tasks.ceph.osd.7:Started 2024-03-30T13:10:33.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:33.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.684+0000 7f2a20fa3640 1 Processor -- start 2024-03-30T13:10:33.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.684+0000 7f2a20fa3640 1 -- start start 2024-03-30T13:10:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a20fa3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15bfe0 0x7f2a1c0b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a20fa3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a1c0716b0 con 0x7f2a1c15f490 2024-03-30T13:10:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a20fa3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a1c0b5b00 con 0x7f2a1c15bc00 2024-03-30T13:10:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a20fa3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1c071990 con 0x7f2a1c15bfe0 2024-03-30T13:10:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2a1c15f490 0x7f2a1c0d11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53146/0 (socket says 172.21.15.43:53146) 2024-03-30T13:10:33.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1bfff640 1 -- 172.21.15.43:0/2898430030 learned_addr learned my addr 172.21.15.43:0/2898430030 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 --2- 172.21.15.43:0/2898430030 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15bfe0 0x7f2a1c0b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3031821161 0 0) 0x7f2a1c0716b0 con 0x7f2a1c15f490 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a04003530 con 0x7f2a1c15f490 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3203479407 0 0) 0x7f2a04003530 con 0x7f2a1c15f490 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15bfe0 msgr2=0x7f2a1c0b65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 --2- 172.21.15.43:0/2898430030 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15bfe0 0x7f2a1c0b65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 >> v1:172.21.15.113:6789/0 conn(0x7f2a1c15bc00 legacy=0x7f2a1c070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c0b5de0 con 0x7f2a1c15f490 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f2a14001a00 con 0x7f2a1c15f490 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 >> v1:172.21.15.43:6789/0 conn(0x7f2a1c15f490 legacy=0x7f2a1c0d11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a040039f0 0x7f2a04003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a04004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a04005220 0x7f2a0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c071990 con 0x7f2a04004810 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0400c150 con 0x7f2a040039f0 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a0400c350 con 0x7f2a04005220 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a04004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1b7fe640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a040039f0 0x7f2a04003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1bfff640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a04005220 0x7f2a0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 -- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a04005220 msgr2=0x7f2a0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a04005220 0x7f2a0400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 -- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a040039f0 msgr2=0x7f2a04003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a040039f0 0x7f2a04003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 -- 172.21.15.43:0/2898430030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a04003530 con 0x7f2a04004810 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1bfff640 1 -- 172.21.15.43:0/2898430030 reap_dead start 2024-03-30T13:10:33.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1b7fe640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a040039f0 0x7f2a04003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1affd640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a04004c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c072880 tx=0x7f2a0c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24464e2ff655d262 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0096b0 con 0x7f2a04004810 2024-03-30T13:10:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0c0a64d0 con 0x7f2a04004810 2024-03-30T13:10:33.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.688+0000 7f2a1a7fc640 1 -- 172.21.15.43:0/2898430030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0a68c0 con 0x7f2a04004810 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- 172.21.15.43:0/2898430030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 msgr2=0x7f2a04004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a04004c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c072880 tx=0x7f2a0c01c9e0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- 172.21.15.43:0/2898430030 shutdown_connections 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 --2- 172.21.15.43:0/2898430030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a04004c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- 172.21.15.43:0/2898430030 >> 172.21.15.43:0/2898430030 conn(0x7f2a1c0ce960 msgr2=0x7f2a1c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- 172.21.15.43:0/2898430030 shutdown_connections 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- 172.21.15.43:0/2898430030 wait complete. 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 Processor -- start 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- start start 2024-03-30T13:10:33.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a1c159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c147800 0x7f2a1c15a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c152330 0x7f2a1c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c0d1550 con 0x7f2a04004810 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c0d13d0 con 0x7f2a1c152330 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a1c159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1c0d16d0 con 0x7f2a1c147800 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a1c159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60164/0 (socket says 172.21.15.43:60164) 2024-03-30T13:10:33.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 -- 172.21.15.43:0/786518645 learned_addr learned my addr 172.21.15.43:0/786518645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1affd640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c147800 0x7f2a1c15a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1bfff640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c152330 0x7f2a1c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 -- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c147800 msgr2=0x7f2a1c15a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c147800 0x7f2a1c15a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 -- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c152330 msgr2=0x7f2a1c1527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c152330 0x7f2a1c1527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 -- 172.21.15.43:0/786518645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c159900 con 0x7f2a04004810 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a1b7fe640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a1c159200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000e440 tx=0x7f2a1000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a03fff640 1 -- 172.21.15.43:0/786518645 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10019070 con 0x7f2a04004810 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a03fff640 1 -- 172.21.15.43:0/786518645 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a100052e0 con 0x7f2a04004810 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c159be0 con 0x7f2a04004810 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a03fff640 1 -- 172.21.15.43:0/786518645 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10003840 con 0x7f2a04004810 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.692+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c15a1a0 con 0x7f2a04004810 2024-03-30T13:10:33.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.696+0000 7f2a03fff640 1 -- 172.21.15.43:0/786518645 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a1001c070 con 0x7f2a04004810 2024-03-30T13:10:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.696+0000 7f2a03fff640 1 -- 172.21.15.43:0/786518645 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f2a10014480 con 0x7f2a04004810 2024-03-30T13:10:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.696+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1c068d80 con 0x7f2a04004810 2024-03-30T13:10:33.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.696+0000 7f2a03fff640 1 -- 172.21.15.43:0/786518645 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2a100039e0 con 0x7f2a04004810 2024-03-30T13:10:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.752+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a1c071b40 con 0x7f2a04004810 2024-03-30T13:10:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a03fff640 1 -- 172.21.15.43:0/786518645 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f2a10003bc0 con 0x7f2a04004810 2024-03-30T13:10:33.761 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:10:33.761 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:10:33.478764+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 msgr2=0x7f2a1c159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a1c159200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000e440 tx=0x7f2a1000e900 comp rx=0 tx=0).stop 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 shutdown_connections 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c152330 0x7f2a1c1527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c147800 0x7f2a1c15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 --2- 172.21.15.43:0/786518645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a04004810 0x7f2a1c159200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 >> 172.21.15.43:0/786518645 conn(0x7f2a1c0ce960 msgr2=0x7f2a1c15cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 shutdown_connections 2024-03-30T13:10:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:33.756+0000 7f2a20fa3640 1 -- 172.21.15.43:0/786518645 wait complete. 2024-03-30T13:10:33.777 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-30T13:10:33.777 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-30T13:10:33.929 INFO:tasks.ceph.mgr.y.smithi043.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-30T13:10:33.929 INFO:tasks.ceph.mgr.y.smithi043.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-30T13:10:33.930 INFO:tasks.ceph.mgr.y.smithi043.stderr: from numpy import show_config as show_numpy_config 2024-03-30T13:10:33.973 INFO:tasks.ceph.mgr.x.smithi113.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-30T13:10:33.973 INFO:tasks.ceph.mgr.x.smithi113.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-30T13:10:33.973 INFO:tasks.ceph.mgr.x.smithi113.stderr: from numpy import show_config as show_numpy_config 2024-03-30T13:10:33.988 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:33.980+0000 7fa7e3a4e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T13:10:34.020 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:34.017+0000 7fd36e30d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-30T13:10:34.120 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-03-30T13:10:34.117+0000 7f534ab5e7c0 -1 Falling back to public interface 2024-03-30T13:10:34.122 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-03-30T13:10:34.116+0000 7f38575c27c0 -1 Falling back to public interface 2024-03-30T13:10:34.122 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-03-30T13:10:34.116+0000 7f2cde9417c0 -1 Falling back to public interface 2024-03-30T13:10:34.129 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-03-30T13:10:34.124+0000 7fa7e01527c0 -1 Falling back to public interface 2024-03-30T13:10:34.136 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-03-30T13:10:34.133+0000 7fd8910ce7c0 -1 Falling back to public interface 2024-03-30T13:10:34.148 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-03-30T13:10:34.145+0000 7ff7a67c67c0 -1 Falling back to public interface 2024-03-30T13:10:34.152 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-03-30T13:10:34.149+0000 7f4eb07a17c0 -1 Falling back to public interface 2024-03-30T13:10:34.153 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-03-30T13:10:34.148+0000 7f94ebb217c0 -1 Falling back to public interface 2024-03-30T13:10:34.167 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:34.161+0000 7fd36e30d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:10:34.173 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:34.168+0000 7fa7e3a4e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:10:34.323 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:34.317+0000 7fd36e30d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T13:10:34.344 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:34.336+0000 7fa7e3a4e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-30T13:10:34.385 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:34.381+0000 7fd36e30d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T13:10:34.410 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:34.404+0000 7fa7e3a4e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-30T13:10:34.519 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:34.513+0000 7fd36e30d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:10:34.551 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:34.544+0000 7fa7e3a4e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-30T13:10:34.585 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:34.700 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.692+0000 7f3483148640 1 Processor -- start 2024-03-30T13:10:34.700 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.692+0000 7f3483148640 1 -- start start 2024-03-30T13:10:34.702 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3483148640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f347c145aa0 0x7f347c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.702 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3483148640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f347c06bea0 0x7f347c0d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.702 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3483148640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f347c0b6c20 con 0x7f347c15d280 2024-03-30T13:10:34.702 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3483148640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f347c0b3f30 con 0x7f347c145aa0 2024-03-30T13:10:34.702 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3483148640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f347c0b4210 con 0x7f347c06bea0 2024-03-30T13:10:34.702 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3480ee4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f347c145aa0 0x7f347c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.702 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f34816e5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f347c15d280 0x7f347c15fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53148/0 (socket says 172.21.15.43:53148) 2024-03-30T13:10:34.703 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f34816e5640 1 -- 172.21.15.43:0/2101063933 learned_addr learned my addr 172.21.15.43:0/2101063933 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:34.703 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f347bfff640 1 --2- 172.21.15.43:0/2101063933 >> v2:172.21.15.113:3300/0 conn(0x7f347c06bea0 0x7f347c0d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.703 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3480ee4640 1 -- 172.21.15.43:0/2101063933 >> v2:172.21.15.113:3300/0 conn(0x7f347c06bea0 msgr2=0x7f347c0d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:34.703 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3480ee4640 1 --2- 172.21.15.43:0/2101063933 >> v2:172.21.15.113:3300/0 conn(0x7f347c06bea0 0x7f347c0d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.703 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3480ee4640 1 -- 172.21.15.43:0/2101063933 >> v1:172.21.15.43:6789/0 conn(0x7f347c15d280 legacy=0x7f347c15fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.703 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3480ee4640 1 -- 172.21.15.43:0/2101063933 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c0b6ec0 con 0x7f347c145aa0 2024-03-30T13:10:34.704 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f3480ee4640 1 --2- 172.21.15.43:0/2101063933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c145aa0 0x7f347c0cfb20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f347c074140 tx=0x7f346801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912f666d391f816 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.704 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f347b7fe640 1 -- 172.21.15.43:0/2101063933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680b0070 con 0x7f347c145aa0 2024-03-30T13:10:34.704 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f347b7fe640 1 -- 172.21.15.43:0/2101063933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34680ab600 con 0x7f347c145aa0 2024-03-30T13:10:34.704 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.696+0000 7f347b7fe640 1 -- 172.21.15.43:0/2101063933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34680ab9d0 con 0x7f347c145aa0 2024-03-30T13:10:34.705 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- 172.21.15.43:0/2101063933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c145aa0 msgr2=0x7f347c0cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.705 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 --2- 172.21.15.43:0/2101063933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c145aa0 0x7f347c0cfb20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f347c074140 tx=0x7f346801c480 comp rx=0 tx=0).stop 2024-03-30T13:10:34.705 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- 172.21.15.43:0/2101063933 shutdown_connections 2024-03-30T13:10:34.705 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 --2- 172.21.15.43:0/2101063933 >> v2:172.21.15.113:3300/0 conn(0x7f347c06bea0 0x7f347c0d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.705 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 --2- 172.21.15.43:0/2101063933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c145aa0 0x7f347c0cfb20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.706 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- 172.21.15.43:0/2101063933 >> 172.21.15.43:0/2101063933 conn(0x7f347c0725d0 msgr2=0x7f347c0729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.706 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- 172.21.15.43:0/2101063933 shutdown_connections 2024-03-30T13:10:34.706 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- 172.21.15.43:0/2101063933 wait complete. 2024-03-30T13:10:34.706 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 Processor -- start 2024-03-30T13:10:34.706 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- start start 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c06bea0 0x7f347c14e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c145aa0 0x7f347c070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c15d280 0x7f347c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f347c169520 con 0x7f347c15d280 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f347c1693a0 con 0x7f347c145aa0 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f347c1696a0 con 0x7f347c06bea0 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c15d280 0x7f347c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c15d280 0x7f347c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60180/0 (socket says 172.21.15.43:60180) 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 -- 172.21.15.43:0/1626915256 learned_addr learned my addr 172.21.15.43:0/1626915256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f347bfff640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c145aa0 0x7f347c070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3480ee4640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c06bea0 0x7f347c14e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:34.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 -- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c06bea0 msgr2=0x7f347c14e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c06bea0 0x7f347c14e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 -- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c145aa0 msgr2=0x7f347c070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c145aa0 0x7f347c070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 -- 172.21.15.43:0/1626915256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347c06e480 con 0x7f347c15d280 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f347bfff640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c145aa0 0x7f347c070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34816e5640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c15d280 0x7f347c0ca2e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f347400cab0 tx=0x7f347400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34797fa640 1 -- 172.21.15.43:0/1626915256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474017070 con 0x7f347c15d280 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34797fa640 1 -- 172.21.15.43:0/1626915256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f347400ac30 con 0x7f347c15d280 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f34797fa640 1 -- 172.21.15.43:0/1626915256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3474012400 con 0x7f347c15d280 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347c14ede0 con 0x7f347c15d280 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.700+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347c06dec0 con 0x7f347c15d280 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.704+0000 7f34797fa640 1 -- 172.21.15.43:0/1626915256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f347401a070 con 0x7f347c15d280 2024-03-30T13:10:34.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.704+0000 7f34797fa640 1 -- 172.21.15.43:0/1626915256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f347401cde0 con 0x7f347c15d280 2024-03-30T13:10:34.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.704+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f347c068fa0 con 0x7f347c15d280 2024-03-30T13:10:34.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.704+0000 7f34797fa640 1 -- 172.21.15.43:0/1626915256 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f347401c710 con 0x7f347c15d280 2024-03-30T13:10:34.768 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.760+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f347c0d0b10 con 0x7f347c15d280 2024-03-30T13:10:34.769 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f34797fa640 1 -- 172.21.15.43:0/1626915256 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f347401c8f0 con 0x7f347c15d280 2024-03-30T13:10:34.769 INFO:teuthology.misc.health.smithi043.stdout: 2024-03-30T13:10:34.769 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":10,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:10:33.478764+0000","last_up_change":"0.000000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:34.770 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c15d280 msgr2=0x7f347c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:34.770 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c15d280 0x7f347c0ca2e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f347400cab0 tx=0x7f347400cf70 comp rx=0 tx=0).stop 2024-03-30T13:10:34.770 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 shutdown_connections 2024-03-30T13:10:34.770 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f347c15d280 0x7f347c0ca2e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.771 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f347c145aa0 0x7f347c070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.771 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 --2- 172.21.15.43:0/1626915256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f347c06bea0 0x7f347c14e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:34.771 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 >> 172.21.15.43:0/1626915256 conn(0x7f347c0725d0 msgr2=0x7f347c15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:34.771 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 shutdown_connections 2024-03-30T13:10:34.771 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:34.764+0000 7f3483148640 1 -- 172.21.15.43:0/1626915256 wait complete. 2024-03-30T13:10:34.784 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-30T13:10:35.118 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:35.113+0000 7fd36e30d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T13:10:35.185 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:35.180+0000 7fa7e3a4e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-30T13:10:35.194 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:35.189+0000 7fd36e30d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T13:10:35.264 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:35.256+0000 7fa7e3a4e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-30T13:10:35.375 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:35.369+0000 7fd36e30d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T13:10:35.448 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:35.445+0000 7fd36e30d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T13:10:35.454 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:35.448+0000 7fa7e3a4e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-30T13:10:35.531 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:35.524+0000 7fa7e3a4e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-30T13:10:35.736 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:35.733+0000 7fd36e30d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T13:10:35.797 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:35.793+0000 7fd36e30d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T13:10:35.837 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:35.832+0000 7fa7e3a4e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-30T13:10:35.902 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-03-30T13:10:35.896+0000 7fa7e3a4e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-30T13:10:36.840 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:36.837+0000 7fd315759640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-30T13:10:36.840 INFO:tasks.ceph.mgr.x.smithi113.stderr:2024-03-30T13:10:36.837+0000 7fd315759640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-30T13:10:37.759 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-03-30T13:10:37.753+0000 7ff7a67c67c0 -1 osd.5 0 log_to_monitors true 2024-03-30T13:10:37.937 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-03-30T13:10:37.933+0000 7fd8910ce7c0 -1 osd.7 0 log_to_monitors true 2024-03-30T13:10:37.947 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-03-30T13:10:37.940+0000 7fa7e01527c0 -1 osd.1 0 log_to_monitors true 2024-03-30T13:10:37.948 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-03-30T13:10:37.945+0000 7f534ab5e7c0 -1 osd.6 0 log_to_monitors true 2024-03-30T13:10:37.954 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-03-30T13:10:37.948+0000 7f2cde9417c0 -1 osd.3 0 log_to_monitors true 2024-03-30T13:10:38.042 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-03-30T13:10:38.036+0000 7f94ebb217c0 -1 osd.0 0 log_to_monitors true 2024-03-30T13:10:38.047 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-03-30T13:10:38.045+0000 7f4eb07a17c0 -1 osd.4 0 log_to_monitors true 2024-03-30T13:10:38.049 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-03-30T13:10:38.044+0000 7f38575c27c0 -1 osd.2 0 log_to_monitors true 2024-03-30T13:10:39.520 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-03-30T13:10:39.517+0000 7ff7a276f640 -1 osd.5 0 waiting for initial osdmap 2024-03-30T13:10:39.553 INFO:tasks.ceph.osd.5.smithi113.stderr:2024-03-30T13:10:39.549+0000 7ff79a5a3640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.124 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-03-30T13:10:41.117+0000 7fd88d077640 -1 osd.7 0 waiting for initial osdmap 2024-03-30T13:10:41.148 INFO:tasks.ceph.osd.7.smithi113.stderr:2024-03-30T13:10:41.145+0000 7fd884eab640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.168 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-03-30T13:10:41.165+0000 7f5347319640 -1 osd.6 0 waiting for initial osdmap 2024-03-30T13:10:41.187 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-03-30T13:10:41.181+0000 7f4eac74a640 -1 osd.4 0 waiting for initial osdmap 2024-03-30T13:10:41.192 INFO:tasks.ceph.osd.6.smithi113.stderr:2024-03-30T13:10:41.189+0000 7f533e93b640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.204 INFO:tasks.ceph.osd.4.smithi113.stderr:2024-03-30T13:10:41.201+0000 7f4ea457e640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.449 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-03-30T13:10:41.440+0000 7f2cda8ea640 -1 osd.3 0 waiting for initial osdmap 2024-03-30T13:10:41.469 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-03-30T13:10:41.464+0000 7f2cd271e640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.491 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-03-30T13:10:41.484+0000 7fa7dc0fb640 -1 osd.1 0 waiting for initial osdmap 2024-03-30T13:10:41.510 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-03-30T13:10:41.504+0000 7f385356b640 -1 osd.2 0 waiting for initial osdmap 2024-03-30T13:10:41.521 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-03-30T13:10:41.516+0000 7fa7d3f2f640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.528 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-03-30T13:10:41.520+0000 7f94e7aca640 -1 osd.0 0 waiting for initial osdmap 2024-03-30T13:10:41.533 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-03-30T13:10:41.528+0000 7f384b39f640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.569 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-03-30T13:10:41.564+0000 7f94df8fe640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-30T13:10:41.594 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:41.708 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.700+0000 7f2483c28640 1 Processor -- start 2024-03-30T13:10:41.709 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f2483c28640 1 -- start start 2024-03-30T13:10:41.710 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f2483c28640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f247c0b73d0 0x7f247c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f2483c28640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f247c0cfcb0 0x7f247c0d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f2483c28640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f247c0b38e0 con 0x7f247c15e390 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f2483c28640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f247c0d22e0 con 0x7f247c0b73d0 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f2483c28640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f247c06ced0 con 0x7f247c0cfcb0 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f247c0b73d0 0x7f247c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f247c0b73d0 0x7f247c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44604/0 (socket says 172.21.15.43:44604) 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 -- 172.21.15.43:0/2561740059 learned_addr learned my addr 172.21.15.43:0/2561740059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24821c5640 1 --2- 172.21.15.43:0/2561740059 >> v2:172.21.15.113:3300/0 conn(0x7f247c0cfcb0 0x7f247c0d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 -- 172.21.15.43:0/2561740059 >> v2:172.21.15.113:3300/0 conn(0x7f247c0cfcb0 msgr2=0x7f247c0d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 --2- 172.21.15.43:0/2561740059 >> v2:172.21.15.113:3300/0 conn(0x7f247c0cfcb0 0x7f247c0d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 -- 172.21.15.43:0/2561740059 >> v1:172.21.15.113:6789/0 conn(0x7f247c15e390 legacy=0x7f247c0cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 -- 172.21.15.43:0/2561740059 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c06c910 con 0x7f247c0b73d0 2024-03-30T13:10:41.711 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24821c5640 1 --2- 172.21.15.43:0/2561740059 >> v2:172.21.15.113:3300/0 conn(0x7f247c0cfcb0 0x7f247c0d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:41.712 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24811c3640 1 --2- 172.21.15.43:0/2561740059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0b73d0 0x7f247c06c330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f247c0ce7f0 tx=0x7f246c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565f1f0fa465482c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.712 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24809c2640 1 -- 172.21.15.43:0/2561740059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0b5070 con 0x7f247c0b73d0 2024-03-30T13:10:41.712 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24809c2640 1 -- 172.21.15.43:0/2561740059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c0a89d0 con 0x7f247c0b73d0 2024-03-30T13:10:41.712 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.704+0000 7f24809c2640 1 -- 172.21.15.43:0/2561740059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0a8da0 con 0x7f247c0b73d0 2024-03-30T13:10:41.713 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- 172.21.15.43:0/2561740059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0b73d0 msgr2=0x7f247c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.713 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 --2- 172.21.15.43:0/2561740059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0b73d0 0x7f247c06c330 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f247c0ce7f0 tx=0x7f246c01ecd0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.713 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- 172.21.15.43:0/2561740059 shutdown_connections 2024-03-30T13:10:41.713 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 --2- 172.21.15.43:0/2561740059 >> v2:172.21.15.113:3300/0 conn(0x7f247c0cfcb0 0x7f247c0d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.714 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 --2- 172.21.15.43:0/2561740059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0b73d0 0x7f247c06c330 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.714 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- 172.21.15.43:0/2561740059 >> 172.21.15.43:0/2561740059 conn(0x7f247c0d4220 msgr2=0x7f247c0d4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.714 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- 172.21.15.43:0/2561740059 shutdown_connections 2024-03-30T13:10:41.714 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- 172.21.15.43:0/2561740059 wait complete. 2024-03-30T13:10:41.714 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 Processor -- start 2024-03-30T13:10:41.714 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- start start 2024-03-30T13:10:41.716 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247c0b73d0 0x7f247c1ea940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.716 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0cfcb0 0x7f247c150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.716 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247c15e390 0x7f247c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.716 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f247c15e770 con 0x7f247c0cfcb0 2024-03-30T13:10:41.716 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f247c0d1f70 con 0x7f247c0b73d0 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f247c15e8f0 con 0x7f247c15e390 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0cfcb0 0x7f247c150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0cfcb0 0x7f247c150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44614/0 (socket says 172.21.15.43:44614) 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 -- 172.21.15.43:0/2993168006 learned_addr learned my addr 172.21.15.43:0/2993168006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24821c5640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247c15e390 0x7f247c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24819c4640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247c0b73d0 0x7f247c1ea940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 -- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247c15e390 msgr2=0x7f247c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247c15e390 0x7f247c0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 -- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247c0b73d0 msgr2=0x7f247c1ea940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247c0b73d0 0x7f247c1ea940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 -- 172.21.15.43:0/2993168006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c0c7d50 con 0x7f247c0cfcb0 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24811c3640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0cfcb0 0x7f247c150770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f246c01d380 tx=0x7f246c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24727fc640 1 -- 172.21.15.43:0/2993168006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0b5070 con 0x7f247c0cfcb0 2024-03-30T13:10:41.717 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24727fc640 1 -- 172.21.15.43:0/2993168006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f246c0b0b60 con 0x7f247c0cfcb0 2024-03-30T13:10:41.718 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c0c8030 con 0x7f247c0cfcb0 2024-03-30T13:10:41.718 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f24727fc640 1 -- 172.21.15.43:0/2993168006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0bb410 con 0x7f247c0cfcb0 2024-03-30T13:10:41.718 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.708+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c0c85f0 con 0x7f247c0cfcb0 2024-03-30T13:10:41.719 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.712+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f247c068fa0 con 0x7f247c0cfcb0 2024-03-30T13:10:41.719 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.712+0000 7f24727fc640 1 -- 172.21.15.43:0/2993168006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) ==== 87747+0+0 (secure 0 0 0) 0x7f246c0c2060 con 0x7f247c0cfcb0 2024-03-30T13:10:41.719 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.712+0000 7f24727fc640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2460067f30 0x7f246006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:41.719 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.712+0000 7f24727fc640 1 -- 172.21.15.43:0/2993168006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f246c122af0 con 0x7f247c0cfcb0 2024-03-30T13:10:41.719 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.712+0000 7f24819c4640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2460067f30 0x7f246006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:41.719 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.712+0000 7f24819c4640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2460067f30 0x7f246006a3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f247c0caa70 tx=0x7f2478008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:41.722 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.716+0000 7f24727fc640 1 -- 172.21.15.43:0/2993168006 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f246c018830 con 0x7f247c0cfcb0 2024-03-30T13:10:41.881 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f247c0cf030 con 0x7f247c0cfcb0 2024-03-30T13:10:41.881 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f24727fc640 1 -- 172.21.15.43:0/2993168006 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7590 (secure 0 0 0) 0x7f246c0c2350 con 0x7f247c0cfcb0 2024-03-30T13:10:41.882 INFO:teuthology.misc.health.smithi043.stdout: 2024-03-30T13:10:41.882 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":14,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:10:40.870145+0000","last_up_change":"2024-03-30T13:10:39.838387+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:41.884 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2460067f30 msgr2=0x7f246006a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.884 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2460067f30 0x7f246006a3f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f247c0caa70 tx=0x7f2478008040 comp rx=0 tx=0).stop 2024-03-30T13:10:41.884 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0cfcb0 msgr2=0x7f247c150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:41.884 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0cfcb0 0x7f247c150770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f246c01d380 tx=0x7f246c01ba70 comp rx=0 tx=0).stop 2024-03-30T13:10:41.884 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 shutdown_connections 2024-03-30T13:10:41.884 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2460067f30 0x7f246006a3f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.885 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f247c15e390 0x7f247c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.885 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f247c0cfcb0 0x7f247c150770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.885 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 --2- 172.21.15.43:0/2993168006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f247c0b73d0 0x7f247c1ea940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:41.885 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 >> 172.21.15.43:0/2993168006 conn(0x7f247c0d4220 msgr2=0x7f247c160b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:41.885 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 shutdown_connections 2024-03-30T13:10:41.885 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:41.876+0000 7f2483c28640 1 -- 172.21.15.43:0/2993168006 wait complete. 2024-03-30T13:10:41.899 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-03-30T13:10:48.706 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:48.833 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f635b3b6640 1 Processor -- start 2024-03-30T13:10:48.833 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f635b3b6640 1 -- start start 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f635b3b6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f635406e620 0x7f635406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f635b3b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63540d0310 0x7f635406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f635b3b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f635406ef40 con 0x7f63540cfed0 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f635b3b6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6354162030 con 0x7f635406e620 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f635b3b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6354162280 con 0x7f63540d0310 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359152640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f635406e620 0x7f635406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359152640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f635406e620 0x7f635406ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44624/0 (socket says 172.21.15.43:44624) 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359152640 1 -- 172.21.15.43:0/147235648 learned_addr learned my addr 172.21.15.43:0/147235648 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359953640 1 --2- 172.21.15.43:0/147235648 >> v2:172.21.15.113:3300/0 conn(0x7f63540d0310 0x7f635406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359152640 1 -- 172.21.15.43:0/147235648 >> v2:172.21.15.113:3300/0 conn(0x7f63540d0310 msgr2=0x7f635406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:48.835 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359152640 1 --2- 172.21.15.43:0/147235648 >> v2:172.21.15.113:3300/0 conn(0x7f63540d0310 0x7f635406d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.836 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359152640 1 -- 172.21.15.43:0/147235648 >> v1:172.21.15.113:6789/0 conn(0x7f63540cfed0 legacy=0x7f635406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:48.836 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.828+0000 7f6359152640 1 -- 172.21.15.43:0/147235648 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6354162560 con 0x7f635406e620 2024-03-30T13:10:48.838 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f6359152640 1 --2- 172.21.15.43:0/147235648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f635406ea00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f63540b3000 tx=0x7f635001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45252c4e029d42a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:48.838 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f634bfff640 1 -- 172.21.15.43:0/147235648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500ab600 con 0x7f635406e620 2024-03-30T13:10:48.838 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f634bfff640 1 -- 172.21.15.43:0/147235648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63500b0070 con 0x7f635406e620 2024-03-30T13:10:48.838 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f634bfff640 1 -- 172.21.15.43:0/147235648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500ab9c0 con 0x7f635406e620 2024-03-30T13:10:48.840 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 -- 172.21.15.43:0/147235648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 msgr2=0x7f635406ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.840 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 --2- 172.21.15.43:0/147235648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f635406ea00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f63540b3000 tx=0x7f635001c400 comp rx=0 tx=0).stop 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 -- 172.21.15.43:0/147235648 shutdown_connections 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 --2- 172.21.15.43:0/147235648 >> v2:172.21.15.113:3300/0 conn(0x7f63540d0310 0x7f635406d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 --2- 172.21.15.43:0/147235648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f635406ea00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 -- 172.21.15.43:0/147235648 >> 172.21.15.43:0/147235648 conn(0x7f635406bc10 msgr2=0x7f63540cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 -- 172.21.15.43:0/147235648 shutdown_connections 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 -- 172.21.15.43:0/147235648 wait complete. 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 Processor -- start 2024-03-30T13:10:48.841 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.832+0000 7f635b3b6640 1 -- start start 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f6354150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63540cfed0 0x7f6354158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63540d0310 0x7f6354152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6354155920 con 0x7f635406e620 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f63541557a0 con 0x7f63540d0310 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f63540d1d00 con 0x7f63540cfed0 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f6354150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f6354150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44626/0 (socket says 172.21.15.43:44626) 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 -- 172.21.15.43:0/3724646876 learned_addr learned my addr 172.21.15.43:0/3724646876 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359953640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63540d0310 0x7f6354152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.844 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6358951640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63540cfed0 0x7f6354158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 -- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63540cfed0 msgr2=0x7f6354158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63540cfed0 0x7f6354158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 -- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63540d0310 msgr2=0x7f6354152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63540d0310 0x7f6354152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 -- 172.21.15.43:0/3724646876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6354154450 con 0x7f635406e620 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6358951640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63540cfed0 0x7f6354158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359953640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63540d0310 0x7f6354152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6359152640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f6354150d90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63500ad040 tx=0x7f635001ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6349ffb640 1 -- 172.21.15.43:0/3724646876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500be5f0 con 0x7f635406e620 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6349ffb640 1 -- 172.21.15.43:0/3724646876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63500be7b0 con 0x7f635406e620 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6354154730 con 0x7f635406e620 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6349ffb640 1 -- 172.21.15.43:0/3724646876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63500beba0 con 0x7f635406e620 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6354154cf0 con 0x7f635406e620 2024-03-30T13:10:48.845 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6354068fa0 con 0x7f635406e620 2024-03-30T13:10:48.846 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6349ffb640 1 -- 172.21.15.43:0/3724646876 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f63500aa070 con 0x7f635406e620 2024-03-30T13:10:48.846 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6349ffb640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6338067fc0 0x7f633806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:48.846 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6349ffb640 1 -- 172.21.15.43:0/3724646876 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f6350120b60 con 0x7f635406e620 2024-03-30T13:10:48.846 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.836+0000 7f6358951640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6338067fc0 0x7f633806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:48.846 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.840+0000 7f6358951640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6338067fc0 0x7f633806a480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63540b37e0 tx=0x7f6344009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:48.849 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:48.844+0000 7f6349ffb640 1 -- 172.21.15.43:0/3724646876 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f63500160b0 con 0x7f635406e620 2024-03-30T13:10:49.011 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.004+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6354068fa0 con 0x7f635406e620 2024-03-30T13:10:49.012 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f6349ffb640 1 -- 172.21.15.43:0/3724646876 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12860 (secure 0 0 0) 0x7f63500f16f0 con 0x7f635406e620 2024-03-30T13:10:49.013 INFO:teuthology.misc.health.smithi043.stdout: 2024-03-30T13:10:49.013 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":19,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:10:46.553543+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:49.015 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6338067fc0 msgr2=0x7f633806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.015 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6338067fc0 0x7f633806a480 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63540b37e0 tx=0x7f6344009040 comp rx=0 tx=0).stop 2024-03-30T13:10:49.015 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 msgr2=0x7f6354150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.015 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f6354150d90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f63500ad040 tx=0x7f635001ac40 comp rx=0 tx=0).stop 2024-03-30T13:10:49.015 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 shutdown_connections 2024-03-30T13:10:49.016 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6338067fc0 0x7f633806a480 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.016 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f63540d0310 0x7f6354152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.016 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63540cfed0 0x7f6354158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.016 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 --2- 172.21.15.43:0/3724646876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f635406e620 0x7f6354150d90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.016 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 >> 172.21.15.43:0/3724646876 conn(0x7f635406bc10 msgr2=0x7f63540b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.016 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 shutdown_connections 2024-03-30T13:10:49.016 INFO:teuthology.misc.health.smithi043.stderr:2024-03-30T13:10:49.008+0000 7f635b3b6640 1 -- 172.21.15.43:0/3724646876 wait complete. 2024-03-30T13:10:49.030 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-30T13:10:49.031 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-30T13:10:49.031 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T13:10:49.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T13:10:49.034 INFO:tasks.ceph.mds.a:Started 2024-03-30T13:10:49.034 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T13:10:49.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T13:10:49.036 INFO:tasks.ceph.mds.c:Started 2024-03-30T13:10:49.036 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T13:10:49.036 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T13:10:49.039 INFO:tasks.ceph.mds.b:Started 2024-03-30T13:10:49.039 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T13:10:49.039 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T13:10:49.042 INFO:tasks.ceph.mds.d:Started 2024-03-30T13:10:49.042 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-30T13:10:49.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:49.101 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-03-30T13:10:49.109 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-03-30T13:10:49.116 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-03-30T13:10:49.134 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-03-30T13:10:49.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab66df2640 1 Processor -- start 2024-03-30T13:10:49.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab66df2640 1 -- start start 2024-03-30T13:10:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab66df2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 0x7fab600b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab66df2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fab600ad8e0 0x7fab6006d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab66df2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab600b08a0 con 0x7fab6006dcc0 2024-03-30T13:10:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab66df2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fab6006f190 con 0x7fab600ad8e0 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab66df2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fab6006f3e0 con 0x7fab6015e400 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab6538f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fab6006dcc0 0x7fab6006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38788/0 (socket says 172.21.15.43:38788) 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab6538f640 1 -- 172.21.15.43:0/1216064299 learned_addr learned my addr 172.21.15.43:0/1216064299 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.144+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743488280 0 0) 0x7fab600b08a0 con 0x7fab6006dcc0 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab50003530 con 0x7fab6006dcc0 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5ffff640 1 --2- 172.21.15.43:0/1216064299 >> v2:172.21.15.113:3300/0 conn(0x7fab600ad8e0 0x7fab6006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 0x7fab600b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3913929615 0 0) 0x7fab50003530 con 0x7fab6006dcc0 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 msgr2=0x7fab600b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 0x7fab600b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 >> v2:172.21.15.113:3300/0 conn(0x7fab600ad8e0 msgr2=0x7fab6006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 --2- 172.21.15.43:0/1216064299 >> v2:172.21.15.113:3300/0 conn(0x7fab600ad8e0 0x7fab6006d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6006f6c0 con 0x7fab6006dcc0 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fab58001a00 con 0x7fab6006dcc0 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 >> v1:172.21.15.43:6789/0 conn(0x7fab6006dcc0 legacy=0x7fab6006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab500039c0 0x7fab50003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 0x7fab50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab50004d50 0x7fab500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab6006f190 con 0x7fab500039c0 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fab6006f3e0 con 0x7fab50004d50 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fab5000ba80 con 0x7fab6015e400 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab500039c0 0x7fab50003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5ffff640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 0x7fab50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab6538f640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab50004d50 0x7fab500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 -- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 msgr2=0x7fab50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab6015e400 0x7fab50004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 -- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab50004d50 msgr2=0x7fab500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab50004d50 0x7fab500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 -- 172.21.15.43:0/1216064299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab50003530 con 0x7fab500039c0 2024-03-30T13:10:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab64b8e640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab500039c0 0x7fab50003de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00ba60 tx=0x7fab4c01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e73beaa88090c82 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0b3070 con 0x7fab500039c0 2024-03-30T13:10:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c0ae410 con 0x7fab500039c0 2024-03-30T13:10:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab5f7fe640 1 -- 172.21.15.43:0/1216064299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c00b600 con 0x7fab500039c0 2024-03-30T13:10:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab66df2640 1 -- 172.21.15.43:0/1216064299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab500039c0 msgr2=0x7fab50003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab66df2640 1 --2- 172.21.15.43:0/1216064299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab500039c0 0x7fab50003de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00ba60 tx=0x7fab4c01c680 comp rx=0 tx=0).stop 2024-03-30T13:10:49.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab6538f640 1 -- 172.21.15.43:0/1216064299 reap_dead start 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab66df2640 1 -- 172.21.15.43:0/1216064299 shutdown_connections 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab66df2640 1 -- 172.21.15.43:0/1216064299 >> 172.21.15.43:0/1216064299 conn(0x7fab60072500 msgr2=0x7fab60072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab66df2640 1 -- 172.21.15.43:0/1216064299 shutdown_connections 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.148+0000 7fab66df2640 1 -- 172.21.15.43:0/1216064299 wait complete. 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 Processor -- start 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 -- start start 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab500039c0 0x7fab60154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab6015a290 0x7fab6015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab60157d80 0x7fab60158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fab600affe0 con 0x7fab60157d80 2024-03-30T13:10:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fab600afe60 con 0x7fab6015a290 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fab60168f10 con 0x7fab500039c0 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab60157d80 0x7fab60158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab60157d80 0x7fab60158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44668/0 (socket says 172.21.15.43:44668) 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 -- 172.21.15.43:0/3792721911 learned_addr learned my addr 172.21.15.43:0/3792721911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab64b8e640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab500039c0 0x7fab60154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab5ffff640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab6015a290 0x7fab6015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 -- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab500039c0 msgr2=0x7fab60154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab500039c0 0x7fab60154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 -- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab6015a290 msgr2=0x7fab6015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab6015a290 0x7fab6015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 -- 172.21.15.43:0/3792721911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab60158740 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab6538f640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab60157d80 0x7fab60158200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab600b49d0 tx=0x7fab58007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab5d7fa640 1 -- 172.21.15.43:0/3792721911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5800aca0 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab5d7fa640 1 -- 172.21.15.43:0/3792721911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab58019070 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab60158a00 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab5d7fa640 1 -- 172.21.15.43:0/3792721911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab58013630 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6015ea30 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab60068890 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab5d7fa640 1 -- 172.21.15.43:0/3792721911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fab580137d0 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.152+0000 7fab5d7fa640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fab44067ff0 0x7fab4406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.156+0000 7fab5d7fa640 1 -- 172.21.15.43:0/3792721911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fab58086630 con 0x7fab60157d80 2024-03-30T13:10:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.156+0000 7fab64b8e640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fab44067ff0 0x7fab4406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.156+0000 7fab64b8e640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fab44067ff0 0x7fab4406a4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b0000 tx=0x7fab4c0046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.160+0000 7fab5d7fa640 1 -- 172.21.15.43:0/3792721911 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fab58056ae0 con 0x7fab60157d80 2024-03-30T13:10:49.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.312+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab6015d8e0 con 0x7fab60157d80 2024-03-30T13:10:49.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab5d7fa640 1 -- 172.21.15.43:0/3792721911 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12860 (secure 0 0 0) 0x7fab5805ade0 con 0x7fab60157d80 2024-03-30T13:10:49.321 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:10:49.321 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":19,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:10:46.553543+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fab44067ff0 msgr2=0x7fab4406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fab44067ff0 0x7fab4406a4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b0000 tx=0x7fab4c0046d0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab60157d80 msgr2=0x7fab60158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab60157d80 0x7fab60158200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab600b49d0 tx=0x7fab58007b20 comp rx=0 tx=0).stop 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 shutdown_connections 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fab44067ff0 0x7fab4406a4b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab60157d80 0x7fab60158200 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fab6015a290 0x7fab6015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 --2- 172.21.15.43:0/3792721911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fab500039c0 0x7fab60154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 >> 172.21.15.43:0/3792721911 conn(0x7fab60072500 msgr2=0x7fab601685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 shutdown_connections 2024-03-30T13:10:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.316+0000 7fab66df2640 1 -- 172.21.15.43:0/3792721911 wait complete. 2024-03-30T13:10:49.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:10:49.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:49.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd0419e7640 1 Processor -- start 2024-03-30T13:10:49.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd0419e7640 1 -- start start 2024-03-30T13:10:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd0419e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd03c1627c0 0x7fd03c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd0419e7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd03c0700f0 0x7fd03c0b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd0419e7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd03c0b4080 con 0x7fd03c070790 2024-03-30T13:10:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd0419e7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd03c0b4c50 con 0x7fd03c1627c0 2024-03-30T13:10:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd0419e7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd03c0b4320 con 0x7fd03c0700f0 2024-03-30T13:10:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd03c1627c0 0x7fd03c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd03c1627c0 0x7fd03c06b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44672/0 (socket says 172.21.15.43:44672) 2024-03-30T13:10:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03affd640 1 -- 172.21.15.43:0/2952669314 learned_addr learned my addr 172.21.15.43:0/2952669314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03b7fe640 1 --2- 172.21.15.43:0/2952669314 >> v2:172.21.15.113:3300/0 conn(0x7fd03c0700f0 0x7fd03c0b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03affd640 1 -- 172.21.15.43:0/2952669314 >> v2:172.21.15.113:3300/0 conn(0x7fd03c0700f0 msgr2=0x7fd03c0b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03affd640 1 --2- 172.21.15.43:0/2952669314 >> v2:172.21.15.113:3300/0 conn(0x7fd03c0700f0 0x7fd03c0b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03affd640 1 -- 172.21.15.43:0/2952669314 >> v1:172.21.15.113:6789/0 conn(0x7fd03c070790 legacy=0x7fd03c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03affd640 1 -- 172.21.15.43:0/2952669314 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c0b4600 con 0x7fd03c1627c0 2024-03-30T13:10:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.460+0000 7fd03b7fe640 1 --2- 172.21.15.43:0/2952669314 >> v2:172.21.15.113:3300/0 conn(0x7fd03c0700f0 0x7fd03c0b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:49.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd03affd640 1 --2- 172.21.15.43:0/2952669314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1627c0 0x7fd03c06b660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0d0ee0 tx=0x7fd02401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dac54d3633a7861 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd039ffb640 1 -- 172.21.15.43:0/2952669314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0240ab650 con 0x7fd03c1627c0 2024-03-30T13:10:49.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd039ffb640 1 -- 172.21.15.43:0/2952669314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0240b0070 con 0x7fd03c1627c0 2024-03-30T13:10:49.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd039ffb640 1 -- 172.21.15.43:0/2952669314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0240aba10 con 0x7fd03c1627c0 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 -- 172.21.15.43:0/2952669314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1627c0 msgr2=0x7fd03c06b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 --2- 172.21.15.43:0/2952669314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1627c0 0x7fd03c06b660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0d0ee0 tx=0x7fd02401c4b0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 -- 172.21.15.43:0/2952669314 shutdown_connections 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 --2- 172.21.15.43:0/2952669314 >> v2:172.21.15.113:3300/0 conn(0x7fd03c0700f0 0x7fd03c0b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 --2- 172.21.15.43:0/2952669314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1627c0 0x7fd03c06b660 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 -- 172.21.15.43:0/2952669314 >> 172.21.15.43:0/2952669314 conn(0x7fd03c0d1070 msgr2=0x7fd03c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 -- 172.21.15.43:0/2952669314 shutdown_connections 2024-03-30T13:10:49.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 -- 172.21.15.43:0/2952669314 wait complete. 2024-03-30T13:10:49.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 Processor -- start 2024-03-30T13:10:49.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.464+0000 7fd0419e7640 1 -- start start 2024-03-30T13:10:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd03c0700f0 0x7fd03c141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c070790 0x7fd03c162e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd03c1627c0 0x7fd03c163da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c16f6c0 con 0x7fd03c070790 2024-03-30T13:10:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c16f540 con 0x7fd03c1627c0 2024-03-30T13:10:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c16f840 con 0x7fd03c0700f0 2024-03-30T13:10:49.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c070790 0x7fd03c162e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c070790 0x7fd03c162e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44680/0 (socket says 172.21.15.43:44680) 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 -- 172.21.15.43:0/3612269717 learned_addr learned my addr 172.21.15.43:0/3612269717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03affd640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd03c0700f0 0x7fd03c141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03b7fe640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd03c1627c0 0x7fd03c163da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 -- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd03c0700f0 msgr2=0x7fd03c141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd03c0700f0 0x7fd03c141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 -- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd03c1627c0 msgr2=0x7fd03c163da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd03c1627c0 0x7fd03c163da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 -- 172.21.15.43:0/3612269717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c141960 con 0x7fd03c070790 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd03a7fc640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c070790 0x7fd03c162e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c480 tx=0x7fd03400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd033fff640 1 -- 172.21.15.43:0/3612269717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034012830 con 0x7fd03c070790 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd033fff640 1 -- 172.21.15.43:0/3612269717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd034017070 con 0x7fd03c070790 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd033fff640 1 -- 172.21.15.43:0/3612269717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034012c30 con 0x7fd03c070790 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c15b620 con 0x7fd03c070790 2024-03-30T13:10:49.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c153670 con 0x7fd03c070790 2024-03-30T13:10:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03c068900 con 0x7fd03c070790 2024-03-30T13:10:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd033fff640 1 -- 172.21.15.43:0/3612269717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd03401c460 con 0x7fd03c070790 2024-03-30T13:10:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd033fff640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd020067fc0 0x7fd02006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.468+0000 7fd033fff640 1 -- 172.21.15.43:0/3612269717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fd034085f30 con 0x7fd03c070790 2024-03-30T13:10:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.472+0000 7fd03affd640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd020067fc0 0x7fd02006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.472+0000 7fd03affd640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd020067fc0 0x7fd02006a480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd0240af000 tx=0x7fd024008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.476+0000 7fd033fff640 1 -- 172.21.15.43:0/3612269717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd034056ac0 con 0x7fd03c070790 2024-03-30T13:10:49.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.636+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd03c0cd860 con 0x7fd03c070790 2024-03-30T13:10:49.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.636+0000 7fd033fff640 1 -- 172.21.15.43:0/3612269717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7fd0340563c0 con 0x7fd03c070790 2024-03-30T13:10:49.644 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:10:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd020067fc0 msgr2=0x7fd02006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd020067fc0 0x7fd02006a480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd0240af000 tx=0x7fd024008040 comp rx=0 tx=0).stop 2024-03-30T13:10:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c070790 msgr2=0x7fd03c162e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c070790 0x7fd03c162e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd03400c480 tx=0x7fd03400c940 comp rx=0 tx=0).stop 2024-03-30T13:10:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 shutdown_connections 2024-03-30T13:10:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd020067fc0 0x7fd02006a480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd03c1627c0 0x7fd03c163da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c070790 0x7fd03c162e30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 --2- 172.21.15.43:0/3612269717 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd03c0700f0 0x7fd03c141420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 >> 172.21.15.43:0/3612269717 conn(0x7fd03c0d1070 msgr2=0x7fd03c161500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 shutdown_connections 2024-03-30T13:10:49.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.640+0000 7fd0419e7640 1 -- 172.21.15.43:0/3612269717 wait complete. 2024-03-30T13:10:49.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f49fcc640 1 Processor -- start 2024-03-30T13:10:49.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f49fcc640 1 -- start start 2024-03-30T13:10:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f49fcc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f440b41f0 0x7f3f440cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f49fcc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f4406ca50 0x7f3f440ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f49fcc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f440ccef0 con 0x7f3f4406c670 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f49fcc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3f440b34c0 con 0x7f3f4406ca50 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f49fcc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3f440b3710 con 0x7f3f440b41f0 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f43fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f4406ca50 0x7f3f440ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f43fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f4406ca50 0x7f3f440ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44682/0 (socket says 172.21.15.43:44682) 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f43fff640 1 -- 172.21.15.43:0/648798145 learned_addr learned my addr 172.21.15.43:0/648798145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f437fe640 1 --1- 172.21.15.43:0/648798145 >> v1:172.21.15.43:6789/0 conn(0x7f3f4406c670 0x7f3f4406e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38804/0 (socket says 172.21.15.43:38804) 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f42ffd640 1 --2- 172.21.15.43:0/648798145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f440b41f0 0x7f3f440cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f43fff640 1 -- 172.21.15.43:0/648798145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f440b41f0 msgr2=0x7f3f440cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f43fff640 1 --2- 172.21.15.43:0/648798145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f440b41f0 0x7f3f440cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f43fff640 1 -- 172.21.15.43:0/648798145 >> v1:172.21.15.43:6789/0 conn(0x7f3f4406c670 legacy=0x7f3f4406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f43fff640 1 -- 172.21.15.43:0/648798145 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f440b39f0 con 0x7f3f4406ca50 2024-03-30T13:10:49.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.780+0000 7f3f42ffd640 1 --2- 172.21.15.43:0/648798145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f440b41f0 0x7f3f440cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:49.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f43fff640 1 --2- 172.21.15.43:0/648798145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406ca50 0x7f3f440ce5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009e50 tx=0x7f3f3c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d1e05632788e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f427fc640 1 -- 172.21.15.43:0/648798145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0a5720 con 0x7f3f4406ca50 2024-03-30T13:10:49.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f427fc640 1 -- 172.21.15.43:0/648798145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0a58c0 con 0x7f3f4406ca50 2024-03-30T13:10:49.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f427fc640 1 -- 172.21.15.43:0/648798145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0a5c90 con 0x7f3f4406ca50 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- 172.21.15.43:0/648798145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406ca50 msgr2=0x7f3f440ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/648798145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406ca50 0x7f3f440ce5b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009e50 tx=0x7f3f3c01ca30 comp rx=0 tx=0).stop 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- 172.21.15.43:0/648798145 shutdown_connections 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/648798145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406ca50 0x7f3f440ce5b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/648798145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f440b41f0 0x7f3f440cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- 172.21.15.43:0/648798145 >> 172.21.15.43:0/648798145 conn(0x7f3f440cfee0 msgr2=0x7f3f440d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- 172.21.15.43:0/648798145 shutdown_connections 2024-03-30T13:10:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- 172.21.15.43:0/648798145 wait complete. 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 Processor -- start 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- start start 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406c670 0x7f3f4415aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f4406ca50 0x7f3f4415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f440b41f0 0x7f3f44133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44169070 con 0x7f3f4406c670 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44168ef0 con 0x7f3f440b41f0 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3f441691f0 con 0x7f3f4406ca50 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406c670 0x7f3f4415aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406c670 0x7f3f4415aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44696/0 (socket says 172.21.15.43:44696) 2024-03-30T13:10:49.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 -- 172.21.15.43:0/1332340366 learned_addr learned my addr 172.21.15.43:0/1332340366 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 -- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f4406ca50 msgr2=0x7f3f4415a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f42ffd640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f4406ca50 0x7f3f4415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f43fff640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f440b41f0 0x7f3f44133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f4406ca50 0x7f3f4415a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 -- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f440b41f0 msgr2=0x7f3f44133ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f440b41f0 0x7f3f44133ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 -- 172.21.15.43:0/1332340366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4415b0b0 con 0x7f3f4406c670 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f437fe640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406c670 0x7f3f4415aad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c370 tx=0x7f3f3400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f2bfff640 1 -- 172.21.15.43:0/1332340366 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340074e0 con 0x7f3f4406c670 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f2bfff640 1 -- 172.21.15.43:0/1332340366 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f34007680 con 0x7f3f4406c670 2024-03-30T13:10:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4415b390 con 0x7f3f4406c670 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.784+0000 7f3f2bfff640 1 -- 172.21.15.43:0/1332340366 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34004a30 con 0x7f3f4406c670 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.788+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4415b950 con 0x7f3f4406c670 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.788+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f44068890 con 0x7f3f4406c670 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.788+0000 7f3f2bfff640 1 -- 172.21.15.43:0/1332340366 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3f3401a020 con 0x7f3f4406c670 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.788+0000 7f3f2bfff640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f30067ff0 0x7f3f3006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.788+0000 7f3f2bfff640 1 -- 172.21.15.43:0/1332340366 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f3f34085920 con 0x7f3f4406c670 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.788+0000 7f3f42ffd640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f30067ff0 0x7f3f3006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:49.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.788+0000 7f3f42ffd640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f30067ff0 0x7f3f3006a4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:49.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.792+0000 7f3f2bfff640 1 -- 172.21.15.43:0/1332340366 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3f34056420 con 0x7f3f4406c670 2024-03-30T13:10:49.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.984+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f440b32a0 con 0x7f3f4406c670 2024-03-30T13:10:49.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.984+0000 7f3f2bfff640 1 -- 172.21.15.43:0/1332340366 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f3f34055d20 con 0x7f3f4406c670 2024-03-30T13:10:49.992 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:10:49.993 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-30T13:10:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-03-30T13:10:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f30067ff0 msgr2=0x7f3f3006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f30067ff0 0x7f3f3006a4b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3f38006fd0 tx=0x7f3f38008040 comp rx=0 tx=0).stop 2024-03-30T13:10:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406c670 msgr2=0x7f3f4415aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406c670 0x7f3f4415aad0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c370 tx=0x7f3f3400c830 comp rx=0 tx=0).stop 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 shutdown_connections 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f30067ff0 0x7f3f3006a4b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f440b41f0 0x7f3f44133ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f4406ca50 0x7f3f4415a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 --2- 172.21.15.43:0/1332340366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f4406c670 0x7f3f4415aad0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 >> 172.21.15.43:0/1332340366 conn(0x7f3f440cfee0 msgr2=0x7f3f44143630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 shutdown_connections 2024-03-30T13:10:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:49.988+0000 7f3f49fcc640 1 -- 172.21.15.43:0/1332340366 wait complete. 2024-03-30T13:10:52.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:10:52.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.136+0000 7fd9e7482640 1 Processor -- start 2024-03-30T13:10:52.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.136+0000 7fd9e7482640 1 -- start start 2024-03-30T13:10:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e7482640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd9e006c670 0x7fd9e00cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e7482640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 0x7fd9e00ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e7482640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9e00ccef0 con 0x7fd9e00b41f0 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e7482640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd9e00b34c0 con 0x7fd9e006c670 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e7482640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e00b3710 con 0x7fd9e006ca50 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e4a1d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd9e00b41f0 0x7fd9e006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53444/0 (socket says 172.21.15.43:53444) 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e4a1d640 1 -- 172.21.15.43:0/2825173813 learned_addr learned my addr 172.21.15.43:0/2825173813 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e521e640 1 --2- 172.21.15.43:0/2825173813 >> v2:172.21.15.113:3300/0 conn(0x7fd9e006c670 0x7fd9e00cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e5a1f640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 0x7fd9e00ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194454393 0 0) 0x7fd9e00ccef0 con 0x7fd9e00b41f0 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9cc003530 con 0x7fd9e00b41f0 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1402637067 0 0) 0x7fd9cc003530 con 0x7fd9e00b41f0 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 msgr2=0x7fd9e00ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 0x7fd9e00ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 >> v2:172.21.15.113:3300/0 conn(0x7fd9e006c670 msgr2=0x7fd9e00cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 --2- 172.21.15.43:0/2825173813 >> v2:172.21.15.113:3300/0 conn(0x7fd9e006c670 0x7fd9e00cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e00b39f0 con 0x7fd9e00b41f0 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fd9d4001680 con 0x7fd9e00b41f0 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 >> v1:172.21.15.43:6789/0 conn(0x7fd9e00b41f0 legacy=0x7fd9e006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0039c0 0x7fd9cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 0x7fd9cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9cc004d50 0x7fd9cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e00b34c0 con 0x7fd9cc0039c0 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e00b3710 con 0x7fd9cc004d50 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd9cc00b9d0 con 0x7fd9e006ca50 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e521e640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0039c0 0x7fd9cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e4a1d640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 0x7fd9cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e5a1f640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9cc004d50 0x7fd9cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e521e640 1 -- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 msgr2=0x7fd9cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e521e640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e006ca50 0x7fd9cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e521e640 1 -- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9cc004d50 msgr2=0x7fd9cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e521e640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9cc004d50 0x7fd9cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e521e640 1 -- 172.21.15.43:0/2825173813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc003530 con 0x7fd9cc0039c0 2024-03-30T13:10:52.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.140+0000 7fd9e5a1f640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9cc004d50 0x7fd9cc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e521e640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0039c0 0x7fd9cc003de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009fc0 tx=0x7fd9d801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ae5bc9b55feeb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d80b2070 con 0x7fd9cc0039c0 2024-03-30T13:10:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d80a5b40 con 0x7fd9cc0039c0 2024-03-30T13:10:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9d3fff640 1 -- 172.21.15.43:0/2825173813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d8009c10 con 0x7fd9cc0039c0 2024-03-30T13:10:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- 172.21.15.43:0/2825173813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0039c0 msgr2=0x7fd9cc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 --2- 172.21.15.43:0/2825173813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9cc0039c0 0x7fd9cc003de0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8009fc0 tx=0x7fd9d801ccc0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e5a1f640 1 -- 172.21.15.43:0/2825173813 reap_dead start 2024-03-30T13:10:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- 172.21.15.43:0/2825173813 shutdown_connections 2024-03-30T13:10:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- 172.21.15.43:0/2825173813 >> 172.21.15.43:0/2825173813 conn(0x7fd9e00cfee0 msgr2=0x7fd9e00d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- 172.21.15.43:0/2825173813 shutdown_connections 2024-03-30T13:10:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- 172.21.15.43:0/2825173813 wait complete. 2024-03-30T13:10:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 Processor -- start 2024-03-30T13:10:52.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- start start 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e00cb010 0x7fd9e015f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9e00cb3f0 0x7fd9e014e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9e014eaa0 0x7fd9e0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e00cef40 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd9e00cedc0 con 0x7fd9e014eaa0 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e7482640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd9e00cf0c0 con 0x7fd9e00cb010 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9e00cb3f0 0x7fd9e014e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9e00cb3f0 0x7fd9e014e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43906/0 (socket says 172.21.15.43:43906) 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 -- 172.21.15.43:0/4165255640 learned_addr learned my addr 172.21.15.43:0/4165255640 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e521e640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e00cb010 0x7fd9e015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e5a1f640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9e014eaa0 0x7fd9e0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 -- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e00cb010 msgr2=0x7fd9e015f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e00cb010 0x7fd9e015f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 -- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9e014eaa0 msgr2=0x7fd9e0143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9e014eaa0 0x7fd9e0143510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.144+0000 7fd9e4a1d640 1 -- 172.21.15.43:0/4165255640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9e0144a50 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9e4a1d640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9e00cb3f0 0x7fd9e014e560 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e700 tx=0x7fd9d400eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9d1ffb640 1 -- 172.21.15.43:0/4165255640 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4012590 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9d1ffb640 1 -- 172.21.15.43:0/4165255640 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d4018070 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9e0144d30 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9d1ffb640 1 -- 172.21.15.43:0/4165255640 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4012950 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9e015caf0 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9e015ccf0 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9d1ffb640 1 -- 172.21.15.43:0/4165255640 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd9d4012af0 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9d1ffb640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9c4067fc0 0x7fd9c406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9d1ffb640 1 -- 172.21.15.43:0/4165255640 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fd9d4085eb0 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9e521e640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9c4067fc0 0x7fd9c406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.148+0000 7fd9e521e640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9c4067fc0 0x7fd9c406a480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d8004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.152+0000 7fd9d1ffb640 1 -- 172.21.15.43:0/4165255640 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd9e015ccf0 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9e00b32a0 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9d1ffb640 1 -- 172.21.15.43:0/4165255640 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7fd9d40569b0 con 0x7fd9e00cb3f0 2024-03-30T13:10:52.358 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:10:52.358 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4239,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-30T13:10:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-03-30T13:10:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9c4067fc0 msgr2=0x7fd9c406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9c4067fc0 0x7fd9c406a480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd9d80af000 tx=0x7fd9d8004ac0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9e00cb3f0 msgr2=0x7fd9e014e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9e00cb3f0 0x7fd9e014e560 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400e700 tx=0x7fd9d400eb00 comp rx=0 tx=0).stop 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 shutdown_connections 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9c4067fc0 0x7fd9c406a480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9e014eaa0 0x7fd9e0143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9e00cb3f0 0x7fd9e014e560 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 --2- 172.21.15.43:0/4165255640 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9e00cb010 0x7fd9e015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 >> 172.21.15.43:0/4165255640 conn(0x7fd9e00cfee0 msgr2=0x7fd9e0071890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 shutdown_connections 2024-03-30T13:10:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.352+0000 7fd9e7482640 1 -- 172.21.15.43:0/4165255640 wait complete. 2024-03-30T13:10:52.378 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:10:52.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.488+0000 7f9600c6e640 1 Processor -- start 2024-03-30T13:10:52.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.488+0000 7f9600c6e640 1 -- start start 2024-03-30T13:10:52.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f9600c6e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc144270 0x7f95fc144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f9600c6e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95fc15f7a0 0x7f95fc06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f9600c6e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95fc0b4ba0 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f9600c6e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f95fc144fa0 con 0x7f95fc15f7a0 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f9600c6e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0b33f0 con 0x7f95fc144270 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95fc15f7a0 0x7f95fc06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f95fc15f7a0 0x7f95fc06c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43918/0 (socket says 172.21.15.43:43918) 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 -- 172.21.15.43:0/1066834697 learned_addr learned my addr 172.21.15.43:0/1066834697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95f9d9b640 1 --1- 172.21.15.43:0/1066834697 >> v1:172.21.15.43:6789/0 conn(0x7f95fc0b2ef0 0x7f95fc0b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53456/0 (socket says 172.21.15.43:53456) 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fa59c640 1 --2- 172.21.15.43:0/1066834697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc144270 0x7f95fc144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 -- 172.21.15.43:0/1066834697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc144270 msgr2=0x7f95fc144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 --2- 172.21.15.43:0/1066834697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc144270 0x7f95fc144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 -- 172.21.15.43:0/1066834697 >> v1:172.21.15.43:6789/0 conn(0x7f95fc0b2ef0 legacy=0x7f95fc0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95f959a640 1 -- 172.21.15.43:0/1066834697 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 866811227 0 0) 0x7f95fc0b4ba0 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 -- 172.21.15.43:0/1066834697 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc0b36d0 con 0x7f95fc15f7a0 2024-03-30T13:10:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95fad9d640 1 --2- 172.21.15.43:0/1066834697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc15f7a0 0x7f95fc06c670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d150 tx=0x7f95f401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8148d1de22739454 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.492+0000 7f95f959a640 1 -- 172.21.15.43:0/1066834697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f400b2d0 con 0x7f95fc15f7a0 2024-03-30T13:10:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95f959a640 1 -- 172.21.15.43:0/1066834697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f400b470 con 0x7f95fc15f7a0 2024-03-30T13:10:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95f959a640 1 -- 172.21.15.43:0/1066834697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f40a7c00 con 0x7f95fc15f7a0 2024-03-30T13:10:52.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- 172.21.15.43:0/1066834697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc15f7a0 msgr2=0x7f95fc06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 --2- 172.21.15.43:0/1066834697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc15f7a0 0x7f95fc06c670 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d150 tx=0x7f95f401ec00 comp rx=0 tx=0).stop 2024-03-30T13:10:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- 172.21.15.43:0/1066834697 shutdown_connections 2024-03-30T13:10:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 --2- 172.21.15.43:0/1066834697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc15f7a0 0x7f95fc06c670 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 --2- 172.21.15.43:0/1066834697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc144270 0x7f95fc144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- 172.21.15.43:0/1066834697 >> 172.21.15.43:0/1066834697 conn(0x7f95fc0cfee0 msgr2=0x7f95fc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- 172.21.15.43:0/1066834697 shutdown_connections 2024-03-30T13:10:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- 172.21.15.43:0/1066834697 wait complete. 2024-03-30T13:10:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 Processor -- start 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- start start 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc0b2ef0 0x7f95fc13ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc144270 0x7f95fc15acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc15f7a0 0x7f95fc138dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0cdd10 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc15fc30 con 0x7f95fc144270 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0cde90 con 0x7f95fc15f7a0 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc0b2ef0 0x7f95fc13ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc0b2ef0 0x7f95fc13ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43926/0 (socket says 172.21.15.43:43926) 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 -- 172.21.15.43:0/3053493514 learned_addr learned my addr 172.21.15.43:0/3053493514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fad9d640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc15f7a0 0x7f95fc138dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95f9d9b640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc144270 0x7f95fc15acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 -- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc15f7a0 msgr2=0x7f95fc138dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc15f7a0 0x7f95fc138dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 -- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc144270 msgr2=0x7f95fc15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc144270 0x7f95fc15acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 -- 172.21.15.43:0/3053493514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc13a9e0 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95f9d9b640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc144270 0x7f95fc15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95fa59c640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc0b2ef0 0x7f95fc13ba80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00d800 tx=0x7f95ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95eaffd640 1 -- 172.21.15.43:0/3053493514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00b6c0 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95eaffd640 1 -- 172.21.15.43:0/3053493514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95ec019070 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc13acc0 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f95eaffd640 1 -- 172.21.15.43:0/3053493514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00bae0 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.496+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc13b280 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.500+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc13b6b0 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.500+0000 7f95eaffd640 1 -- 172.21.15.43:0/3053493514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f95ec00bc80 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.500+0000 7f95eaffd640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f95dc067fc0 0x7f95dc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.500+0000 7f95eaffd640 1 -- 172.21.15.43:0/3053493514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f95ec086710 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.500+0000 7f95f9d9b640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f95dc067fc0 0x7f95dc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.500+0000 7f95f9d9b640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f95dc067fc0 0x7f95dc06a480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95fc13bfc0 tx=0x7f95f0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.504+0000 7f95eaffd640 1 -- 172.21.15.43:0/3053493514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f95ec057210 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.668+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95fc068890 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.668+0000 7f95eaffd640 1 -- 172.21.15.43:0/3053493514 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12860 (secure 0 0 0) 0x7f95ec056b10 con 0x7f95fc0b2ef0 2024-03-30T13:10:52.675 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:10:52.675 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":19,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:10:46.553543+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:10:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f95dc067fc0 msgr2=0x7f95dc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f95dc067fc0 0x7f95dc06a480 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95fc13bfc0 tx=0x7f95f0009040 comp rx=0 tx=0).stop 2024-03-30T13:10:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc0b2ef0 msgr2=0x7f95fc13ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc0b2ef0 0x7f95fc13ba80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00d800 tx=0x7f95ec00dcc0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 shutdown_connections 2024-03-30T13:10:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f95dc067fc0 0x7f95dc06a480 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f95fc15f7a0 0x7f95fc138dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f95fc144270 0x7f95fc15acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 --2- 172.21.15.43:0/3053493514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f95fc0b2ef0 0x7f95fc13ba80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 >> 172.21.15.43:0/3053493514 conn(0x7f95fc0cfee0 msgr2=0x7f95fc06e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 shutdown_connections 2024-03-30T13:10:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.672+0000 7f9600c6e640 1 -- 172.21.15.43:0/3053493514 wait complete. 2024-03-30T13:10:52.694 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:10:52.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:10:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be988c640 1 Processor -- start 2024-03-30T13:10:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be988c640 1 -- start start 2024-03-30T13:10:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be988c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4be40b56b0 0x7f4be4073370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be988c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4be40cea50 0x7f4be40cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be988c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be406c280 con 0x7f4be415c620 2024-03-30T13:10:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be988c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4be415ca00 con 0x7f4be40cea50 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be988c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4be40b5160 con 0x7f4be40b56b0 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4be40cea50 0x7f4be40cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4be40cea50 0x7f4be40cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43942/0 (socket says 172.21.15.43:43942) 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be37fe640 1 -- 172.21.15.43:0/3688322904 learned_addr learned my addr 172.21.15.43:0/3688322904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be27fc640 1 --2- 172.21.15.43:0/3688322904 >> v2:172.21.15.113:3300/0 conn(0x7f4be40b56b0 0x7f4be4073370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be37fe640 1 -- 172.21.15.43:0/3688322904 >> v2:172.21.15.113:3300/0 conn(0x7f4be40b56b0 msgr2=0x7f4be4073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be37fe640 1 --2- 172.21.15.43:0/3688322904 >> v2:172.21.15.113:3300/0 conn(0x7f4be40b56b0 0x7f4be4073370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be37fe640 1 -- 172.21.15.43:0/3688322904 >> v1:172.21.15.113:6789/0 conn(0x7f4be415c620 legacy=0x7f4be4072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.816+0000 7f4be37fe640 1 -- 172.21.15.43:0/3688322904 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be40b5440 con 0x7f4be40cea50 2024-03-30T13:10:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be37fe640 1 --2- 172.21.15.43:0/3688322904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40cea50 0x7f4be40cee80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4be4071070 tx=0x7f4bdc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d60ab26f5b32b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be1ffb640 1 -- 172.21.15.43:0/3688322904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0aa610 con 0x7f4be40cea50 2024-03-30T13:10:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be1ffb640 1 -- 172.21.15.43:0/3688322904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bdc0af070 con 0x7f4be40cea50 2024-03-30T13:10:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be1ffb640 1 -- 172.21.15.43:0/3688322904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0aa9d0 con 0x7f4be40cea50 2024-03-30T13:10:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- 172.21.15.43:0/3688322904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40cea50 msgr2=0x7f4be40cee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 --2- 172.21.15.43:0/3688322904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40cea50 0x7f4be40cee80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4be4071070 tx=0x7f4bdc01c490 comp rx=0 tx=0).stop 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- 172.21.15.43:0/3688322904 shutdown_connections 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 --2- 172.21.15.43:0/3688322904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40cea50 0x7f4be40cee80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 --2- 172.21.15.43:0/3688322904 >> v2:172.21.15.113:3300/0 conn(0x7f4be40b56b0 0x7f4be4073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- 172.21.15.43:0/3688322904 >> 172.21.15.43:0/3688322904 conn(0x7f4be40b1f00 msgr2=0x7f4be40b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- 172.21.15.43:0/3688322904 shutdown_connections 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- 172.21.15.43:0/3688322904 wait complete. 2024-03-30T13:10:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 Processor -- start 2024-03-30T13:10:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- start start 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40b56b0 0x7f4be40b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4be40cea50 0x7f4be41546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be415c620 0x7f4be40ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4169280 con 0x7f4be40b56b0 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4be4169100 con 0x7f4be415c620 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be988c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4be4169400 con 0x7f4be40cea50 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40b56b0 0x7f4be40b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40b56b0 0x7f4be40b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43958/0 (socket says 172.21.15.43:43958) 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.820+0000 7f4be2ffd640 1 -- 172.21.15.43:0/2033674222 learned_addr learned my addr 172.21.15.43:0/2033674222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be27fc640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4be40cea50 0x7f4be41546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be2ffd640 1 -- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4be40cea50 msgr2=0x7f4be41546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be2ffd640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4be40cea50 0x7f4be41546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be37fe640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be415c620 0x7f4be40ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be2ffd640 1 -- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be415c620 msgr2=0x7f4be40ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be2ffd640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be415c620 0x7f4be40ae230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be2ffd640 1 -- 172.21.15.43:0/2033674222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be40b12a0 con 0x7f4be40b56b0 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be2ffd640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40b56b0 0x7f4be40b0b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400c3a0 tx=0x7f4bd400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4bcbfff640 1 -- 172.21.15.43:0/2033674222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd40052d0 con 0x7f4be40b56b0 2024-03-30T13:10:52.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4bcbfff640 1 -- 172.21.15.43:0/2033674222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd4005470 con 0x7f4be40b56b0 2024-03-30T13:10:52.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be40b1580 con 0x7f4be40b56b0 2024-03-30T13:10:52.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4bcbfff640 1 -- 172.21.15.43:0/2033674222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4007260 con 0x7f4be40b56b0 2024-03-30T13:10:52.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be40b1b40 con 0x7f4be40b56b0 2024-03-30T13:10:52.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be4068900 con 0x7f4be40b56b0 2024-03-30T13:10:52.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4bcbfff640 1 -- 172.21.15.43:0/2033674222 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4bd4007400 con 0x7f4be40b56b0 2024-03-30T13:10:52.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4bcbfff640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4bd0067fc0 0x7f4bd006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:52.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4bcbfff640 1 -- 172.21.15.43:0/2033674222 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f4bd40855d0 con 0x7f4be40b56b0 2024-03-30T13:10:52.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be27fc640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4bd0067fc0 0x7f4bd006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:52.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.824+0000 7f4be27fc640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4bd0067fc0 0x7f4bd006a480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4be40cc870 tx=0x7f4bd800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:52.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.828+0000 7f4bcbfff640 1 -- 172.21.15.43:0/2033674222 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4bd4056160 con 0x7f4be40b56b0 2024-03-30T13:10:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.984+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4be4152880 con 0x7f4be40b56b0 2024-03-30T13:10:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.984+0000 7f4bcbfff640 1 -- 172.21.15.43:0/2033674222 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f4bd4055a60 con 0x7f4be40b56b0 2024-03-30T13:10:52.992 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:10:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4bd0067fc0 msgr2=0x7f4bd006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4bd0067fc0 0x7f4bd006a480 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4be40cc870 tx=0x7f4bd800b040 comp rx=0 tx=0).stop 2024-03-30T13:10:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40b56b0 msgr2=0x7f4be40b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:52.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40b56b0 0x7f4be40b0b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400c3a0 tx=0x7f4bd400c860 comp rx=0 tx=0).stop 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 shutdown_connections 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4bd0067fc0 0x7f4bd006a480 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4be415c620 0x7f4be40ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4be40cea50 0x7f4be41546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 --2- 172.21.15.43:0/2033674222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4be40b56b0 0x7f4be40b0b50 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400c3a0 tx=0x7f4bd400c860 comp rx=0 tx=0).stop 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 >> 172.21.15.43:0/2033674222 conn(0x7f4be40b1f00 msgr2=0x7f4be40d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 shutdown_connections 2024-03-30T13:10:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:52.988+0000 7f4be988c640 1 -- 172.21.15.43:0/2033674222 wait complete. 2024-03-30T13:10:53.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T13:10:53.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3ffde7640 1 Processor -- start 2024-03-30T13:10:53.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3ffde7640 1 -- start start 2024-03-30T13:10:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3ffde7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 0x7fe3f80cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3ffde7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f80ccef0 con 0x7fe3f806c670 2024-03-30T13:10:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3ffde7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f80b3710 con 0x7fe3f806ca50 2024-03-30T13:10:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3ffde7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f80b34c0 con 0x7fe3f80b41f0 2024-03-30T13:10:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fdb83640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe3f806c670 0x7fe3f806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53472/0 (socket says 172.21.15.43:53472) 2024-03-30T13:10:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fdb83640 1 -- 172.21.15.43:0/2397000649 learned_addr learned my addr 172.21.15.43:0/2397000649 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 949221706 0 0) 0x7fe3f80ccef0 con 0x7fe3f806c670 2024-03-30T13:10:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3e4003530 con 0x7fe3f806c670 2024-03-30T13:10:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fd382640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 0x7fe3f80cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2431571544 0 0) 0x7fe3e4003530 con 0x7fe3f806c670 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 msgr2=0x7fe3f80cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 0x7fe3f80cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 >> v1:172.21.15.113:6789/0 conn(0x7fe3f806ca50 legacy=0x7fe3f80ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f80b39f0 con 0x7fe3f806c670 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fe3f4001680 con 0x7fe3f806c670 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 >> v1:172.21.15.43:6789/0 conn(0x7fe3f806c670 legacy=0x7fe3f806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40039f0 0x7fe3e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 0x7fe3e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f80b34c0 con 0x7fe3e4004810 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e400bab0 con 0x7fe3e40039f0 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e400bcb0 con 0x7fe3f80b41f0 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fd382640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fdb83640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40039f0 0x7fe3e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fe384640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 0x7fe3e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fd382640 1 -- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 msgr2=0x7fe3e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fd382640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 0x7fe3e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fd382640 1 -- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40039f0 msgr2=0x7fe3e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fd382640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3e40039f0 0x7fe3e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fd382640 1 -- 172.21.15.43:0/2397000649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e4003530 con 0x7fe3e4004810 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.120+0000 7fe3fe384640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f80b41f0 0x7fe3e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3fd382640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3e4004c90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009ab0 tx=0x7fe3e801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a23c198fa8d1ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80b2070 con 0x7fe3e4004810 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e80a5510 con 0x7fe3e4004810 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3fcb81640 1 -- 172.21.15.43:0/2397000649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80096b0 con 0x7fe3e4004810 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 -- 172.21.15.43:0/2397000649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 msgr2=0x7fe3e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 --2- 172.21.15.43:0/2397000649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3e4004c90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009ab0 tx=0x7fe3e801ca30 comp rx=0 tx=0).stop 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3fe384640 1 -- 172.21.15.43:0/2397000649 reap_dead start 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 -- 172.21.15.43:0/2397000649 shutdown_connections 2024-03-30T13:10:53.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 -- 172.21.15.43:0/2397000649 >> 172.21.15.43:0/2397000649 conn(0x7fe3f80cfee0 msgr2=0x7fe3f80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 -- 172.21.15.43:0/2397000649 shutdown_connections 2024-03-30T13:10:53.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 -- 172.21.15.43:0/2397000649 wait complete. 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 Processor -- start 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.124+0000 7fe3ffde7640 1 -- start start 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3f813c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8139e80 0x7fe3f806b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f813a2e0 0x7fe3f806bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f806d020 con 0x7fe3e4004810 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f806cea0 con 0x7fe3f8139e80 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f8145e80 con 0x7fe3f813a2e0 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3f813c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3f813c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43980/0 (socket says 172.21.15.43:43980) 2024-03-30T13:10:53.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 -- 172.21.15.43:0/3676143588 learned_addr learned my addr 172.21.15.43:0/3676143588 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fd382640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8139e80 0x7fe3f806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 -- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f813a2e0 msgr2=0x7fe3f806bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f813a2e0 0x7fe3f806bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 -- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8139e80 msgr2=0x7fe3f806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8139e80 0x7fe3f806b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 -- 172.21.15.43:0/3676143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f81390c0 con 0x7fe3e4004810 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fdb83640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3f813c200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40117f0 tx=0x7fe3f4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ee7fc640 1 -- 172.21.15.43:0/3676143588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4010040 con 0x7fe3e4004810 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ee7fc640 1 -- 172.21.15.43:0/3676143588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3f401a070 con 0x7fe3e4004810 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f81393a0 con 0x7fe3e4004810 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ee7fc640 1 -- 172.21.15.43:0/3676143588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f400fdc0 con 0x7fe3e4004810 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f8139960 con 0x7fe3e4004810 2024-03-30T13:10:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f8068890 con 0x7fe3e4004810 2024-03-30T13:10:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ee7fc640 1 -- 172.21.15.43:0/3676143588 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe3f401f460 con 0x7fe3e4004810 2024-03-30T13:10:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ee7fc640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe3dc067fc0 0x7fe3dc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3ee7fc640 1 -- 172.21.15.43:0/3676143588 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fe3f40893b0 con 0x7fe3e4004810 2024-03-30T13:10:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fd382640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe3dc067fc0 0x7fe3dc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.128+0000 7fe3fd382640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe3dc067fc0 0x7fe3dc06a480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80af000 tx=0x7fe3e80046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.132+0000 7fe3ee7fc640 1 -- 172.21.15.43:0/3676143588 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe3f4059f40 con 0x7fe3e4004810 2024-03-30T13:10:53.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.288+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe3f80b32a0 con 0x7fe3e4004810 2024-03-30T13:10:53.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ee7fc640 1 -- 172.21.15.43:0/3676143588 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7fe3f4059840 con 0x7fe3e4004810 2024-03-30T13:10:53.297 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:10:53.297 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-03-30T13:10:53.297 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:10:53.297 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-03-30T13:10:53.297 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-03-30T13:10:53.298 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:10:53.298 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-03-30T13:10:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe3dc067fc0 msgr2=0x7fe3dc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe3dc067fc0 0x7fe3dc06a480 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80af000 tx=0x7fe3e80046d0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 msgr2=0x7fe3f813c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3f813c200 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe3f40117f0 tx=0x7fe3f4011cb0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 shutdown_connections 2024-03-30T13:10:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe3dc067fc0 0x7fe3dc06a480 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3f813a2e0 0x7fe3f806bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe3f8139e80 0x7fe3f806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 --2- 172.21.15.43:0/3676143588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3e4004810 0x7fe3f813c200 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 >> 172.21.15.43:0/3676143588 conn(0x7fe3f80cfee0 msgr2=0x7fe3f80cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 shutdown_connections 2024-03-30T13:10:53.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.292+0000 7fe3ffde7640 1 -- 172.21.15.43:0/3676143588 wait complete. 2024-03-30T13:10:53.314 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T13:10:53.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T13:10:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7e02b2640 1 Processor -- start 2024-03-30T13:10:53.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7e02b2640 1 -- start start 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7e02b2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe7d806bbb0 0x7fe7d815f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7e02b2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806ebd0 0x7fe7d806f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7e02b2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d8155820 con 0x7fe7d8155440 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7e02b2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d80b0420 con 0x7fe7d806bbb0 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7e02b2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d806f580 con 0x7fe7d806ebd0 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7dd84d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe7d806bbb0 0x7fe7d815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7dd84d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe7d806bbb0 0x7fe7d815f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43992/0 (socket says 172.21.15.43:43992) 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7dd84d640 1 -- 172.21.15.43:0/14351698 learned_addr learned my addr 172.21.15.43:0/14351698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7de84f640 1 --2- 172.21.15.43:0/14351698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806ebd0 0x7fe7d806f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7dd84d640 1 -- 172.21.15.43:0/14351698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806ebd0 msgr2=0x7fe7d806f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7dd84d640 1 --2- 172.21.15.43:0/14351698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806ebd0 0x7fe7d806f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7dd84d640 1 -- 172.21.15.43:0/14351698 >> v1:172.21.15.113:6789/0 conn(0x7fe7d8155440 legacy=0x7fe7d80afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.440+0000 7fe7dd84d640 1 -- 172.21.15.43:0/14351698 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d80b0640 con 0x7fe7d806bbb0 2024-03-30T13:10:53.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7dd84d640 1 --2- 172.21.15.43:0/14351698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d806bbb0 0x7fe7d815f520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009e80 tx=0x7fe7c001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66817d4de4c5f66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7dd04c640 1 -- 172.21.15.43:0/14351698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00a58a0 con 0x7fe7d806bbb0 2024-03-30T13:10:53.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7dd04c640 1 -- 172.21.15.43:0/14351698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7c00a5a40 con 0x7fe7d806bbb0 2024-03-30T13:10:53.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7dd04c640 1 -- 172.21.15.43:0/14351698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c00a5e10 con 0x7fe7d806bbb0 2024-03-30T13:10:53.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- 172.21.15.43:0/14351698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d806bbb0 msgr2=0x7fe7d815f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/14351698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d806bbb0 0x7fe7d815f520 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009e80 tx=0x7fe7c001c9e0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- 172.21.15.43:0/14351698 shutdown_connections 2024-03-30T13:10:53.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/14351698 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806ebd0 0x7fe7d806f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/14351698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d806bbb0 0x7fe7d815f520 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- 172.21.15.43:0/14351698 >> 172.21.15.43:0/14351698 conn(0x7fe7d80b3460 msgr2=0x7fe7d806e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- 172.21.15.43:0/14351698 shutdown_connections 2024-03-30T13:10:53.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- 172.21.15.43:0/14351698 wait complete. 2024-03-30T13:10:53.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 Processor -- start 2024-03-30T13:10:53.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- start start 2024-03-30T13:10:53.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806bbb0 0x7fe7d813e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d806ebd0 0x7fe7d81513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8155440 0x7fe7d8165500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8162730 con 0x7fe7d8155440 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d815f000 con 0x7fe7d806ebd0 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7e02b2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d81628b0 con 0x7fe7d806bbb0 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7de84f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8155440 0x7fe7d8165500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7de84f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8155440 0x7fe7d8165500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44000/0 (socket says 172.21.15.43:44000) 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.444+0000 7fe7de84f640 1 -- 172.21.15.43:0/1317071498 learned_addr learned my addr 172.21.15.43:0/1317071498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7dd84d640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d806ebd0 0x7fe7d81513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de04e640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806bbb0 0x7fe7d813e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de84f640 1 -- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806bbb0 msgr2=0x7fe7d813e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de84f640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806bbb0 0x7fe7d813e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de84f640 1 -- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d806ebd0 msgr2=0x7fe7d81513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de84f640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d806ebd0 0x7fe7d81513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de84f640 1 -- 172.21.15.43:0/1317071498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d8168360 con 0x7fe7d8155440 2024-03-30T13:10:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7dd84d640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d806ebd0 0x7fe7d81513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de84f640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8155440 0x7fe7d8165500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d830 tx=0x7fe7d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/1317071498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000b6c0 con 0x7fe7d8155440 2024-03-30T13:10:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/1317071498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d0019070 con 0x7fe7d8155440 2024-03-30T13:10:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d8168640 con 0x7fe7d8155440 2024-03-30T13:10:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/1317071498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000baa0 con 0x7fe7d8155440 2024-03-30T13:10:53.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d8168c00 con 0x7fe7d8155440 2024-03-30T13:10:53.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d80687f0 con 0x7fe7d8155440 2024-03-30T13:10:53.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/1317071498 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe7d001e460 con 0x7fe7d8155440 2024-03-30T13:10:53.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7d6ffd640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7b8067fc0 0x7fe7b806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:53.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/1317071498 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fe7d0086870 con 0x7fe7d8155440 2024-03-30T13:10:53.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de04e640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7b8067fc0 0x7fe7b806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:53.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.448+0000 7fe7de04e640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7b8067fc0 0x7fe7b806a480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000f70 tx=0x7fe7c8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:53.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.452+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/1317071498 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe7d0057400 con 0x7fe7d8155440 2024-03-30T13:10:53.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.620+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe7d80b46a0 con 0x7fe7d8155440 2024-03-30T13:10:53.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.896+0000 7fe7d6ffd640 1 -- 172.21.15.43:0/1317071498 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7fe7d0056d00 con 0x7fe7d8155440 2024-03-30T13:10:53.902 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-03-30T13:10:53.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7b8067fc0 msgr2=0x7fe7b806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7b8067fc0 0x7fe7b806a480 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8000f70 tx=0x7fe7c8009040 comp rx=0 tx=0).stop 2024-03-30T13:10:53.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8155440 msgr2=0x7fe7d8165500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:53.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8155440 0x7fe7d8165500 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d830 tx=0x7fe7d000dcf0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 shutdown_connections 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7b8067fc0 0x7fe7b806a480 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8155440 0x7fe7d8165500 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d806ebd0 0x7fe7d81513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 --2- 172.21.15.43:0/1317071498 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d806bbb0 0x7fe7d813e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 >> 172.21.15.43:0/1317071498 conn(0x7fe7d80b3460 msgr2=0x7fe7d80b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 shutdown_connections 2024-03-30T13:10:53.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:53.900+0000 7fe7e02b2640 1 -- 172.21.15.43:0/1317071498 wait complete. 2024-03-30T13:10:53.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-03-30T13:10:54.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.060+0000 7f2001611640 1 Processor -- start 2024-03-30T13:10:54.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.060+0000 7f2001611640 1 -- start start 2024-03-30T13:10:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f2001611640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ffc070d30 0x7f1ffc0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f2001611640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ffc0703c0 con 0x7f1ffc0cd9c0 2024-03-30T13:10:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f2001611640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ffc0cfac0 con 0x7f1ffc06f9e0 2024-03-30T13:10:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f2001611640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1ffc0d03f0 con 0x7f1ffc070d30 2024-03-30T13:10:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f1ffa7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ffc070d30 0x7f1ffc0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f1ffa7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ffc070d30 0x7f1ffc0d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44004/0 (socket says 172.21.15.43:44004) 2024-03-30T13:10:54.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/4292276807 learned_addr learned my addr 172.21.15.43:0/4292276807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:54.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/4292276807 >> v1:172.21.15.113:6789/0 conn(0x7f1ffc06f9e0 legacy=0x7f1ffc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/4292276807 >> v1:172.21.15.43:6789/0 conn(0x7f1ffc0cd9c0 legacy=0x7f1ffc0cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.064+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/4292276807 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc0cfda0 con 0x7f1ffc070d30 2024-03-30T13:10:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f1ffa7fc640 1 --2- 172.21.15.43:0/4292276807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0d0880 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009870 tx=0x7f1ff001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e64a57109c2b134 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f1ff9ffb640 1 -- 172.21.15.43:0/4292276807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00a9610 con 0x7f1ffc070d30 2024-03-30T13:10:54.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f1ff9ffb640 1 -- 172.21.15.43:0/4292276807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff00a97b0 con 0x7f1ffc070d30 2024-03-30T13:10:54.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f1ff9ffb640 1 -- 172.21.15.43:0/4292276807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00a9b80 con 0x7f1ffc070d30 2024-03-30T13:10:54.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f2001611640 1 -- 172.21.15.43:0/4292276807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 msgr2=0x7f1ffc0d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f2001611640 1 --2- 172.21.15.43:0/4292276807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0d0880 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009870 tx=0x7f1ff001c530 comp rx=0 tx=0).stop 2024-03-30T13:10:54.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f2001611640 1 -- 172.21.15.43:0/4292276807 shutdown_connections 2024-03-30T13:10:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f2001611640 1 --2- 172.21.15.43:0/4292276807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0d0880 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f2001611640 1 -- 172.21.15.43:0/4292276807 >> 172.21.15.43:0/4292276807 conn(0x7f1ffc06e4c0 msgr2=0x7f1ffc06e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f2001611640 1 -- 172.21.15.43:0/4292276807 shutdown_connections 2024-03-30T13:10:54.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.068+0000 7f2001611640 1 -- 172.21.15.43:0/4292276807 wait complete. 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 Processor -- start 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 -- start start 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ffc06f9e0 0x7f1ffc15cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ffc0cd9c0 0x7f1ffc15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc16a1f0 con 0x7f1ffc070d30 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc16a070 con 0x7f1ffc0cd9c0 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1ffc16a370 con 0x7f1ffc06f9e0 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44018/0 (socket says 172.21.15.43:44018) 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/3465648416 learned_addr learned my addr 172.21.15.43:0/3465648416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffaffd640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ffc06f9e0 0x7f1ffc15cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffb7fe640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ffc0cd9c0 0x7f1ffc15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ffc06f9e0 msgr2=0x7f1ffc15cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ffc06f9e0 0x7f1ffc15cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ffc0cd9c0 msgr2=0x7f1ffc15b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ffc0cd9c0 0x7f1ffc15b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 -- 172.21.15.43:0/3465648416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc1540f0 con 0x7f1ffc070d30 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1ffa7fc640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0b5a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00ab040 tx=0x7f1ff001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1fe37fe640 1 -- 172.21.15.43:0/3465648416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00b1670 con 0x7f1ffc070d30 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1fe37fe640 1 -- 172.21.15.43:0/3465648416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff00b1810 con 0x7f1ffc070d30 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc15d500 con 0x7f1ffc070d30 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f1fe37fe640 1 -- 172.21.15.43:0/3465648416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00b1be0 con 0x7f1ffc070d30 2024-03-30T13:10:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc0b5f60 con 0x7f1ffc070d30 2024-03-30T13:10:54.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.072+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ffc068900 con 0x7f1ffc070d30 2024-03-30T13:10:54.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.076+0000 7f1fe37fe640 1 -- 172.21.15.43:0/3465648416 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1ff00c2060 con 0x7f1ffc070d30 2024-03-30T13:10:54.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.076+0000 7f1fe37fe640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1fe8067ff0 0x7f1fe806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:54.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.076+0000 7f1fe37fe640 1 -- 172.21.15.43:0/3465648416 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f1ff0120b80 con 0x7f1ffc070d30 2024-03-30T13:10:54.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.076+0000 7f1ffaffd640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1fe8067ff0 0x7f1fe806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.076+0000 7f1ffaffd640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1fe8067ff0 0x7f1fe806a4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001e00 tx=0x7f1fec009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:54.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.080+0000 7f1fe37fe640 1 -- 172.21.15.43:0/3465648416 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1ff00b6070 con 0x7f1ffc070d30 2024-03-30T13:10:54.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.236+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f1ffc13a0a0 con 0x7f1ffc070d30 2024-03-30T13:10:54.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.900+0000 7f1fe37fe640 1 -- 172.21.15.43:0/3465648416 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) ==== 171+0+0 (secure 0 0 0) 0x7f1ff00ad020 con 0x7f1ffc070d30 2024-03-30T13:10:54.908 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1fe8067ff0 msgr2=0x7f1fe806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1fe8067ff0 0x7f1fe806a4b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1fec001e00 tx=0x7f1fec009040 comp rx=0 tx=0).stop 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 msgr2=0x7f1ffc0b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0b5a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00ab040 tx=0x7f1ff001ad70 comp rx=0 tx=0).stop 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 shutdown_connections 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1fe8067ff0 0x7f1fe806a4b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1ffc0cd9c0 0x7f1ffc15b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ffc070d30 0x7f1ffc0b5a20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 --2- 172.21.15.43:0/3465648416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ffc06f9e0 0x7f1ffc15cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 >> 172.21.15.43:0/3465648416 conn(0x7f1ffc06e4c0 msgr2=0x7f1ffc072520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 shutdown_connections 2024-03-30T13:10:54.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:54.904+0000 7f2001611640 1 -- 172.21.15.43:0/3465648416 wait complete. 2024-03-30T13:10:54.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T13:10:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06759e640 1 Processor -- start 2024-03-30T13:10:55.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06759e640 1 -- start start 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff0680acf60 0x7ff0680ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06759e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0680ad880 con 0x7ff068162870 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06759e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff06806f220 con 0x7ff06806cc10 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06759e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff06806f500 con 0x7ff0680acf60 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff0680acf60 0x7ff0680ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff066d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff068162870 0x7ff0680cefa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53494/0 (socket says 172.21.15.43:53494) 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff066d9d640 1 -- 172.21.15.43:0/2082751169 learned_addr learned my addr 172.21.15.43:0/2082751169 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff0680acf60 0x7ff0680ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44028/0 (socket says 172.21.15.43:44028) 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06659c640 1 -- 172.21.15.43:0/2082751169 >> v1:172.21.15.113:6789/0 conn(0x7ff06806cc10 legacy=0x7ff06806e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06659c640 1 -- 172.21.15.43:0/2082751169 >> v1:172.21.15.43:6789/0 conn(0x7ff068162870 legacy=0x7ff0680cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06559a640 1 -- 172.21.15.43:0/2082751169 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578255556 0 0) 0x7ff0680ad880 con 0x7ff068162870 2024-03-30T13:10:55.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.068+0000 7ff06659c640 1 -- 172.21.15.43:0/2082751169 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff068161d90 con 0x7ff0680acf60 2024-03-30T13:10:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06659c640 1 --2- 172.21.15.43:0/2082751169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0680acf60 0x7ff0680ad340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0097c0 tx=0x7ff05c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c921992455323df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06559a640 1 -- 172.21.15.43:0/2082751169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0b0070 con 0x7ff0680acf60 2024-03-30T13:10:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06559a640 1 -- 172.21.15.43:0/2082751169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff05c0ab600 con 0x7ff0680acf60 2024-03-30T13:10:55.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06559a640 1 -- 172.21.15.43:0/2082751169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0ab9d0 con 0x7ff0680acf60 2024-03-30T13:10:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- 172.21.15.43:0/2082751169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0680acf60 msgr2=0x7ff0680ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 --2- 172.21.15.43:0/2082751169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0680acf60 0x7ff0680ad340 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0097c0 tx=0x7ff05c01c380 comp rx=0 tx=0).stop 2024-03-30T13:10:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- 172.21.15.43:0/2082751169 shutdown_connections 2024-03-30T13:10:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 --2- 172.21.15.43:0/2082751169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0680acf60 0x7ff0680ad340 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- 172.21.15.43:0/2082751169 >> 172.21.15.43:0/2082751169 conn(0x7ff0680d0150 msgr2=0x7ff0680d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- 172.21.15.43:0/2082751169 shutdown_connections 2024-03-30T13:10:55.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- 172.21.15.43:0/2082751169 wait complete. 2024-03-30T13:10:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 Processor -- start 2024-03-30T13:10:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- start start 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff06806cc10 0x7ff06809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0680acf60 0x7ff068161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068162870 0x7ff068072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff06816c200 con 0x7ff068162870 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff06816c080 con 0x7ff06806cc10 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.072+0000 7ff06759e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff06816c380 con 0x7ff0680acf60 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068162870 0x7ff068072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068162870 0x7ff068072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44030/0 (socket says 172.21.15.43:44030) 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 -- 172.21.15.43:0/1145870257 learned_addr learned my addr 172.21.15.43:0/1145870257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff06659c640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff06806cc10 0x7ff06809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff065d9b640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0680acf60 0x7ff068161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 -- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0680acf60 msgr2=0x7ff068161130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0680acf60 0x7ff068161130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 -- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff06806cc10 msgr2=0x7ff06809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff06806cc10 0x7ff06809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 -- 172.21.15.43:0/1145870257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff068154a90 con 0x7ff068162870 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff066d9d640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068162870 0x7ff068072920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c730 tx=0x7ff06000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff056ffd640 1 -- 172.21.15.43:0/1145870257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060018070 con 0x7ff068162870 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff056ffd640 1 -- 172.21.15.43:0/1145870257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06000ce40 con 0x7ff068162870 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff068154d70 con 0x7ff068162870 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff056ffd640 1 -- 172.21.15.43:0/1145870257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06000a880 con 0x7ff068162870 2024-03-30T13:10:55.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff068072120 con 0x7ff068162870 2024-03-30T13:10:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff068068670 con 0x7ff068162870 2024-03-30T13:10:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff056ffd640 1 -- 172.21.15.43:0/1145870257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff06001b070 con 0x7ff068162870 2024-03-30T13:10:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff056ffd640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff048067fc0 0x7ff04806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff056ffd640 1 -- 172.21.15.43:0/1145870257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7ff060085c60 con 0x7ff068162870 2024-03-30T13:10:55.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.076+0000 7ff06659c640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff048067fc0 0x7ff04806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:55.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.080+0000 7ff06659c640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff048067fc0 0x7ff04806a480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0ad000 tx=0x7ff05c0a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:55.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.080+0000 7ff056ffd640 1 -- 172.21.15.43:0/1145870257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff0600565e0 con 0x7ff068162870 2024-03-30T13:10:55.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.280+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff068169e50 con 0x7ff068162870 2024-03-30T13:10:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.944+0000 7ff056ffd640 1 -- 172.21.15.43:0/1145870257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ff060055ee0 con 0x7ff068162870 2024-03-30T13:10:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-03-30T13:10:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff048067fc0 msgr2=0x7ff04806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff048067fc0 0x7ff04806a480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff05c0ad000 tx=0x7ff05c0a6000 comp rx=0 tx=0).stop 2024-03-30T13:10:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068162870 msgr2=0x7ff068072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068162870 0x7ff068072920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff06000c730 tx=0x7ff06000cbf0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 shutdown_connections 2024-03-30T13:10:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff048067fc0 0x7ff04806a480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff068162870 0x7ff068072920 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff0680acf60 0x7ff068161130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 --2- 172.21.15.43:0/1145870257 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff06806cc10 0x7ff06809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 >> 172.21.15.43:0/1145870257 conn(0x7ff0680d0150 msgr2=0x7ff06806dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 shutdown_connections 2024-03-30T13:10:55.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:55.964+0000 7ff06759e640 1 -- 172.21.15.43:0/1145870257 wait complete. 2024-03-30T13:10:56.000 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-03-30T13:10:56.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-03-30T13:10:56.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.116+0000 7f5b86921640 1 Processor -- start 2024-03-30T13:10:56.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.116+0000 7f5b86921640 1 -- start start 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b86921640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5b80161af0 0x7f5b800d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b86921640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800b7cd0 0x7f5b8006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b86921640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b80074030 con 0x7f5b800d24e0 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b86921640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5b800d2c50 con 0x7f5b80161af0 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b86921640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5b800d2f30 con 0x7f5b800b7cd0 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5b80161af0 0x7f5b800d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5b80161af0 0x7f5b800d3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44046/0 (socket says 172.21.15.43:44046) 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 -- 172.21.15.43:0/1773068428 learned_addr learned my addr 172.21.15.43:0/1773068428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:56.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b84ebe640 1 --2- 172.21.15.43:0/1773068428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800b7cd0 0x7f5b8006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 -- 172.21.15.43:0/1773068428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800b7cd0 msgr2=0x7f5b8006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:56.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 --2- 172.21.15.43:0/1773068428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800b7cd0 0x7f5b8006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 -- 172.21.15.43:0/1773068428 >> v1:172.21.15.113:6789/0 conn(0x7f5b800d24e0 legacy=0x7f5b800b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:56.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 -- 172.21.15.43:0/1773068428 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b800d3210 con 0x7f5b80161af0 2024-03-30T13:10:56.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7ffff640 1 --2- 172.21.15.43:0/1773068428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b80161af0 0x7f5b800d3cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009e50 tx=0x7f5b7001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6ef95635abc7c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:56.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7effd640 1 -- 172.21.15.43:0/1773068428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b4070 con 0x7f5b80161af0 2024-03-30T13:10:56.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7effd640 1 -- 172.21.15.43:0/1773068428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b700af400 con 0x7f5b80161af0 2024-03-30T13:10:56.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.120+0000 7f5b7effd640 1 -- 172.21.15.43:0/1773068428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700af7d0 con 0x7f5b80161af0 2024-03-30T13:10:56.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- 172.21.15.43:0/1773068428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b80161af0 msgr2=0x7f5b800d3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 --2- 172.21.15.43:0/1773068428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b80161af0 0x7f5b800d3cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009e50 tx=0x7f5b7001f1e0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- 172.21.15.43:0/1773068428 shutdown_connections 2024-03-30T13:10:56.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 --2- 172.21.15.43:0/1773068428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800b7cd0 0x7f5b8006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 --2- 172.21.15.43:0/1773068428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b80161af0 0x7f5b800d3cf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- 172.21.15.43:0/1773068428 >> 172.21.15.43:0/1773068428 conn(0x7f5b800d42e0 msgr2=0x7f5b800d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:56.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- 172.21.15.43:0/1773068428 shutdown_connections 2024-03-30T13:10:56.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- 172.21.15.43:0/1773068428 wait complete. 2024-03-30T13:10:56.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 Processor -- start 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- start start 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b800b7cd0 0x7f5b801497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800d24e0 0x7f5b800cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b80161af0 0x7f5b801586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b800d2940 con 0x7f5b800b7cd0 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5b800d37d0 con 0x7f5b80161af0 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5b80161ed0 con 0x7f5b800d24e0 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b800b7cd0 0x7f5b801497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b800b7cd0 0x7f5b801497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44048/0 (socket says 172.21.15.43:44048) 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 -- 172.21.15.43:0/3271785451 learned_addr learned my addr 172.21.15.43:0/3271785451 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 -- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800d24e0 msgr2=0x7f5b800cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800d24e0 0x7f5b800cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 -- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b80161af0 msgr2=0x7f5b801586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b80161af0 0x7f5b801586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 -- 172.21.15.43:0/3271785451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b800cdd50 con 0x7f5b800b7cd0 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7ffff640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b800b7cd0 0x7f5b801497c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b2040 tx=0x7f5b7001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7cff9640 1 -- 172.21.15.43:0/3271785451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b4040 con 0x7f5b800b7cd0 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7cff9640 1 -- 172.21.15.43:0/3271785451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b700c0070 con 0x7f5b800b7cd0 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b7cff9640 1 -- 172.21.15.43:0/3271785451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70009510 con 0x7f5b800b7cd0 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b800c8640 con 0x7f5b800b7cd0 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.124+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b800c88e0 con 0x7f5b800b7cd0 2024-03-30T13:10:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.128+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b800687f0 con 0x7f5b800b7cd0 2024-03-30T13:10:56.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.128+0000 7f5b7cff9640 1 -- 172.21.15.43:0/3271785451 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5b700ba660 con 0x7f5b800b7cd0 2024-03-30T13:10:56.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.128+0000 7f5b7cff9640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b6c067fc0 0x7f5b6c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:56.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.128+0000 7f5b7cff9640 1 -- 172.21.15.43:0/3271785451 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5b701248a0 con 0x7f5b800b7cd0 2024-03-30T13:10:56.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.128+0000 7f5b7f7fe640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b6c067fc0 0x7f5b6c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:56.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.128+0000 7f5b7f7fe640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b6c067fc0 0x7f5b6c06a480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b800ce0f0 tx=0x7f5b740079c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:56.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.132+0000 7f5b7cff9640 1 -- 172.21.15.43:0/3271785451 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5b70018d40 con 0x7f5b800b7cd0 2024-03-30T13:10:56.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:56.210 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:10:56.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.304+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f5b800b9100 con 0x7f5b800b7cd0 2024-03-30T13:10:56.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.952+0000 7f5b7cff9640 1 -- 172.21.15.43:0/3271785451 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) ==== 157+0+0 (secure 0 0 0) 0x7f5b700f5110 con 0x7f5b800b7cd0 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b6c067fc0 msgr2=0x7f5b6c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b6c067fc0 0x7f5b6c06a480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5b800ce0f0 tx=0x7f5b740079c0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b800b7cd0 msgr2=0x7f5b801497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b800b7cd0 0x7f5b801497c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b2040 tx=0x7f5b7001bf80 comp rx=0 tx=0).stop 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 shutdown_connections 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b6c067fc0 0x7f5b6c06a480 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b80161af0 0x7f5b801586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b800d24e0 0x7f5b800cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 --2- 172.21.15.43:0/3271785451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b800b7cd0 0x7f5b801497c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 >> 172.21.15.43:0/3271785451 conn(0x7f5b800d42e0 msgr2=0x7f5b800b9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 shutdown_connections 2024-03-30T13:10:56.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:56.956+0000 7f5b86921640 1 -- 172.21.15.43:0/3271785451 wait complete. 2024-03-30T13:10:56.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T13:10:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f1665ba3640 1 Processor -- start 2024-03-30T13:10:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f1665ba3640 1 -- start start 2024-03-30T13:10:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f1665ba3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f16600700f0 0x7f16600b5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f1665ba3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f16600b58e0 0x7f16600b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f1665ba3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f166006b800 con 0x7f166015e3c0 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f1665ba3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f166006c560 con 0x7f16600b58e0 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f1665ba3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f166006bae0 con 0x7f16600700f0 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f16600b58e0 0x7f16600b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f16600b58e0 0x7f16600b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44064/0 (socket says 172.21.15.43:44064) 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165ffff640 1 -- 172.21.15.43:0/3723563384 learned_addr learned my addr 172.21.15.43:0/3723563384 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165effd640 1 --2- 172.21.15.43:0/3723563384 >> v2:172.21.15.113:3300/0 conn(0x7f16600700f0 0x7f16600b5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165ffff640 1 -- 172.21.15.43:0/3723563384 >> v2:172.21.15.113:3300/0 conn(0x7f16600700f0 msgr2=0x7f16600b5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165ffff640 1 --2- 172.21.15.43:0/3723563384 >> v2:172.21.15.113:3300/0 conn(0x7f16600700f0 0x7f16600b5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165ffff640 1 -- 172.21.15.43:0/3723563384 >> v1:172.21.15.113:6789/0 conn(0x7f166015e3c0 legacy=0x7f166006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:57.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.108+0000 7f165ffff640 1 -- 172.21.15.43:0/3723563384 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f166006c1f0 con 0x7f16600b58e0 2024-03-30T13:10:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f165ffff640 1 --2- 172.21.15.43:0/3723563384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600b58e0 0x7f16600b3f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f16600cd270 tx=0x7f165801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8168c1940395c7c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f165e7fc640 1 -- 172.21.15.43:0/3723563384 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580a56f0 con 0x7f16600b58e0 2024-03-30T13:10:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f165e7fc640 1 -- 172.21.15.43:0/3723563384 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16580a5890 con 0x7f16600b58e0 2024-03-30T13:10:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f165e7fc640 1 -- 172.21.15.43:0/3723563384 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16580a5c60 con 0x7f16600b58e0 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 -- 172.21.15.43:0/3723563384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600b58e0 msgr2=0x7f16600b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 --2- 172.21.15.43:0/3723563384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600b58e0 0x7f16600b3f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f16600cd270 tx=0x7f165801c920 comp rx=0 tx=0).stop 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 -- 172.21.15.43:0/3723563384 shutdown_connections 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 --2- 172.21.15.43:0/3723563384 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600b58e0 0x7f16600b3f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 --2- 172.21.15.43:0/3723563384 >> v2:172.21.15.113:3300/0 conn(0x7f16600700f0 0x7f16600b5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 -- 172.21.15.43:0/3723563384 >> 172.21.15.43:0/3723563384 conn(0x7f166006e540 msgr2=0x7f166006e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 -- 172.21.15.43:0/3723563384 shutdown_connections 2024-03-30T13:10:57.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 -- 172.21.15.43:0/3723563384 wait complete. 2024-03-30T13:10:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 Processor -- start 2024-03-30T13:10:57.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.112+0000 7f1665ba3640 1 -- start start 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600700f0 0x7f166015f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16600b58e0 0x7f1660147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166015e3c0 0x7f16600b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f166016b450 con 0x7f16600700f0 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f166016b2d0 con 0x7f166015e3c0 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f166016b5d0 con 0x7f16600b58e0 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600700f0 0x7f166015f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600700f0 0x7f166015f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44066/0 (socket says 172.21.15.43:44066) 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 -- 172.21.15.43:0/888380415 learned_addr learned my addr 172.21.15.43:0/888380415 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165effd640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16600b58e0 0x7f1660147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165ffff640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166015e3c0 0x7f16600b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 -- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16600b58e0 msgr2=0x7f1660147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16600b58e0 0x7f1660147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 -- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166015e3c0 msgr2=0x7f16600b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166015e3c0 0x7f16600b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 -- 172.21.15.43:0/888380415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1660157000 con 0x7f16600700f0 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165ffff640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166015e3c0 0x7f16600b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165f7fe640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600700f0 0x7f166015f3c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f165000b440 tx=0x7f165000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1647fff640 1 -- 172.21.15.43:0/888380415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650015020 con 0x7f16600700f0 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1647fff640 1 -- 172.21.15.43:0/888380415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1650011890 con 0x7f16600700f0 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f166015fa50 con 0x7f16600700f0 2024-03-30T13:10:57.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1647fff640 1 -- 172.21.15.43:0/888380415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650011c80 con 0x7f16600700f0 2024-03-30T13:10:57.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1660160010 con 0x7f16600700f0 2024-03-30T13:10:57.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1660068e60 con 0x7f16600700f0 2024-03-30T13:10:57.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1647fff640 1 -- 172.21.15.43:0/888380415 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1650018020 con 0x7f16600700f0 2024-03-30T13:10:57.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1647fff640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f164c067ff0 0x7f164c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:57.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f1647fff640 1 -- 172.21.15.43:0/888380415 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6007+0+0 (secure 0 0 0) 0x7f16500867d0 con 0x7f16600700f0 2024-03-30T13:10:57.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.116+0000 7f165effd640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f164c067ff0 0x7f164c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:57.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.120+0000 7f165effd640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f164c067ff0 0x7f164c06a4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1660168b40 tx=0x7f1654008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:57.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.120+0000 7f1647fff640 1 -- 172.21.15.43:0/888380415 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f165001f030 con 0x7f16600700f0 2024-03-30T13:10:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:57.288+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1660161a60 con 0x7f16600700f0 2024-03-30T13:10:58.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.960+0000 7f1647fff640 1 -- 172.21.15.43:0/888380415 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) ==== 228+0+0 (secure 0 0 0) 0x7f165000d040 con 0x7f16600700f0 2024-03-30T13:10:58.966 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' created 2024-03-30T13:10:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f164c067ff0 msgr2=0x7f164c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f164c067ff0 0x7f164c06a4b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1660168b40 tx=0x7f1654008040 comp rx=0 tx=0).stop 2024-03-30T13:10:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600700f0 msgr2=0x7f166015f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600700f0 0x7f166015f3c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f165000b440 tx=0x7f165000b900 comp rx=0 tx=0).stop 2024-03-30T13:10:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 shutdown_connections 2024-03-30T13:10:58.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f164c067ff0 0x7f164c06a4b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f166015e3c0 0x7f16600b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16600b58e0 0x7f1660147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 --2- 172.21.15.43:0/888380415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16600700f0 0x7f166015f3c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:58.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 >> 172.21.15.43:0/888380415 conn(0x7f166006e540 msgr2=0x7f16600adc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:58.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 shutdown_connections 2024-03-30T13:10:58.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:58.964+0000 7f1665ba3640 1 -- 172.21.15.43:0/888380415 wait complete. 2024-03-30T13:10:59.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-03-30T13:10:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea2f199640 1 Processor -- start 2024-03-30T13:10:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea2f199640 1 -- start start 2024-03-30T13:10:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea2f199640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fea2815e660 0x7fea280af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea2f199640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea2815ea40 con 0x7fea2815ec80 2024-03-30T13:10:59.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea2f199640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea2815c520 con 0x7fea2815d860 2024-03-30T13:10:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea2f199640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fea280b4230 con 0x7fea2815e660 2024-03-30T13:10:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea27fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fea2815ec80 0x7fea2815b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53508/0 (socket says 172.21.15.43:53508) 2024-03-30T13:10:59.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea27fff640 1 -- 172.21.15.43:0/4150325140 learned_addr learned my addr 172.21.15.43:0/4150325140 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:10:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749595988 0 0) 0x7fea2815ea40 con 0x7fea2815ec80 2024-03-30T13:10:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea2cf35640 1 --2- 172.21.15.43:0/4150325140 >> v2:172.21.15.113:3300/0 conn(0x7fea2815e660 0x7fea280af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea10003530 con 0x7fea2815ec80 2024-03-30T13:10:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2965886439 0 0) 0x7fea10003530 con 0x7fea2815ec80 2024-03-30T13:10:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 >> v2:172.21.15.113:3300/0 conn(0x7fea2815e660 msgr2=0x7fea280af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 --2- 172.21.15.43:0/4150325140 >> v2:172.21.15.113:3300/0 conn(0x7fea2815e660 0x7fea280af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 >> v1:172.21.15.113:6789/0 conn(0x7fea2815d860 legacy=0x7fea2815be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea280b4510 con 0x7fea2815ec80 2024-03-30T13:10:59.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fea1c001680 con 0x7fea2815ec80 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 >> v1:172.21.15.43:6789/0 conn(0x7fea2815ec80 legacy=0x7fea2815b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.116+0000 7fea277fe640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100039c0 0x7fea10003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004810 0x7fea10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea100051d0 0x7fea1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea280b4230 con 0x7fea100051d0 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea1000c120 con 0x7fea10004810 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fea1000c320 con 0x7fea100039c0 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea100051d0 0x7fea1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea27fff640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004810 0x7fea10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2cf35640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100039c0 0x7fea10003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 -- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100039c0 msgr2=0x7fea10003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100039c0 0x7fea10003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 -- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004810 msgr2=0x7fea10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:10:59.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea10004810 0x7fea10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 -- 172.21.15.43:0/4150325140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea10003530 con 0x7fea100051d0 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 -- 172.21.15.43:0/4150325140 reap_dead start 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2d736640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea100051d0 0x7fea1000b9d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea20001780 tx=0x7fea2001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e6862ac34e41a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200013a0 con 0x7fea100051d0 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea200a9070 con 0x7fea100051d0 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea277fe640 1 -- 172.21.15.43:0/4150325140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20003600 con 0x7fea100051d0 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2f199640 1 -- 172.21.15.43:0/4150325140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea100051d0 msgr2=0x7fea1000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2f199640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea100051d0 0x7fea1000b9d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fea20001780 tx=0x7fea2001c420 comp rx=0 tx=0).stop 2024-03-30T13:10:59.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2f199640 1 -- 172.21.15.43:0/4150325140 shutdown_connections 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2f199640 1 --2- 172.21.15.43:0/4150325140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea100051d0 0x7fea1000b9d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2f199640 1 -- 172.21.15.43:0/4150325140 >> 172.21.15.43:0/4150325140 conn(0x7fea280d06e0 msgr2=0x7fea280d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2f199640 1 -- 172.21.15.43:0/4150325140 shutdown_connections 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.120+0000 7fea2f199640 1 -- 172.21.15.43:0/4150325140 wait complete. 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 Processor -- start 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 -- start start 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100051d0 0x7fea281585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea281599a0 0x7fea2814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea280cfa20 0x7fea280cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea280b4ad0 con 0x7fea280cfa20 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea280b47f0 con 0x7fea281599a0 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fea280b4db0 con 0x7fea100051d0 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea280cfa20 0x7fea280cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea280cfa20 0x7fea280cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44082/0 (socket says 172.21.15.43:44082) 2024-03-30T13:10:59.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 -- 172.21.15.43:0/2914200650 learned_addr learned my addr 172.21.15.43:0/2914200650 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea27fff640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea281599a0 0x7fea2814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2cf35640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100051d0 0x7fea281585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 -- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100051d0 msgr2=0x7fea281585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100051d0 0x7fea281585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 -- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea281599a0 msgr2=0x7fea2814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea281599a0 0x7fea2814af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 -- 172.21.15.43:0/2914200650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea28159f60 con 0x7fea280cfa20 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea27fff640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea281599a0 0x7fea2814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2d736640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea280cfa20 0x7fea280cfea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea20097e30 tx=0x7fea10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea257fa640 1 -- 172.21.15.43:0/2914200650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea200013a0 con 0x7fea280cfa20 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea257fa640 1 -- 172.21.15.43:0/2914200650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea200050d0 con 0x7fea280cfa20 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2815a240 con 0x7fea280cfa20 2024-03-30T13:10:59.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea257fa640 1 -- 172.21.15.43:0/2914200650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea20007a40 con 0x7fea280cfa20 2024-03-30T13:10:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2815a800 con 0x7fea280cfa20 2024-03-30T13:10:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.124+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea280d03e0 con 0x7fea280cfa20 2024-03-30T13:10:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.128+0000 7fea257fa640 1 -- 172.21.15.43:0/2914200650 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fea200ab070 con 0x7fea280cfa20 2024-03-30T13:10:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.128+0000 7fea257fa640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea14067fc0 0x7fea1406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:10:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.128+0000 7fea257fa640 1 -- 172.21.15.43:0/2914200650 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7fea20120040 con 0x7fea280cfa20 2024-03-30T13:10:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.128+0000 7fea2cf35640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea14067fc0 0x7fea1406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:10:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.128+0000 7fea2cf35640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea14067fc0 0x7fea1406a480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea180042b0 tx=0x7fea1800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:10:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.132+0000 7fea257fa640 1 -- 172.21.15.43:0/2914200650 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fea20016130 con 0x7fea280cfa20 2024-03-30T13:10:59.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.320+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fea2815bcd0 con 0x7fea280cfa20 2024-03-30T13:10:59.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.968+0000 7fea257fa640 1 -- 172.21.15.43:0/2914200650 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) ==== 169+0+0 (secure 0 0 0) 0x7fea200f0430 con 0x7fea280cfa20 2024-03-30T13:10:59.974 INFO:teuthology.orchestra.run.smithi043.stderr:set pool 4 allow_ec_overwrites to true 2024-03-30T13:10:59.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea14067fc0 msgr2=0x7fea1406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea14067fc0 0x7fea1406a480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea180042b0 tx=0x7fea1800b040 comp rx=0 tx=0).stop 2024-03-30T13:10:59.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea280cfa20 msgr2=0x7fea280cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:10:59.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea280cfa20 0x7fea280cfea0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fea20097e30 tx=0x7fea10005840 comp rx=0 tx=0).stop 2024-03-30T13:10:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 shutdown_connections 2024-03-30T13:10:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea14067fc0 0x7fea1406a480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea280cfa20 0x7fea280cfea0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea281599a0 0x7fea2814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 --2- 172.21.15.43:0/2914200650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea100051d0 0x7fea281585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:10:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.976+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 >> 172.21.15.43:0/2914200650 conn(0x7fea280d06e0 msgr2=0x7fea2815c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:10:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.980+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 shutdown_connections 2024-03-30T13:10:59.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:10:59.980+0000 7fea2f199640 1 -- 172.21.15.43:0/2914200650 wait complete. 2024-03-30T13:11:00.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-03-30T13:11:00.139 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-03-30T13:11:00.132+0000 7f4242b42640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T13:11:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.136+0000 7f5c74b1f640 1 Processor -- start 2024-03-30T13:11:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.136+0000 7f5c74b1f640 1 -- start start 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c74b1f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5c7015f5b0 0x7f5c700ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c74b1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c7006cc30 0x7f5c700ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c74b1f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c700ccfd0 con 0x7f5c7006c850 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c74b1f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5c700b4970 con 0x7f5c7006cc30 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c74b1f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5c700b4c50 con 0x7f5c7015f5b0 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c7006cc30 0x7f5c700ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c7006cc30 0x7f5c700ae2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44090/0 (socket says 172.21.15.43:44090) 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 -- 172.21.15.43:0/164227951 learned_addr learned my addr 172.21.15.43:0/164227951 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6e59c640 1 --1- 172.21.15.43:0/164227951 >> v1:172.21.15.43:6789/0 conn(0x7f5c7006c850 0x7f5c700b5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53512/0 (socket says 172.21.15.43:53512) 2024-03-30T13:11:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6dd9b640 1 --2- 172.21.15.43:0/164227951 >> v2:172.21.15.113:3300/0 conn(0x7f5c7015f5b0 0x7f5c700ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 -- 172.21.15.43:0/164227951 >> v2:172.21.15.113:3300/0 conn(0x7f5c7015f5b0 msgr2=0x7f5c700ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 --2- 172.21.15.43:0/164227951 >> v2:172.21.15.113:3300/0 conn(0x7f5c7015f5b0 0x7f5c700ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6d59a640 1 -- 172.21.15.43:0/164227951 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952626437 0 0) 0x7f5c700ccfd0 con 0x7f5c7006c850 2024-03-30T13:11:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 -- 172.21.15.43:0/164227951 >> v1:172.21.15.43:6789/0 conn(0x7f5c7006c850 legacy=0x7f5c700b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 -- 172.21.15.43:0/164227951 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c700b4f30 con 0x7f5c7006cc30 2024-03-30T13:11:00.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6ed9d640 1 --2- 172.21.15.43:0/164227951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006cc30 0x7f5c700ae2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c70067db0 tx=0x7f5c6801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3ca8b4f1debb32 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:00.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.140+0000 7f5c6d59a640 1 -- 172.21.15.43:0/164227951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c680a56c0 con 0x7f5c7006cc30 2024-03-30T13:11:00.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c6d59a640 1 -- 172.21.15.43:0/164227951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c680a5860 con 0x7f5c7006cc30 2024-03-30T13:11:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c6d59a640 1 -- 172.21.15.43:0/164227951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c680a5c30 con 0x7f5c7006cc30 2024-03-30T13:11:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 -- 172.21.15.43:0/164227951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006cc30 msgr2=0x7f5c700ae2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/164227951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006cc30 0x7f5c700ae2b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5c70067db0 tx=0x7f5c6801c9f0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 -- 172.21.15.43:0/164227951 shutdown_connections 2024-03-30T13:11:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/164227951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006cc30 0x7f5c700ae2b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/164227951 >> v2:172.21.15.113:3300/0 conn(0x7f5c7015f5b0 0x7f5c700ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 -- 172.21.15.43:0/164227951 >> 172.21.15.43:0/164227951 conn(0x7f5c700d0c90 msgr2=0x7f5c700d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 -- 172.21.15.43:0/164227951 shutdown_connections 2024-03-30T13:11:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 -- 172.21.15.43:0/164227951 wait complete. 2024-03-30T13:11:00.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 Processor -- start 2024-03-30T13:11:00.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.144+0000 7f5c74b1f640 1 -- start start 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006c850 0x7f5c70157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c7006cc30 0x7f5c70149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c7015f5b0 0x7f5c701499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5c700aec00 con 0x7f5c7006c850 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7006c080 con 0x7f5c7006cc30 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5c7016b4c0 con 0x7f5c7015f5b0 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006c850 0x7f5c70157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006c850 0x7f5c70157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44094/0 (socket says 172.21.15.43:44094) 2024-03-30T13:11:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 -- 172.21.15.43:0/633836672 learned_addr learned my addr 172.21.15.43:0/633836672 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6dd9b640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c7006cc30 0x7f5c70149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6ed9d640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c7015f5b0 0x7f5c701499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 -- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c7015f5b0 msgr2=0x7f5c701499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c7015f5b0 0x7f5c701499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 -- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c7006cc30 msgr2=0x7f5c70149480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c7006cc30 0x7f5c70149480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 -- 172.21.15.43:0/633836672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c70153950 con 0x7f5c7006c850 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6dd9b640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c7006cc30 0x7f5c70149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6e59c640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006c850 0x7f5c70157170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c600011f0 tx=0x7f5c6000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c5effd640 1 -- 172.21.15.43:0/633836672 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60018070 con 0x7f5c7006c850 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c5effd640 1 -- 172.21.15.43:0/633836672 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c600074e0 con 0x7f5c7006c850 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c70152b90 con 0x7f5c7006c850 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c5effd640 1 -- 172.21.15.43:0/633836672 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60004a10 con 0x7f5c7006c850 2024-03-30T13:11:00.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c70153150 con 0x7f5c7006c850 2024-03-30T13:11:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c70068900 con 0x7f5c7006c850 2024-03-30T13:11:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c5effd640 1 -- 172.21.15.43:0/633836672 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5c6001b070 con 0x7f5c7006c850 2024-03-30T13:11:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c5effd640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c50067ff0 0x7f5c5006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c5effd640 1 -- 172.21.15.43:0/633836672 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6500+0+0 (secure 0 0 0) 0x7f5c60085d60 con 0x7f5c7006c850 2024-03-30T13:11:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.148+0000 7f5c6dd9b640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c50067ff0 0x7f5c5006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:00.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.152+0000 7f5c6dd9b640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c50067ff0 0x7f5c5006a4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c701fa900 tx=0x7f5c64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.160+0000 7f5c5effd640 1 -- 172.21.15.43:0/633836672 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5c60056360 con 0x7f5c7006c850 2024-03-30T13:11:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.360+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f5c700b4150 con 0x7f5c7006c850 2024-03-30T13:11:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.980+0000 7f5c5effd640 1 -- 172.21.15.43:0/633836672 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f5c60055c60 con 0x7f5c7006c850 2024-03-30T13:11:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:added data pool 4 to fsmap 2024-03-30T13:11:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c50067ff0 msgr2=0x7f5c5006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c50067ff0 0x7f5c5006a4b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c701fa900 tx=0x7f5c64008040 comp rx=0 tx=0).stop 2024-03-30T13:11:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006c850 msgr2=0x7f5c70157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006c850 0x7f5c70157170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5c600011f0 tx=0x7f5c6000c9b0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 shutdown_connections 2024-03-30T13:11:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c50067ff0 0x7f5c5006a4b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5c7015f5b0 0x7f5c701499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5c7006cc30 0x7f5c70149480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 --2- 172.21.15.43:0/633836672 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c7006c850 0x7f5c70157170 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 >> 172.21.15.43:0/633836672 conn(0x7f5c700d0c90 msgr2=0x7f5c70154d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 shutdown_connections 2024-03-30T13:11:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:00.984+0000 7f5c74b1f640 1 -- 172.21.15.43:0/633836672 wait complete. 2024-03-30T13:11:01.008 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:01.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f7c8f640 1 Processor -- start 2024-03-30T13:11:01.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f7c8f640 1 -- start start 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f7c8f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f00b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f7c8f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5f00d1100 0x7fe5f00d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f7c8f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f00d3e60 con 0x7fe5f00b8160 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f7c8f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe5f00b7450 con 0x7fe5f00d1100 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f7c8f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f00d1aa0 con 0x7fe5f015e600 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f522a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5f00d1100 0x7fe5f00d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f522a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5f00d1100 0x7fe5f00d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44110/0 (socket says 172.21.15.43:44110) 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f522a640 1 -- 172.21.15.43:0/3185341148 learned_addr learned my addr 172.21.15.43:0/3185341148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f622c640 1 --1- 172.21.15.43:0/3185341148 >> v1:172.21.15.43:6789/0 conn(0x7fe5f00b8160 0x7fe5f00b85f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53528/0 (socket says 172.21.15.43:53528) 2024-03-30T13:11:01.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f5a2b640 1 --2- 172.21.15.43:0/3185341148 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f00b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f522a640 1 -- 172.21.15.43:0/3185341148 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 msgr2=0x7fe5f00b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f522a640 1 --2- 172.21.15.43:0/3185341148 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f00b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f522a640 1 -- 172.21.15.43:0/3185341148 >> v1:172.21.15.43:6789/0 conn(0x7fe5f00b8160 legacy=0x7fe5f00b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f522a640 1 -- 172.21.15.43:0/3185341148 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f00b7680 con 0x7fe5f00d1100 2024-03-30T13:11:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f5a2b640 1 --2- 172.21.15.43:0/3185341148 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f00b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.124+0000 7fe5f4a29640 1 -- 172.21.15.43:0/3185341148 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997372573 0 0) 0x7fe5f00d3e60 con 0x7fe5f00b8160 2024-03-30T13:11:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 --2- 172.21.15.43:0/3185341148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f00d1560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009870 tx=0x7fe5e001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85edea13758ff90f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f4a29640 1 -- 172.21.15.43:0/3185341148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e00ab650 con 0x7fe5f00d1100 2024-03-30T13:11:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f4a29640 1 -- 172.21.15.43:0/3185341148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e00b0070 con 0x7fe5f00d1100 2024-03-30T13:11:01.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f4a29640 1 -- 172.21.15.43:0/3185341148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e00aba10 con 0x7fe5f00d1100 2024-03-30T13:11:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3185341148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 msgr2=0x7fe5f00d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3185341148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f00d1560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0009870 tx=0x7fe5e001c560 comp rx=0 tx=0).stop 2024-03-30T13:11:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3185341148 shutdown_connections 2024-03-30T13:11:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3185341148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f00d1560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3185341148 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f00b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3185341148 >> 172.21.15.43:0/3185341148 conn(0x7fe5f00b6230 msgr2=0x7fe5f00b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3185341148 shutdown_connections 2024-03-30T13:11:01.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3185341148 wait complete. 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 Processor -- start 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- start start 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f00b8160 0x7fe5f015ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f014e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f014e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f016afd0 con 0x7fe5f00d1100 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5f016b150 con 0x7fe5f00b8160 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f7c8f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe5f016b2d0 con 0x7fe5f015e600 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f014e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f014e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44122/0 (socket says 172.21.15.43:44122) 2024-03-30T13:11:01.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 -- 172.21.15.43:0/3988226121 learned_addr learned my addr 172.21.15.43:0/3988226121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f5a2b640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f00b8160 0x7fe5f015ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f622c640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f014e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 -- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 msgr2=0x7fe5f014e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f014e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 -- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f00b8160 msgr2=0x7fe5f015ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f00b8160 0x7fe5f015ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 -- 172.21.15.43:0/3988226121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f00c9930 con 0x7fe5f00d1100 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f622c640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f014e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5f522a640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f014e3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00ad040 tx=0x7fe5e001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.128+0000 7fe5e67fc640 1 -- 172.21.15.43:0/3988226121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e00abe60 con 0x7fe5f00d1100 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5e67fc640 1 -- 172.21.15.43:0/3988226121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5e00b4800 con 0x7fe5f00d1100 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f00c9c10 con 0x7fe5f00d1100 2024-03-30T13:11:01.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5e67fc640 1 -- 172.21.15.43:0/3988226121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5e00b4bf0 con 0x7fe5f00d1100 2024-03-30T13:11:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5f00ca1d0 con 0x7fe5f00d1100 2024-03-30T13:11:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5f00ca6c0 con 0x7fe5f00d1100 2024-03-30T13:11:01.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5e67fc640 1 -- 172.21.15.43:0/3988226121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe5e00c7440 con 0x7fe5f00d1100 2024-03-30T13:11:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5e67fc640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe5d4067fc0 0x7fe5d406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5e67fc640 1 -- 172.21.15.43:0/3988226121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe5e0121150 con 0x7fe5f00d1100 2024-03-30T13:11:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5f5a2b640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe5d4067fc0 0x7fe5d406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.132+0000 7fe5f5a2b640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe5d4067fc0 0x7fe5d406a480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5f015b710 tx=0x7fe5ec008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.136+0000 7fe5e67fc640 1 -- 172.21.15.43:0/3988226121 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe5e0016150 con 0x7fe5f00d1100 2024-03-30T13:11:01.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.328+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5f0068890 con 0x7fe5f00d1100 2024-03-30T13:11:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5e67fc640 1 -- 172.21.15.43:0/3988226121 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7fe5e00f1770 con 0x7fe5f00d1100 2024-03-30T13:11:01.337 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:01.337 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:00.365049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:11:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2024-03-30T13:11:01.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe5d4067fc0 msgr2=0x7fe5d406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe5d4067fc0 0x7fe5d406a480 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5f015b710 tx=0x7fe5ec008040 comp rx=0 tx=0).stop 2024-03-30T13:11:01.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 msgr2=0x7fe5f014e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f014e3a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5e00ad040 tx=0x7fe5e001ada0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 shutdown_connections 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe5d4067fc0 0x7fe5d406a480 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5f015e600 0x7fe5f014e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5f00d1100 0x7fe5f014e3a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 --2- 172.21.15.43:0/3988226121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5f00b8160 0x7fe5f015ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 >> 172.21.15.43:0/3988226121 conn(0x7fe5f00b6230 msgr2=0x7fe5f015bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 shutdown_connections 2024-03-30T13:11:01.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.332+0000 7fe5f7c8f640 1 -- 172.21.15.43:0/3988226121 wait complete. 2024-03-30T13:11:01.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:01.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392c1e4640 1 Processor -- start 2024-03-30T13:11:01.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392c1e4640 1 -- start start 2024-03-30T13:11:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392c1e4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f392406c670 0x7f39240cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392c1e4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392406e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392c1e4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f392406e5c0 con 0x7f392406ca50 2024-03-30T13:11:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392c1e4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f39240b34c0 con 0x7f392406c670 2024-03-30T13:11:01.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392c1e4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f39240ccef0 con 0x7f39240b41f0 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392a781640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f392406ca50 0x7f392415fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34582/0 (socket says 172.21.15.43:34582) 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.460+0000 7f392a781640 1 -- 172.21.15.43:0/602623283 learned_addr learned my addr 172.21.15.43:0/602623283 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3929f80640 1 --2- 172.21.15.43:0/602623283 >> v2:172.21.15.113:3300/0 conn(0x7f392406c670 0x7f39240cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f392977f640 1 --2- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392406e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3928f7e640 1 -- 172.21.15.43:0/602623283 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85360843 0 0) 0x7f392406e5c0 con 0x7f392406ca50 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3928f7e640 1 -- 172.21.15.43:0/602623283 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3910003530 con 0x7f392406ca50 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3929f80640 1 -- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 msgr2=0x7f392406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3929f80640 1 --2- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392406e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3928f7e640 1 -- 172.21.15.43:0/602623283 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3893985966 0 0) 0x7f3910003530 con 0x7f392406ca50 2024-03-30T13:11:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3929f80640 1 -- 172.21.15.43:0/602623283 >> v1:172.21.15.43:6789/0 conn(0x7f392406ca50 legacy=0x7f392415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.464+0000 7f3929f80640 1 -- 172.21.15.43:0/602623283 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240b3710 con 0x7f392406c670 2024-03-30T13:11:01.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f3929f80640 1 --2- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f39240cc910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39200097c0 tx=0x7f392001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b54e23df650d643 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f3928f7e640 1 -- 172.21.15.43:0/602623283 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39200b0070 con 0x7f392406c670 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f3928f7e640 1 -- 172.21.15.43:0/602623283 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39200ab400 con 0x7f392406c670 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f3928f7e640 1 -- 172.21.15.43:0/602623283 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c001a00 con 0x7f392406c670 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 msgr2=0x7f39240cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 --2- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f39240cc910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f39200097c0 tx=0x7f392001c3c0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- 172.21.15.43:0/602623283 shutdown_connections 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 --2- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392406e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 --2- 172.21.15.43:0/602623283 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f39240cc910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- 172.21.15.43:0/602623283 >> 172.21.15.43:0/602623283 conn(0x7f39240cfee0 msgr2=0x7f39240d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- 172.21.15.43:0/602623283 shutdown_connections 2024-03-30T13:11:01.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- 172.21.15.43:0/602623283 wait complete. 2024-03-30T13:11:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 Processor -- start 2024-03-30T13:11:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- start start 2024-03-30T13:11:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f3924153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392406ca50 0x7f392415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3924146000 con 0x7f392406ca50 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3924145e80 con 0x7f392406c670 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3924168ef0 con 0x7f39240b41f0 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392406ca50 0x7f392415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392406ca50 0x7f392415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59606/0 (socket says 172.21.15.43:59606) 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 -- 172.21.15.43:0/2341990909 learned_addr learned my addr 172.21.15.43:0/2341990909 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392a781640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f3929f80640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f3924153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 -- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 msgr2=0x7f392415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392415aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 -- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 msgr2=0x7f3924153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f3924153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 -- 172.21.15.43:0/2341990909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39240714a0 con 0x7f392406ca50 2024-03-30T13:11:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f3929f80640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f3924153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392977f640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392406ca50 0x7f392415a560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f391400e870 tx=0x7f391400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f391a7fc640 1 -- 172.21.15.43:0/2341990909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3914017070 con 0x7f392406ca50 2024-03-30T13:11:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f391a7fc640 1 -- 172.21.15.43:0/2341990909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391400c940 con 0x7f392406ca50 2024-03-30T13:11:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.468+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392415d870 con 0x7f392406ca50 2024-03-30T13:11:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.472+0000 7f391a7fc640 1 -- 172.21.15.43:0/2341990909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391400ccf0 con 0x7f392406ca50 2024-03-30T13:11:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.472+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3924154760 con 0x7f392406ca50 2024-03-30T13:11:01.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.472+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3924068890 con 0x7f392406ca50 2024-03-30T13:11:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.472+0000 7f391a7fc640 1 -- 172.21.15.43:0/2341990909 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f391401a070 con 0x7f392406ca50 2024-03-30T13:11:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.472+0000 7f391a7fc640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3908067ff0 0x7f390806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.472+0000 7f391a7fc640 1 -- 172.21.15.43:0/2341990909 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3914086e30 con 0x7f392406ca50 2024-03-30T13:11:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.472+0000 7f3929f80640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3908067ff0 0x7f390806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.476+0000 7f3929f80640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3908067ff0 0x7f390806a4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39200ad000 tx=0x7f39200a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.476+0000 7f391a7fc640 1 -- 172.21.15.43:0/2341990909 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3914057380 con 0x7f392406ca50 2024-03-30T13:11:01.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.732+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39240b32a0 con 0x7f392406ca50 2024-03-30T13:11:01.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f391a7fc640 1 -- 172.21.15.43:0/2341990909 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7f3914056c80 con 0x7f392406ca50 2024-03-30T13:11:01.743 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:01.743 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3908067ff0 msgr2=0x7f390806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3908067ff0 0x7f390806a4b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39200ad000 tx=0x7f39200a6000 comp rx=0 tx=0).stop 2024-03-30T13:11:01.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392406ca50 msgr2=0x7f392415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392406ca50 0x7f392415a560 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f391400e870 tx=0x7f391400ec70 comp rx=0 tx=0).stop 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 shutdown_connections 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3908067ff0 0x7f390806a4b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39240b41f0 0x7f392415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f392406ca50 0x7f392415a560 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 --2- 172.21.15.43:0/2341990909 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f392406c670 0x7f3924153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 >> 172.21.15.43:0/2341990909 conn(0x7f39240cfee0 msgr2=0x7f392413e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 shutdown_connections 2024-03-30T13:11:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.736+0000 7f392c1e4640 1 -- 172.21.15.43:0/2341990909 wait complete. 2024-03-30T13:11:01.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:01.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4fd212640 1 Processor -- start 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4fd212640 1 -- start start 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4fd212640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f8160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4fd212640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4f81609b0 0x7fa4f806d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4fd212640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4f8141970 con 0x7fa4f806dcc0 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4fd212640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fa4f806e110 con 0x7fa4f81609b0 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4fd212640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f8141c50 con 0x7fa4f813f0a0 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4f81609b0 0x7fa4f806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f759e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa4f806dcc0 0x7fa4f806b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34598/0 (socket says 172.21.15.43:34598) 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4f81609b0 0x7fa4f806d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59610/0 (socket says 172.21.15.43:59610) 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f759e640 1 -- 172.21.15.43:0/2051487201 learned_addr learned my addr 172.21.15.43:0/2051487201 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:01.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f6d9d640 1 --2- 172.21.15.43:0/2051487201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f8160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f659c640 1 -- 172.21.15.43:0/2051487201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 msgr2=0x7fa4f8160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f659c640 1 --2- 172.21.15.43:0/2051487201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f8160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f659c640 1 -- 172.21.15.43:0/2051487201 >> v1:172.21.15.43:6789/0 conn(0x7fa4f806dcc0 legacy=0x7fa4f806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.876+0000 7fa4f659c640 1 -- 172.21.15.43:0/2051487201 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f8141f30 con 0x7fa4f81609b0 2024-03-30T13:11:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4f659c640 1 --2- 172.21.15.43:0/2051487201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81609b0 0x7fa4f806d780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0097c0 tx=0x7fa4ec01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc6b55fb0409adba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4f5d9b640 1 -- 172.21.15.43:0/2051487201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0a7070 con 0x7fa4f81609b0 2024-03-30T13:11:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4f5d9b640 1 -- 172.21.15.43:0/2051487201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ec0b0070 con 0x7fa4f81609b0 2024-03-30T13:11:01.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4f5d9b640 1 -- 172.21.15.43:0/2051487201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec0ab890 con 0x7fa4f81609b0 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 -- 172.21.15.43:0/2051487201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81609b0 msgr2=0x7fa4f806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2051487201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81609b0 0x7fa4f806d780 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0097c0 tx=0x7fa4ec01c470 comp rx=0 tx=0).stop 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 -- 172.21.15.43:0/2051487201 shutdown_connections 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2051487201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f81609b0 0x7fa4f806d780 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2051487201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f8160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 -- 172.21.15.43:0/2051487201 >> 172.21.15.43:0/2051487201 conn(0x7fa4f8072500 msgr2=0x7fa4f8072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 -- 172.21.15.43:0/2051487201 shutdown_connections 2024-03-30T13:11:01.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 -- 172.21.15.43:0/2051487201 wait complete. 2024-03-30T13:11:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 Processor -- start 2024-03-30T13:11:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.880+0000 7fa4fd212640 1 -- start start 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f806dcc0 0x7fa4f815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f80b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4f81609b0 0x7fa4f815b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f81600a0 con 0x7fa4f806dcc0 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f815ff20 con 0x7fa4f81609b0 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f8168eb0 con 0x7fa4f813f0a0 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f806dcc0 0x7fa4f815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f806dcc0 0x7fa4f815f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59614/0 (socket says 172.21.15.43:59614) 2024-03-30T13:11:01.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 -- 172.21.15.43:0/2531740087 learned_addr learned my addr 172.21.15.43:0/2531740087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f659c640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f80b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f759e640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4f81609b0 0x7fa4f815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 -- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 msgr2=0x7fa4f80b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f80b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 -- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4f81609b0 msgr2=0x7fa4f815b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4f81609b0 0x7fa4f815b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 -- 172.21.15.43:0/2531740087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f8166b30 con 0x7fa4f806dcc0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f659c640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f80b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f759e640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4f81609b0 0x7fa4f815b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4f6d9d640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f806dcc0 0x7fa4f815f330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800d830 tx=0x7fa4e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4df7fe640 1 -- 172.21.15.43:0/2531740087 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e800b6c0 con 0x7fa4f806dcc0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4df7fe640 1 -- 172.21.15.43:0/2531740087 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4e8019070 con 0x7fa4f806dcc0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f80b41e0 con 0x7fa4f806dcc0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4df7fe640 1 -- 172.21.15.43:0/2531740087 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e800ba60 con 0x7fa4f806dcc0 2024-03-30T13:11:01.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f815b950 con 0x7fa4f806dcc0 2024-03-30T13:11:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f8068890 con 0x7fa4f806dcc0 2024-03-30T13:11:01.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4df7fe640 1 -- 172.21.15.43:0/2531740087 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa4e801e460 con 0x7fa4f806dcc0 2024-03-30T13:11:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.884+0000 7fa4df7fe640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa4e4067fc0 0x7fa4e406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.888+0000 7fa4df7fe640 1 -- 172.21.15.43:0/2531740087 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa4e8086fa0 con 0x7fa4f806dcc0 2024-03-30T13:11:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.888+0000 7fa4f659c640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa4e4067fc0 0x7fa4e406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.888+0000 7fa4f659c640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa4e4067fc0 0x7fa4e406a480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0af000 tx=0x7fa4ec0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:01.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:01.892+0000 7fa4df7fe640 1 -- 172.21.15.43:0/2531740087 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa4e8057410 con 0x7fa4f806dcc0 2024-03-30T13:11:02.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.068+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4f806da80 con 0x7fa4f806dcc0 2024-03-30T13:11:02.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4df7fe640 1 -- 172.21.15.43:0/2531740087 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7fa4e8056d10 con 0x7fa4f806dcc0 2024-03-30T13:11:02.078 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:02.078 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:02.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa4e4067fc0 msgr2=0x7fa4e406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa4e4067fc0 0x7fa4e406a480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec0af000 tx=0x7fa4ec0a7000 comp rx=0 tx=0).stop 2024-03-30T13:11:02.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f806dcc0 msgr2=0x7fa4f815f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f806dcc0 0x7fa4f815f330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800d830 tx=0x7fa4e800dcf0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 shutdown_connections 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa4e4067fc0 0x7fa4e406a480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa4f81609b0 0x7fa4f815b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa4f813f0a0 0x7fa4f80b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 --2- 172.21.15.43:0/2531740087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4f806dcc0 0x7fa4f815f330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 >> 172.21.15.43:0/2531740087 conn(0x7fa4f8072500 msgr2=0x7fa4f815c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 shutdown_connections 2024-03-30T13:11:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.072+0000 7fa4fd212640 1 -- 172.21.15.43:0/2531740087 wait complete. 2024-03-30T13:11:02.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d7a6e640 1 Processor -- start 2024-03-30T13:11:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d7a6e640 1 -- start start 2024-03-30T13:11:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d7a6e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 0x7fa3d006dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d7a6e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa3d00cfa20 0x7fa3d00cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d7a6e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3d0141970 con 0x7fa3d013f0a0 2024-03-30T13:11:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d7a6e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa3d006e100 con 0x7fa3d00cfa20 2024-03-30T13:11:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d7a6e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d0141c50 con 0x7fa3d006d780 2024-03-30T13:11:02.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa3d013f0a0 0x7fa3d0160470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34602/0 (socket says 172.21.15.43:34602) 2024-03-30T13:11:02.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 -- 172.21.15.43:0/1900705884 learned_addr learned my addr 172.21.15.43:0/1900705884 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d600b640 1 --2- 172.21.15.43:0/1900705884 >> v2:172.21.15.113:3300/0 conn(0x7fa3d00cfa20 0x7fa3d00cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d5009640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 0x7fa3d006dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1927650304 0 0) 0x7fa3d0141970 con 0x7fa3d013f0a0 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3bc003530 con 0x7fa3d013f0a0 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1871602475 0 0) 0x7fa3bc003530 con 0x7fa3d013f0a0 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 msgr2=0x7fa3d006dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 0x7fa3d006dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 >> v2:172.21.15.113:3300/0 conn(0x7fa3d00cfa20 msgr2=0x7fa3d00cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 --2- 172.21.15.43:0/1900705884 >> v2:172.21.15.113:3300/0 conn(0x7fa3d00cfa20 0x7fa3d00cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d0141f30 con 0x7fa3d013f0a0 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fa3cc001680 con 0x7fa3d013f0a0 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 >> v1:172.21.15.43:6789/0 conn(0x7fa3d013f0a0 legacy=0x7fa3d0160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc0039c0 0x7fa3bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc004810 0x7fa3bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 0x7fa3bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d006e100 con 0x7fa3bc0039c0 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d0141c50 con 0x7fa3bc004810 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa3bc00b9d0 con 0x7fa3d006d780 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc0039c0 0x7fa3bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d5009640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc004810 0x7fa3bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d600b640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 0x7fa3bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 -- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 msgr2=0x7fa3bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d006d780 0x7fa3bc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 -- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc004810 msgr2=0x7fa3bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc004810 0x7fa3bc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.216+0000 7fa3d580a640 1 -- 172.21.15.43:0/1900705884 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3bc003530 con 0x7fa3bc0039c0 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d580a640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc0039c0 0x7fa3bc003de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00c240 tx=0x7fa3cc01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61868ad189ef0157 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc007d40 con 0x7fa3bc0039c0 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0a9020 con 0x7fa3bc0039c0 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d4808640 1 -- 172.21.15.43:0/1900705884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc005b00 con 0x7fa3bc0039c0 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1900705884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc0039c0 msgr2=0x7fa3bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d7a6e640 1 --2- 172.21.15.43:0/1900705884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3bc0039c0 0x7fa3bc003de0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00c240 tx=0x7fa3cc01cc70 comp rx=0 tx=0).stop 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d600b640 1 -- 172.21.15.43:0/1900705884 reap_dead start 2024-03-30T13:11:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1900705884 shutdown_connections 2024-03-30T13:11:02.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1900705884 >> 172.21.15.43:0/1900705884 conn(0x7fa3d0072500 msgr2=0x7fa3d0072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1900705884 shutdown_connections 2024-03-30T13:11:02.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.220+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1900705884 wait complete. 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 Processor -- start 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 -- start start 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc0039c0 0x7fa3d0138720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d0140650 0x7fa3d0140ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d01eabe0 0x7fa3d01eb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d015ff20 con 0x7fa3d0140650 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d013f480 con 0x7fa3bc0039c0 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa3d01600a0 con 0x7fa3d01eabe0 2024-03-30T13:11:02.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d0140650 0x7fa3d0140ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d0140650 0x7fa3d0140ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59644/0 (socket says 172.21.15.43:59644) 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 -- 172.21.15.43:0/1312111415 learned_addr learned my addr 172.21.15.43:0/1312111415 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d580a640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc0039c0 0x7fa3d0138720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d600b640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d01eabe0 0x7fa3d01eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 -- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d01eabe0 msgr2=0x7fa3d01eb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d01eabe0 0x7fa3d01eb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 -- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc0039c0 msgr2=0x7fa3d0138720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc0039c0 0x7fa3d0138720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 -- 172.21.15.43:0/1312111415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d00b3f90 con 0x7fa3d0140650 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d600b640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d01eabe0 0x7fa3d01eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d5009640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d0140650 0x7fa3d0140ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00108c0 tx=0x7fa3c0010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3c67fc640 1 -- 172.21.15.43:0/1312111415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c000f040 con 0x7fa3d0140650 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3c67fc640 1 -- 172.21.15.43:0/1312111415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c000eb60 con 0x7fa3d0140650 2024-03-30T13:11:02.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d01eb9b0 con 0x7fa3d0140650 2024-03-30T13:11:02.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3c67fc640 1 -- 172.21.15.43:0/1312111415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c001c400 con 0x7fa3d0140650 2024-03-30T13:11:02.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d00b4530 con 0x7fa3d0140650 2024-03-30T13:11:02.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d01603f0 con 0x7fa3d0140650 2024-03-30T13:11:02.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3c67fc640 1 -- 172.21.15.43:0/1312111415 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa3c001c5a0 con 0x7fa3d0140650 2024-03-30T13:11:02.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3c67fc640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa3b4067fc0 0x7fa3b406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3c67fc640 1 -- 172.21.15.43:0/1312111415 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa3c0086f60 con 0x7fa3d0140650 2024-03-30T13:11:02.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.224+0000 7fa3d580a640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa3b4067fc0 0x7fa3b406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.228+0000 7fa3d580a640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa3b4067fc0 0x7fa3b406a480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0aa000 tx=0x7fa3d015c770 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.232+0000 7fa3c67fc640 1 -- 172.21.15.43:0/1312111415 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa3c0018030 con 0x7fa3d0140650 2024-03-30T13:11:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.424+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3d0068890 con 0x7fa3d0140650 2024-03-30T13:11:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.424+0000 7fa3c67fc640 1 -- 172.21.15.43:0/1312111415 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7fa3c005d030 con 0x7fa3d0140650 2024-03-30T13:11:02.431 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:02.431 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:00.365049+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:11:02.432 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 7 2024-03-30T13:11:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa3b4067fc0 msgr2=0x7fa3b406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa3b4067fc0 0x7fa3b406a480 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0aa000 tx=0x7fa3d015c770 comp rx=0 tx=0).stop 2024-03-30T13:11:02.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d0140650 msgr2=0x7fa3d0140ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d0140650 0x7fa3d0140ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3c00108c0 tx=0x7fa3c0010d80 comp rx=0 tx=0).stop 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 shutdown_connections 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa3b4067fc0 0x7fa3b406a480 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa3d01eabe0 0x7fa3d01eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3d0140650 0x7fa3d0140ad0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 --2- 172.21.15.43:0/1312111415 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa3bc0039c0 0x7fa3d0138720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 >> 172.21.15.43:0/1312111415 conn(0x7fa3d0072500 msgr2=0x7fa3d00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 shutdown_connections 2024-03-30T13:11:02.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.428+0000 7fa3d7a6e640 1 -- 172.21.15.43:0/1312111415 wait complete. 2024-03-30T13:11:02.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff090bfc640 1 Processor -- start 2024-03-30T13:11:02.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff090bfc640 1 -- start start 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff090bfc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 0x7ff08c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff090bfc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08c0cf630 con 0x7ff08c0b5060 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff090bfc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08c0ccba0 con 0x7ff08c0d0900 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff090bfc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c0ce3a0 con 0x7ff08c15c1c0 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08ad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff08c0b5060 0x7ff08c0cef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34606/0 (socket says 172.21.15.43:34606) 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08ad9d640 1 -- 172.21.15.43:0/3481887066 learned_addr learned my addr 172.21.15.43:0/3481887066 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08a59c640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 0x7ff08c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075775159 0 0) 0x7ff08c0cf630 con 0x7ff08c0b5060 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff074003530 con 0x7ff08c0b5060 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 610071659 0 0) 0x7ff074003530 con 0x7ff08c0b5060 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 msgr2=0x7ff08c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 0x7ff08c0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 >> v1:172.21.15.113:6789/0 conn(0x7ff08c0d0900 legacy=0x7ff08c0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c0b26f0 con 0x7ff08c0b5060 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ff084001a00 con 0x7ff08c0b5060 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 >> v1:172.21.15.43:6789/0 conn(0x7ff08c0b5060 legacy=0x7ff08c0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740039f0 0x7ff074003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074004810 0x7ff074004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 0x7ff0740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff07400ba80 con 0x7ff074004810 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff07400bc30 con 0x7ff0740039f0 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff07400be30 con 0x7ff08c15c1c0 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff089d9b640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074004810 0x7ff074004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08a59c640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740039f0 0x7ff074003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 0x7ff0740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff089d9b640 1 -- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 msgr2=0x7ff0740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff089d9b640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c15c1c0 0x7ff0740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff089d9b640 1 -- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740039f0 msgr2=0x7ff074003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:02.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff089d9b640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740039f0 0x7ff074003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff089d9b640 1 -- 172.21.15.43:0/3481887066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff074003530 con 0x7ff074004810 2024-03-30T13:11:02.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.552+0000 7ff08a59c640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0740039f0 0x7ff074003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:02.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff089d9b640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074004810 0x7ff074004c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0d0d40 tx=0x7ff08001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7930ac8cf09c930 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0800013a0 con 0x7ff074004810 2024-03-30T13:11:02.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0800052f0 con 0x7ff074004810 2024-03-30T13:11:02.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08959a640 1 -- 172.21.15.43:0/3481887066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff080003840 con 0x7ff074004810 2024-03-30T13:11:02.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- 172.21.15.43:0/3481887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074004810 msgr2=0x7ff074004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 --2- 172.21.15.43:0/3481887066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff074004810 0x7ff074004c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0d0d40 tx=0x7ff08001c3a0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 -- 172.21.15.43:0/3481887066 reap_dead start 2024-03-30T13:11:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- 172.21.15.43:0/3481887066 shutdown_connections 2024-03-30T13:11:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- 172.21.15.43:0/3481887066 >> 172.21.15.43:0/3481887066 conn(0x7ff08c0b3d60 msgr2=0x7ff08c0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- 172.21.15.43:0/3481887066 shutdown_connections 2024-03-30T13:11:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- 172.21.15.43:0/3481887066 wait complete. 2024-03-30T13:11:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 Processor -- start 2024-03-30T13:11:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- start start 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff074004810 0x7ff08c0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0c5e80 0x7ff08c0c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c7a30 0x7ff08c0d2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0b2cb0 con 0x7ff08c0c7a30 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c0cce80 con 0x7ff074004810 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff090bfc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff08c0cc7e0 con 0x7ff08c0c5e80 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c7a30 0x7ff08c0d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c7a30 0x7ff08c0d2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59664/0 (socket says 172.21.15.43:59664) 2024-03-30T13:11:02.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 -- 172.21.15.43:0/562562922 learned_addr learned my addr 172.21.15.43:0/562562922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff089d9b640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0c5e80 0x7ff08c0c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08a59c640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff074004810 0x7ff08c0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 -- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0c5e80 msgr2=0x7ff08c0c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0c5e80 0x7ff08c0c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 -- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff074004810 msgr2=0x7ff08c0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff074004810 0x7ff08c0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.556+0000 7ff08ad9d640 1 -- 172.21.15.43:0/562562922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c0cb830 con 0x7ff08c0c7a30 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff08ad9d640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c7a30 0x7ff08c0d2610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0b4900 tx=0x7ff084005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff07affd640 1 -- 172.21.15.43:0/562562922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08401a070 con 0x7ff08c0c7a30 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff07affd640 1 -- 172.21.15.43:0/562562922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff084011040 con 0x7ff08c0c7a30 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff07affd640 1 -- 172.21.15.43:0/562562922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400ed80 con 0x7ff08c0c7a30 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c0c8040 con 0x7ff08c0c7a30 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c0c8600 con 0x7ff08c0c7a30 2024-03-30T13:11:02.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08c068890 con 0x7ff08c0c7a30 2024-03-30T13:11:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff07affd640 1 -- 172.21.15.43:0/562562922 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff08401d070 con 0x7ff08c0c7a30 2024-03-30T13:11:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff07affd640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff06c067fc0 0x7ff06c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:02.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff07affd640 1 -- 172.21.15.43:0/562562922 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff084089080 con 0x7ff08c0c7a30 2024-03-30T13:11:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff08a59c640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff06c067fc0 0x7ff06c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.560+0000 7ff08a59c640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff06c067fc0 0x7ff06c06a480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009ac0 tx=0x7ff07c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:02.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.564+0000 7ff07affd640 1 -- 172.21.15.43:0/562562922 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff084059580 con 0x7ff08c0c7a30 2024-03-30T13:11:02.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.744+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff08c160920 con 0x7ff08c0c7a30 2024-03-30T13:11:02.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff07affd640 1 -- 172.21.15.43:0/562562922 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7ff084058e80 con 0x7ff08c0c7a30 2024-03-30T13:11:02.755 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:02.755 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff06c067fc0 msgr2=0x7ff06c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff06c067fc0 0x7ff06c06a480 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009ac0 tx=0x7ff07c008040 comp rx=0 tx=0).stop 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c7a30 msgr2=0x7ff08c0d2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c7a30 0x7ff08c0d2610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0b4900 tx=0x7ff084005a60 comp rx=0 tx=0).stop 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 shutdown_connections 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff06c067fc0 0x7ff06c06a480 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff08c0c7a30 0x7ff08c0d2610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff08c0c5e80 0x7ff08c0c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 --2- 172.21.15.43:0/562562922 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff074004810 0x7ff08c0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.748+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 >> 172.21.15.43:0/562562922 conn(0x7ff08c0b3d60 msgr2=0x7ff08c06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.752+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 shutdown_connections 2024-03-30T13:11:02.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:02.752+0000 7ff090bfc640 1 -- 172.21.15.43:0/562562922 wait complete. 2024-03-30T13:11:03.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T13:10:55.286025+0000', 'modified': '2024-03-30T13:11:00.365049+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/855012184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 855012184}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 855012184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4239, 'qdb_cluster': [4239]} 2024-03-30T13:11:03.180 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T13:11:03.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T13:11:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa9776e3640 1 Processor -- start 2024-03-30T13:11:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa9776e3640 1 -- start start 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa9776e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 0x7fa970166030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa9776e3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97015edd0 con 0x7fa970070490 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa9776e3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa97015f0b0 con 0x7fa9700b5b20 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa9776e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa97015f390 con 0x7fa97006de90 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa97547f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa970070490 0x7fa97015c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34620/0 (socket says 172.21.15.43:34620) 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa97547f640 1 -- 172.21.15.43:0/911305697 learned_addr learned my addr 172.21.15.43:0/911305697 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa975c80640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 0x7fa970166030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.312+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1720023319 0 0) 0x7fa97015edd0 con 0x7fa970070490 2024-03-30T13:11:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa95c003430 con 0x7fa970070490 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4242444504 0 0) 0x7fa95c003430 con 0x7fa970070490 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 msgr2=0x7fa970166030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 0x7fa970166030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 >> v1:172.21.15.113:6789/0 conn(0x7fa9700b5b20 legacy=0x7fa97006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97015f670 con 0x7fa970070490 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fa96c001680 con 0x7fa970070490 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 >> v1:172.21.15.43:6789/0 conn(0x7fa970070490 legacy=0x7fa97015c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 0x7fa95c004ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa95c005050 0x7fa95c0036b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa95c003df0 0x7fa95c00fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa97015f390 con 0x7fa95c005050 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c0100c0 con 0x7fa95c003df0 2024-03-30T13:11:03.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa95c010240 con 0x7fa97006de90 2024-03-30T13:11:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa974c7e640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa95c005050 0x7fa95c0036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa97547f640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 0x7fa95c004ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa975c80640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa95c003df0 0x7fa95c00fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa974c7e640 1 -- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 msgr2=0x7fa95c004ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa974c7e640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 0x7fa95c004ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa974c7e640 1 -- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa95c003df0 msgr2=0x7fa95c00fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa974c7e640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa95c003df0 0x7fa95c00fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa974c7e640 1 -- 172.21.15.43:0/911305697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c003430 con 0x7fa95c005050 2024-03-30T13:11:03.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa97547f640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa97006de90 0x7fa95c004ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa974c7e640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa95c005050 0x7fa95c0036b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa960001750 tx=0x7fa96001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fe9001f74a0e78 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600013a0 con 0x7fa95c005050 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9600052f0 con 0x7fa95c005050 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa967fff640 1 -- 172.21.15.43:0/911305697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa960003840 con 0x7fa95c005050 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa9776e3640 1 -- 172.21.15.43:0/911305697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa95c005050 msgr2=0x7fa95c0036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa9776e3640 1 --2- 172.21.15.43:0/911305697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa95c005050 0x7fa95c0036b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa960001750 tx=0x7fa96001c3e0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa975c80640 1 -- 172.21.15.43:0/911305697 reap_dead start 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa9776e3640 1 -- 172.21.15.43:0/911305697 shutdown_connections 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa9776e3640 1 -- 172.21.15.43:0/911305697 >> 172.21.15.43:0/911305697 conn(0x7fa970072500 msgr2=0x7fa970072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa9776e3640 1 -- 172.21.15.43:0/911305697 shutdown_connections 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.316+0000 7fa9776e3640 1 -- 172.21.15.43:0/911305697 wait complete. 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 Processor -- start 2024-03-30T13:11:03.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 -- start start 2024-03-30T13:11:03.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97013b740 0x7fa970151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9700d0230 0x7fa970152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97013bb20 0x7fa97014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa97006c240 con 0x7fa97013b740 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa97006c0c0 con 0x7fa97013bb20 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa97006c3c0 con 0x7fa9700d0230 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97013b740 0x7fa970151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97013b740 0x7fa970151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59688/0 (socket says 172.21.15.43:59688) 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 -- 172.21.15.43:0/2671042535 learned_addr learned my addr 172.21.15.43:0/2671042535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa974c7e640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9700d0230 0x7fa970152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa975c80640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97013bb20 0x7fa97014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 -- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9700d0230 msgr2=0x7fa970152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9700d0230 0x7fa970152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 -- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97013bb20 msgr2=0x7fa97014b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97013bb20 0x7fa97014b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 -- 172.21.15.43:0/2671042535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9700cf190 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa975c80640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97013bb20 0x7fa97014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa97547f640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97013b740 0x7fa970151df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00e780 tx=0x7fa96c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa965ffb640 1 -- 172.21.15.43:0/2671042535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c00ee90 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa965ffb640 1 -- 172.21.15.43:0/2671042535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96c018070 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9700cf470 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa965ffb640 1 -- 172.21.15.43:0/2671042535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c00ca70 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.320+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9700cfa30 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa965ffb640 1 -- 172.21.15.43:0/2671042535 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa96c00cc10 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa965ffb640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa954068160 0x7fa95406a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa965ffb640 1 -- 172.21.15.43:0/2671042535 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa96c086720 con 0x7fa97013b740 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa974c7e640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa954068160 0x7fa95406a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 Processor -- start 2024-03-30T13:11:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 -- start start 2024-03-30T13:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa974c7e640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa954068160 0x7fa95406a620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9700b5fd0 tx=0x7fa9600011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fd3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77fd3dc0 0x563a77f86b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77f870b0 0x563a77f87490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a77dd9950 con 0x563a77f870b0 2024-03-30T13:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563a77d76990 con 0x563a77fd3dc0 2024-03-30T13:11:03.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563a77dd5900 con 0x563a77fd34a0 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77f870b0 0x563a77f87490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77f870b0 0x563a77f87490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59690/0 (socket says 172.21.15.43:59690) 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 -- 172.21.15.43:0/866005165 learned_addr learned my addr 172.21.15.43:0/866005165 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa97547f640 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fd3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa974c7e640 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77fd3dc0 0x563a77f86b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 -- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 msgr2=0x563a77fd3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fd3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 -- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77fd3dc0 msgr2=0x563a77f86b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77fd3dc0 0x563a77f86b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 -- 172.21.15.43:0/866005165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a77e1e100 con 0x563a77f870b0 2024-03-30T13:11:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa975c80640 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77f870b0 0x563a77f87490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9700cd1c0 tx=0x7fa968009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae997f8504a7497 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.324+0000 7fa95affd640 1 -- 172.21.15.43:0/866005165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96801a020 con 0x563a77f870b0 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa95affd640 1 -- 172.21.15.43:0/866005165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa968016870 con 0x563a77f870b0 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa95affd640 1 -- 172.21.15.43:0/866005165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968016c60 con 0x563a77f870b0 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/866005165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77f870b0 msgr2=0x563a77f87490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77f870b0 0x563a77f87490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa9700cd1c0 tx=0x7fa968009750 comp rx=0 tx=0).stop 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/866005165 shutdown_connections 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77f870b0 0x563a77f87490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77fd3dc0 0x563a77f86b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- 172.21.15.43:0/866005165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fd3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/866005165 >> 172.21.15.43:0/866005165 conn(0x563a77e71b80 msgr2=0x563a77fb6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/866005165 shutdown_connections 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/866005165 wait complete. 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- start start 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 Processor -- start 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77f8ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fb7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd3dc0 0x563a77fb7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a77d717b0 con 0x563a77fd3dc0 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563a77d973e0 con 0x563a77f870b0 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563a77dddf80 con 0x563a77fd34a0 2024-03-30T13:11:03.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 10 client.0 _ll_register_callbacks cb 0x563a77e3a0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:03.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd3dc0 0x563a77fb7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd3dc0 0x563a77fb7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59700/0 (socket says 172.21.15.43:59700) 2024-03-30T13:11:03.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 -- 172.21.15.43:0/2456535333 learned_addr learned my addr 172.21.15.43:0/2456535333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:03.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97547f640 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77f8ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa974c7e640 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fb7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 -- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 msgr2=0x563a77fb7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fb7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 -- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 msgr2=0x563a77f8ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77f8ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 -- 172.21.15.43:0/2456535333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a77da38c0 con 0x563a77fd3dc0 2024-03-30T13:11:03.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa975c80640 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd3dc0 0x563a77fb7bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa968009790 tx=0x7fa968004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a34eb31e7db1e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa94ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:11:03.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa94ffff640 1 -- 172.21.15.43:0/2456535333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9680209f0 con 0x563a77fd3dc0 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa94ffff640 1 -- 172.21.15.43:0/2456535333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa968020b90 con 0x563a77fd3dc0 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa968016870 con 0x563a77fd3dc0 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa94ffff640 1 -- 172.21.15.43:0/2456535333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa968020d90 con 0x563a77fd3dc0 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 10 client.4398 fetch_fsmap learned FSMap version 7 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563a77d61e80 con 0x563a77fd3dc0 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa94ffff640 1 -- 172.21.15.43:0/2456535333 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa968016870 con 0x563a77fd3dc0 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa94ffff640 1 -- 172.21.15.43:0/2456535333 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa968027070 con 0x563a77fd3dc0 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 10 client.4398 fetch_fsmap finished waiting for FSMap version 7 2024-03-30T13:11:03.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 10 client.4398 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 20 client.4398 populate_metadata read hostname 'smithi043' 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fe3540 0x563a77fe5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 4 client.4398 mds_command: new command op to 4239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 --> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a77d74bc0 con 0x563a77fe3540 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.328+0000 7fa97547f640 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fe3540 0x563a77fe5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97547f640 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fe3540 0x563a77fe5930 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e10f559fe74ac91 server_cookie=df93445bfc904c6a in_seq=0 out_seq=0 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa94ffff640 10 client.4398 ms_handle_connect on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa94ffff640 1 -- 172.21.15.43:0/2456535333 <== mds.0 v2:172.21.15.43:6834/855012184 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x563a77d74bc0 con 0x563a77fe3540 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa94ffff640 10 client.4398 handle_command_reply: tid=0 2024-03-30T13:11:03.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 client.4398 shutdown 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd3dc0 msgr2=0x563a77fb7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd3dc0 0x563a77fb7bd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa968009790 tx=0x7fa968004a50 comp rx=0 tx=0).stop 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 shutdown_connections 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fe3540 0x563a77fe5930 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd3dc0 0x563a77fb7bd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77fd34a0 0x563a77fb7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa94ffff640 0 client.4398 ms_handle_reset on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 --2- 172.21.15.43:0/2456535333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77f8ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 >> 172.21.15.43:0/2456535333 conn(0x563a77e71b80 msgr2=0x563a77f86270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 shutdown_connections 2024-03-30T13:11:03.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 1 -- 172.21.15.43:0/2456535333 wait complete. 2024-03-30T13:11:03.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.332+0000 7fa97a865000 20 client.4398 trim_cache size 0 max 16384 2024-03-30T13:11:03.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 Processor -- start 2024-03-30T13:11:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 -- start start 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a77fe5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77fe5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9702017b0 con 0x563a77fd34a0 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9702007f0 con 0x563a77f870b0 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa9701f42e0 con 0x563a77e71b80 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59712/0 (socket says 172.21.15.43:59712) 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa975c80640 1 -- 172.21.15.43:0/2617557120 learned_addr learned my addr 172.21.15.43:0/2617557120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:03.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa97547f640 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a77fe5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.348+0000 7fa974c7e640 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77fe5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 -- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 msgr2=0x563a77fe5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a77fe5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 -- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 msgr2=0x563a77fe5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77fe5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 -- 172.21.15.43:0/2617557120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a77d94490 con 0x563a77fd34a0 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97547f640 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a77fe5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa974c7e640 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77fe5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe63b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa968008010 tx=0x7fa968022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc462fd656fb748 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa95a7fc640 1 -- 172.21.15.43:0/2617557120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96801a040 con 0x563a77fd34a0 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa95a7fc640 1 -- 172.21.15.43:0/2617557120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9680209f0 con 0x563a77fd34a0 2024-03-30T13:11:03.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa95a7fc640 1 -- 172.21.15.43:0/2617557120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968020ba0 con 0x563a77fd34a0 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- 172.21.15.43:0/2617557120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 msgr2=0x563a77fe63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe63b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa968008010 tx=0x7fa968022040 comp rx=0 tx=0).stop 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- 172.21.15.43:0/2617557120 shutdown_connections 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe63b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a77fe5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 --2- 172.21.15.43:0/2617557120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a77fe5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- 172.21.15.43:0/2617557120 >> 172.21.15.43:0/2617557120 conn(0x563a77f8f100 msgr2=0x563a77fe0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- 172.21.15.43:0/2617557120 shutdown_connections 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- 172.21.15.43:0/2617557120 wait complete. 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- start start 2024-03-30T13:11:03.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 Processor -- start 2024-03-30T13:11:03.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a78045fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a78046520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563a77d7fe80 con 0x563a77fd34a0 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9702001a0 con 0x563a77f870b0 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563a77d7a830 con 0x563a77e71b80 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97a865000 10 client.0 _ll_register_callbacks cb 0x7fa968007330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59716/0 (socket says 172.21.15.43:59716) 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 -- 172.21.15.43:0/1095845041 learned_addr learned my addr 172.21.15.43:0/1095845041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa974c7e640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a78046520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa97547f640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a78045fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 -- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 msgr2=0x563a78045fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a78045fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 -- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 msgr2=0x563a78046520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a78046520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 -- 172.21.15.43:0/1095845041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a77d92e80 con 0x563a77fd34a0 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa974c7e640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a78046520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa975c80640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe72f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x563a77d78fe0 tx=0x7fa96800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26a5a12fed98d49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.352+0000 7fa94ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 1 -- 172.21.15.43:0/1095845041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96801a040 con 0x563a77fd34a0 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 1 -- 172.21.15.43:0/1095845041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa968029ad0 con 0x563a77fd34a0 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9680209f0 con 0x563a77fd34a0 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 1 -- 172.21.15.43:0/1095845041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7fa96804c050 con 0x563a77fd34a0 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 10 client.4404 fetch_fsmap learned FSMap version 7 2024-03-30T13:11:03.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x563a77d80270 con 0x563a77fd34a0 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 1 -- 172.21.15.43:0/1095845041 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9680209f0 con 0x563a77fd34a0 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 1 -- 172.21.15.43:0/1095845041 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7fa968027050 con 0x563a77fd34a0 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 10 client.4404 fetch_fsmap finished waiting for FSMap version 7 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 10 client.4404 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 20 client.4404 populate_metadata read hostname 'smithi043' 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fd3dc0 0x563a77fe7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 4 client.4404 mds_command: new command op to 4239 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 --> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] -- command(tid 0: {"prefix": "status"}) -- 0x563a77de6e90 con 0x563a77fd3dc0 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97547f640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fd3dc0 0x563a77fe7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 10 client.4404 ms_handle_connect on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97547f640 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fd3dc0 0x563a77fe7830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1814f09a2d2cf684 server_cookie=e00c67f09cc9d4f9 in_seq=0 out_seq=0 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 1 -- 172.21.15.43:0/1095845041 <== mds.0 v2:172.21.15.43:6834/855012184 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x563a77de6e90 con 0x563a77fd3dc0 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 10 client.4404 handle_command_reply: tid=0 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 client.4404 shutdown 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 msgr2=0x563a77fe72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe72f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x563a77d78fe0 tx=0x7fa96800b040 comp rx=0 tx=0).stop 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 shutdown_connections 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x563a77fd3dc0 0x563a77fe7830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a77fd34a0 0x563a77fe72f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563a77f870b0 0x563a78046520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa94ffff640 0 client.4404 ms_handle_reset on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:03.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 --2- 172.21.15.43:0/1095845041 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563a77e71b80 0x563a78045fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 >> 172.21.15.43:0/1095845041 conn(0x563a77f8f100 msgr2=0x563a77fe0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 shutdown_connections 2024-03-30T13:11:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 1 -- 172.21.15.43:0/1095845041 wait complete. 2024-03-30T13:11:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa97a865000 20 client.4404 trim_cache size 0 max 16384 2024-03-30T13:11:03.370 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:11:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "ace86ea7-ebbf-4c39-b9cf-d22b6fea8142", 2024-03-30T13:11:03.370 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4239, 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 7.4110184950000004, 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 7, 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 27, 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 27, 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 14.251687413999999 2024-03-30T13:11:03.371 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:11:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa954068160 msgr2=0x7fa95406a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa954068160 0x7fa95406a620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa9700b5fd0 tx=0x7fa9600011a0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97013b740 msgr2=0x7fa970151df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97013b740 0x7fa970151df0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00e780 tx=0x7fa96c00ec40 comp rx=0 tx=0).stop 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 shutdown_connections 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa954068160 0x7fa95406a620 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa97013bb20 0x7fa97014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9700d0230 0x7fa970152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 --2- 172.21.15.43:0/2671042535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa97013b740 0x7fa970151df0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 >> 172.21.15.43:0/2671042535 conn(0x7fa970072500 msgr2=0x7fa9700b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 shutdown_connections 2024-03-30T13:11:03.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.356+0000 7fa9776e3640 1 -- 172.21.15.43:0/2671042535 wait complete. 2024-03-30T13:11:03.379 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:11:03.379 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:03.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:11:03.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.488+0000 7f39d3c4d640 1 Processor -- start 2024-03-30T13:11:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.488+0000 7f39d3c4d640 1 -- start start 2024-03-30T13:11:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d3c4d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f39cc15c1c0 0x7f39cc0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d3c4d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 0x7f39cc0d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d3c4d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39cc0ccba0 con 0x7f39cc0d0900 2024-03-30T13:11:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d3c4d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f39cc0ce3a0 con 0x7f39cc15c1c0 2024-03-30T13:11:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d3c4d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc0cce80 con 0x7f39cc0b5060 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d11e8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f39cc0d0900 0x7f39cc0ce810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34634/0 (socket says 172.21.15.43:34634) 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d11e8640 1 -- 172.21.15.43:0/2549717487 learned_addr learned my addr 172.21.15.43:0/2549717487 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2549717487 >> v2:172.21.15.113:3300/0 conn(0x7f39cc15c1c0 0x7f39cc0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443203723 0 0) 0x7f39cc0ccba0 con 0x7f39cc0d0900 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39b8003530 con 0x7f39cc0d0900 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d21ea640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 0x7f39cc0d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2913828129 0 0) 0x7f39b8003530 con 0x7f39cc0d0900 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 msgr2=0x7f39cc0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 0x7f39cc0d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 >> v2:172.21.15.113:3300/0 conn(0x7f39cc15c1c0 msgr2=0x7f39cc0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 --2- 172.21.15.43:0/2549717487 >> v2:172.21.15.113:3300/0 conn(0x7f39cc15c1c0 0x7f39cc0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc0b26f0 con 0x7f39cc0d0900 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f39bc001680 con 0x7f39cc0d0900 2024-03-30T13:11:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 >> v1:172.21.15.43:6789/0 conn(0x7f39cc0d0900 legacy=0x7f39cc0ce810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b80039c0 0x7f39b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 0x7f39b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b8004d50 0x7f39b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0cce80 con 0x7f39b80039c0 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f39b800b9d0 con 0x7f39b8004d50 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f39b800bb50 con 0x7f39cc0b5060 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b80039c0 0x7f39b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d11e8640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 0x7f39b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 -- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 msgr2=0x7f39b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 0x7f39b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 -- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b8004d50 msgr2=0x7f39b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d21ea640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b8004d50 0x7f39b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b8004d50 0x7f39b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 -- 172.21.15.43:0/2549717487 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b8003530 con 0x7f39b80039c0 2024-03-30T13:11:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d11e8640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0b5060 0x7f39b8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d21ea640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b8004d50 0x7f39b80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.492+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b80039c0 0x7f39b8003de0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009b10 tx=0x7f39c801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c56c10975de04be server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80096b0 con 0x7f39b80039c0 2024-03-30T13:11:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c80a6490 con 0x7f39b80039c0 2024-03-30T13:11:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d09e7640 1 -- 172.21.15.43:0/2549717487 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80a6880 con 0x7f39b80039c0 2024-03-30T13:11:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2549717487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b80039c0 msgr2=0x7f39b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 --2- 172.21.15.43:0/2549717487 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b80039c0 0x7f39b8003de0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009b10 tx=0x7f39c801cab0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 -- 172.21.15.43:0/2549717487 reap_dead start 2024-03-30T13:11:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2549717487 shutdown_connections 2024-03-30T13:11:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2549717487 >> 172.21.15.43:0/2549717487 conn(0x7f39cc0b3d60 msgr2=0x7f39cc0b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2549717487 shutdown_connections 2024-03-30T13:11:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2549717487 wait complete. 2024-03-30T13:11:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 Processor -- start 2024-03-30T13:11:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- start start 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b80039c0 0x7f39cc14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0ca9b0 0x7f39cc0cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39cc0c9260 0x7f39cc0c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc167230 con 0x7f39cc0c9260 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f39cc0d06b0 con 0x7f39b80039c0 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f39cc1673b0 con 0x7f39cc0ca9b0 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39cc0c9260 0x7f39cc0c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39cc0c9260 0x7f39cc0c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59724/0 (socket says 172.21.15.43:59724) 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 -- 172.21.15.43:0/2108202284 learned_addr learned my addr 172.21.15.43:0/2108202284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b80039c0 0x7f39cc14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d11e8640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0ca9b0 0x7f39cc0cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 -- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0ca9b0 msgr2=0x7f39cc0cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0ca9b0 0x7f39cc0cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 -- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b80039c0 msgr2=0x7f39cc14ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b80039c0 0x7f39cc14ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 -- 172.21.15.43:0/2108202284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39cc1f7fc0 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d21ea640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39cc0c9260 0x7f39cc0c96e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0b54f0 tx=0x7f39c400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39c27fc640 1 -- 172.21.15.43:0/2108202284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c401c070 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39c27fc640 1 -- 172.21.15.43:0/2108202284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c4017400 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39c27fc640 1 -- 172.21.15.43:0/2108202284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40177d0 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39cc0c9e70 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.496+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39cc0ca430 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.500+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39cc156000 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.500+0000 7f39c27fc640 1 -- 172.21.15.43:0/2108202284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f39c401f070 con 0x7f39cc0c9260 2024-03-30T13:11:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.500+0000 7f39c27fc640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f39b0067ff0 0x7f39b006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.500+0000 7f39c27fc640 1 -- 172.21.15.43:0/2108202284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f39c4086df0 con 0x7f39cc0c9260 2024-03-30T13:11:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.500+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f39b0067ff0 0x7f39b006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.500+0000 7f39d19e9640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f39b0067ff0 0x7f39b006a4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39c80ad000 tx=0x7f39c80b2040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.504+0000 7f39c27fc640 1 -- 172.21.15.43:0/2108202284 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f39c4012070 con 0x7f39cc0c9260 2024-03-30T13:11:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.696+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39cc0b33c0 con 0x7f39cc0c9260 2024-03-30T13:11:03.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.696+0000 7f39c27fc640 1 -- 172.21.15.43:0/2108202284 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f39c4057340 con 0x7f39cc0c9260 2024-03-30T13:11:03.703 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:11:03.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f39b0067ff0 msgr2=0x7f39b006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f39b0067ff0 0x7f39b006a4b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f39c80ad000 tx=0x7f39c80b2040 comp rx=0 tx=0).stop 2024-03-30T13:11:03.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39cc0c9260 msgr2=0x7f39cc0c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:03.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39cc0c9260 0x7f39cc0c96e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0b54f0 tx=0x7f39c400f4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 shutdown_connections 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f39b0067ff0 0x7f39b006a4b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39cc0c9260 0x7f39cc0c96e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f39cc0ca9b0 0x7f39cc0cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 --2- 172.21.15.43:0/2108202284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f39b80039c0 0x7f39cc14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 >> 172.21.15.43:0/2108202284 conn(0x7f39cc0b3d60 msgr2=0x7f39cc06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 shutdown_connections 2024-03-30T13:11:03.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03.700+0000 7f39d3c4d640 1 -- 172.21.15.43:0/2108202284 wait complete. 2024-03-30T13:11:03.721 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-30T13:11:03.721 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:11:03.722 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-03-30T13:11:03.722 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-03-30T13:11:03.722 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:03.722 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-30T13:11:03.722 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:11:03.722 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:11:03.722 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T13:11:03.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:03.723 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:03.729 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:03.730 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-03-30T13:11:03.731 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T13:11:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:bf:2f:c7:16 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T13:11:03.731 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:03.731 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T13:11:03.732 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:03.732 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:03.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-03-30T13:11:03.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-03-30T13:11:03.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-03-30T13:11:03.732 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T13:11:03.732 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:03.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:03.857 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:11:03.909 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-03-30T13:11:03.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:03.910 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-03-30T13:11:03.973 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-03-30T13:11:03.973 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:11:03.973 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-30T13:11:03.973 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T13:11:03.973 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-30T13:11:03.973 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:11:03.973 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:03.974 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:03.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T13:11:03.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T13:11:03.974 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T13:11:03.974 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:04.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:04.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:04.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:04.098 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-30T13:11:04.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:04.145 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-03-30T13:11:04.193 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:04.193 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:04.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:04.193 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-03-30T13:11:04.194 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:04.306 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-03-30T13:11:04.306 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:04.306 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:04.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-03-30T13:11:04.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T13:11:04.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-03-30T13:11:04.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-03-30T13:11:04.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-03-30T13:11:04.307 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:04.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:04.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:04.415 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:04.415 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:04.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-03-30T13:11:04.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T13:11:04.415 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:04.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:04.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:04.519 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-03-30T13:11:04.520 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:04.520 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:04.568 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-03-30T13:11:04.568 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:04.568 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:04.617 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:04.678 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:04.726 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:04.727 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:04.728 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:04.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:04.729 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:04.777 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T13:11:04.777 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-03-30T13:11:04.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:04.835 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:04.881 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:04.881 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:04.881 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:04.881 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:04.881 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:04.881 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:04.881 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:04.883 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:04.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:04.884 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:04.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:2024-03-30T13:11:04.896+0000 7f5d9fc28580 -1 init, newargv = 0x55fcba656960 newargc=15 2024-03-30T13:11:04.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[33458]: starting ceph client 2024-03-30T13:11:04.909 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[33458]: starting fuse 2024-03-30T13:11:06.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:06.022 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:06.070 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:06.073 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:06.073 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:06.073 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:06.073 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:06.073 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:06.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:06.074 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:06.125 INFO:teuthology.orchestra.run.smithi043.stdout:45 2024-03-30T13:11:06.126 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-03-30T13:11:06.126 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-03-30T13:11:06.127 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:11:06.128 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:06.128 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.admin")) 2024-03-30T13:11:06.128 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-03-30T13:11:06.264 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.admin.33458.asok 2024-03-30T13:11:06.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:06.277 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33458.asok 2024-03-30T13:11:06.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:06.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33458.asok status 2024-03-30T13:11:06.437 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "admin", 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "33458", 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-03-30T13:11:06.438 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4419, 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4419 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 666357254 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:11:06.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 666357254 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4419 192.168.0.1:0/666357254", 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/666357254", 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 7, 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 27, 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-03-30T13:11:06.440 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:11:06.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:06.452 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-03-30T13:11:06.452 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:06.458 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-03-30T13:11:06.458 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:06.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:06.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:06.512 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-03-30T13:11:06.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-30T13:11:06.638 INFO:teuthology.orchestra.run.smithi043.stdout:# file: . 2024-03-30T13:11:06.639 INFO:teuthology.orchestra.run.smithi043.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-03-30T13:11:06.639 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:06.647 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:11:06.647 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:06.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94ce45640 1 Processor -- start 2024-03-30T13:11:06.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94ce45640 1 -- start start 2024-03-30T13:11:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94ce45640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff94815c0a0 0x7ff94806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94ce45640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9480cdc10 con 0x7ff948141990 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94ce45640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9480cf060 con 0x7ff94806bba0 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94ce45640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff9480cf670 con 0x7ff94815c0a0 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff94815c0a0 0x7ff94806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff94815c0a0 0x7ff94806b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59728/0 (socket says 172.21.15.43:59728) 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94659c640 1 -- 172.21.15.43:0/4099260392 learned_addr learned my addr 172.21.15.43:0/4099260392 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94659c640 1 -- 172.21.15.43:0/4099260392 >> v1:172.21.15.113:6789/0 conn(0x7ff94806bba0 legacy=0x7ff9480cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94659c640 1 -- 172.21.15.43:0/4099260392 >> v1:172.21.15.43:6789/0 conn(0x7ff948141990 legacy=0x7ff9480cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff94659c640 1 -- 172.21.15.43:0/4099260392 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9480cd650 con 0x7ff94815c0a0 2024-03-30T13:11:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.820+0000 7ff945d9b640 1 -- 172.21.15.43:0/4099260392 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1217926620 0 0) 0x7ff9480cdc10 con 0x7ff948141990 2024-03-30T13:11:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94659c640 1 --2- 172.21.15.43:0/4099260392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff94806b660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9480cf7f0 tx=0x7ff93801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1773dde31243594 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff945d9b640 1 -- 172.21.15.43:0/4099260392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380ab620 con 0x7ff94815c0a0 2024-03-30T13:11:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff945d9b640 1 -- 172.21.15.43:0/4099260392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9380b0070 con 0x7ff94815c0a0 2024-03-30T13:11:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff945d9b640 1 -- 172.21.15.43:0/4099260392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380ab9e0 con 0x7ff94815c0a0 2024-03-30T13:11:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 -- 172.21.15.43:0/4099260392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 msgr2=0x7ff94806b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4099260392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff94806b660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff9480cf7f0 tx=0x7ff93801c510 comp rx=0 tx=0).stop 2024-03-30T13:11:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 -- 172.21.15.43:0/4099260392 shutdown_connections 2024-03-30T13:11:06.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4099260392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff94806b660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 -- 172.21.15.43:0/4099260392 >> 172.21.15.43:0/4099260392 conn(0x7ff9480b2980 msgr2=0x7ff9480d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 -- 172.21.15.43:0/4099260392 shutdown_connections 2024-03-30T13:11:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 -- 172.21.15.43:0/4099260392 wait complete. 2024-03-30T13:11:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 Processor -- start 2024-03-30T13:11:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.824+0000 7ff94ce45640 1 -- start start 2024-03-30T13:11:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff94806bba0 0x7ff9481556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff948141990 0x7ff94813b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff9481396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff94815c600 con 0x7ff94815c0a0 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff94815c480 con 0x7ff94806bba0 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff9480d0210 con 0x7ff948141990 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff9481396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff9481396c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59734/0 (socket says 172.21.15.43:59734) 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 -- 172.21.15.43:0/4180806344 learned_addr learned my addr 172.21.15.43:0/4180806344 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94659c640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff94806bba0 0x7ff9481556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff93dd9b640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff948141990 0x7ff94813b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 -- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff948141990 msgr2=0x7ff94813b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff948141990 0x7ff94813b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 -- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff94806bba0 msgr2=0x7ff9481556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff94806bba0 0x7ff9481556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:06.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 -- 172.21.15.43:0/4180806344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff948139010 con 0x7ff94815c0a0 2024-03-30T13:11:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff946d9d640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff9481396c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c780 tx=0x7ff94000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff93f7fe640 1 -- 172.21.15.43:0/4180806344 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940003db0 con 0x7ff94815c0a0 2024-03-30T13:11:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff93f7fe640 1 -- 172.21.15.43:0/4180806344 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff94000ce30 con 0x7ff94815c0a0 2024-03-30T13:11:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94813a230 con 0x7ff94815c0a0 2024-03-30T13:11:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff93f7fe640 1 -- 172.21.15.43:0/4180806344 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940005290 con 0x7ff94815c0a0 2024-03-30T13:11:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94813a7f0 con 0x7ff94815c0a0 2024-03-30T13:11:06.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.828+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff948068890 con 0x7ff94815c0a0 2024-03-30T13:11:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.832+0000 7ff93f7fe640 1 -- 172.21.15.43:0/4180806344 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff940005430 con 0x7ff94815c0a0 2024-03-30T13:11:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.832+0000 7ff93f7fe640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff928067fc0 0x7ff92806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.832+0000 7ff93f7fe640 1 -- 172.21.15.43:0/4180806344 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff940087230 con 0x7ff94815c0a0 2024-03-30T13:11:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.832+0000 7ff94659c640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff928067fc0 0x7ff92806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:06.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.832+0000 7ff94659c640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff928067fc0 0x7ff92806a480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9380af000 tx=0x7ff9380a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:06.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:06.836+0000 7ff93f7fe640 1 -- 172.21.15.43:0/4180806344 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff940057690 con 0x7ff94815c0a0 2024-03-30T13:11:07.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.040+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff948145c20 con 0x7ff94815c0a0 2024-03-30T13:11:07.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.040+0000 7ff93f7fe640 1 -- 172.21.15.43:0/4180806344 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7ff940056f90 con 0x7ff94815c0a0 2024-03-30T13:11:07.049 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:07.049 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:07.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff928067fc0 msgr2=0x7ff92806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff928067fc0 0x7ff92806a480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9380af000 tx=0x7ff9380a7000 comp rx=0 tx=0).stop 2024-03-30T13:11:07.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 msgr2=0x7ff9481396c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff9481396c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c780 tx=0x7ff94000cc40 comp rx=0 tx=0).stop 2024-03-30T13:11:07.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 shutdown_connections 2024-03-30T13:11:07.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff928067fc0 0x7ff92806a480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff94815c0a0 0x7ff9481396c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff948141990 0x7ff94813b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 --2- 172.21.15.43:0/4180806344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff94806bba0 0x7ff9481556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 >> 172.21.15.43:0/4180806344 conn(0x7ff9480b2980 msgr2=0x7ff94806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 shutdown_connections 2024-03-30T13:11:07.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.044+0000 7ff94ce45640 1 -- 172.21.15.43:0/4180806344 wait complete. 2024-03-30T13:11:07.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:07.069 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc2483640 1 Processor -- start 2024-03-30T13:11:07.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc2483640 1 -- start start 2024-03-30T13:11:07.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc2483640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cbc06dee0 0x7f9cbc15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc2483640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9cbc06f060 0x7f9cbc15f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc2483640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cbc06f5d0 con 0x7f9cbc06db00 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc2483640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9cbc06f8b0 con 0x7f9cbc06dee0 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc2483640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9cbc06fb90 con 0x7f9cbc06f060 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cbc06dee0 0x7f9cbc15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cbc06dee0 0x7f9cbc15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59738/0 (socket says 172.21.15.43:59738) 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9cbc06db00 0x7f9cbc0cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34646/0 (socket says 172.21.15.43:34646) 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbb7fe640 1 -- 172.21.15.43:0/2566048141 learned_addr learned my addr 172.21.15.43:0/2566048141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cc0a20640 1 --2- 172.21.15.43:0/2566048141 >> v2:172.21.15.113:3300/0 conn(0x7f9cbc06f060 0x7f9cbc15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbb7fe640 1 -- 172.21.15.43:0/2566048141 >> v2:172.21.15.113:3300/0 conn(0x7f9cbc06f060 msgr2=0x7f9cbc15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbb7fe640 1 --2- 172.21.15.43:0/2566048141 >> v2:172.21.15.113:3300/0 conn(0x7f9cbc06f060 0x7f9cbc15f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbb7fe640 1 -- 172.21.15.43:0/2566048141 >> v1:172.21.15.43:6789/0 conn(0x7f9cbc06db00 legacy=0x7f9cbc0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbaffd640 1 -- 172.21.15.43:0/2566048141 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109188613 0 0) 0x7f9cbc06f5d0 con 0x7f9cbc06db00 2024-03-30T13:11:07.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.195+0000 7f9cbb7fe640 1 -- 172.21.15.43:0/2566048141 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc06fe70 con 0x7f9cbc06dee0 2024-03-30T13:11:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cbb7fe640 1 --2- 172.21.15.43:0/2566048141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06dee0 0x7f9cbc15ecc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0cb990 tx=0x7f9cb001ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850afd9f94f8429e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cbaffd640 1 -- 172.21.15.43:0/2566048141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00b4070 con 0x7f9cbc06dee0 2024-03-30T13:11:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cbaffd640 1 -- 172.21.15.43:0/2566048141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb00a7cb0 con 0x7f9cbc06dee0 2024-03-30T13:11:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cbaffd640 1 -- 172.21.15.43:0/2566048141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00a7e60 con 0x7f9cbc06dee0 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- 172.21.15.43:0/2566048141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06dee0 msgr2=0x7f9cbc15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2566048141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06dee0 0x7f9cbc15ecc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0cb990 tx=0x7f9cb001ebc0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- 172.21.15.43:0/2566048141 shutdown_connections 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2566048141 >> v2:172.21.15.113:3300/0 conn(0x7f9cbc06f060 0x7f9cbc15f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2566048141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06dee0 0x7f9cbc15ecc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- 172.21.15.43:0/2566048141 >> 172.21.15.43:0/2566048141 conn(0x7f9cbc0b2c70 msgr2=0x7f9cbc0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- 172.21.15.43:0/2566048141 shutdown_connections 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- 172.21.15.43:0/2566048141 wait complete. 2024-03-30T13:11:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 Processor -- start 2024-03-30T13:11:07.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- start start 2024-03-30T13:11:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cbc06db00 0x7f9cbc14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cbc06dee0 0x7f9cbc0b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06f060 0x7f9cbc0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc06c200 con 0x7f9cbc06f060 2024-03-30T13:11:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9cbc06c080 con 0x7f9cbc06dee0 2024-03-30T13:11:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc2483640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9cbc06c380 con 0x7f9cbc06db00 2024-03-30T13:11:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06f060 0x7f9cbc0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06f060 0x7f9cbc0b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59740/0 (socket says 172.21.15.43:59740) 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 -- 172.21.15.43:0/2739268600 learned_addr learned my addr 172.21.15.43:0/2739268600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cbbfff640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cbc06db00 0x7f9cbc14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cbb7fe640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cbc06dee0 0x7f9cbc0b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 -- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cbc06db00 msgr2=0x7f9cbc14eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cbc06db00 0x7f9cbc14eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 -- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cbc06dee0 msgr2=0x7f9cbc0b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cbc06dee0 0x7f9cbc0b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 -- 172.21.15.43:0/2739268600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cbc14f2e0 con 0x7f9cbc06f060 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.199+0000 7f9cc0a20640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06f060 0x7f9cbc0b4210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4007e70 tx=0x7f9cb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cb8ff9640 1 -- 172.21.15.43:0/2739268600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4014620 con 0x7f9cbc06f060 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cb8ff9640 1 -- 172.21.15.43:0/2739268600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb4019070 con 0x7f9cbc06f060 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cb8ff9640 1 -- 172.21.15.43:0/2739268600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb4014a20 con 0x7f9cbc06f060 2024-03-30T13:11:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cbc15aa70 con 0x7f9cbc06f060 2024-03-30T13:11:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cbc15aff0 con 0x7f9cbc06f060 2024-03-30T13:11:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cbc149860 con 0x7f9cbc06f060 2024-03-30T13:11:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cb8ff9640 1 -- 172.21.15.43:0/2739268600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9cb4014bc0 con 0x7f9cbc06f060 2024-03-30T13:11:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cb8ff9640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ca8067fc0 0x7f9ca806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cb8ff9640 1 -- 172.21.15.43:0/2739268600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9cb4087170 con 0x7f9cbc06f060 2024-03-30T13:11:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cbbfff640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ca8067fc0 0x7f9ca806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.203+0000 7f9cbbfff640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ca8067fc0 0x7f9ca806a480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9cac000f70 tx=0x7f9cac007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.207+0000 7f9cb8ff9640 1 -- 172.21.15.43:0/2739268600 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9cb4011260 con 0x7f9cbc06f060 2024-03-30T13:11:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.379+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9cbc068900 con 0x7f9cbc06f060 2024-03-30T13:11:07.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.379+0000 7f9cb8ff9640 1 -- 172.21.15.43:0/2739268600 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9cb40575e0 con 0x7f9cbc06f060 2024-03-30T13:11:07.387 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ca8067fc0 msgr2=0x7f9ca806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ca8067fc0 0x7f9ca806a480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9cac000f70 tx=0x7f9cac007c10 comp rx=0 tx=0).stop 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06f060 msgr2=0x7f9cbc0b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06f060 0x7f9cbc0b4210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4007e70 tx=0x7f9cb400e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 shutdown_connections 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ca8067fc0 0x7f9ca806a480 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc06f060 0x7f9cbc0b4210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cbc06dee0 0x7f9cbc0b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 --2- 172.21.15.43:0/2739268600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cbc06db00 0x7f9cbc14eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 >> 172.21.15.43:0/2739268600 conn(0x7f9cbc0b2c70 msgr2=0x7f9cbc151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 shutdown_connections 2024-03-30T13:11:07.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.383+0000 7f9cc2483640 1 -- 172.21.15.43:0/2739268600 wait complete. 2024-03-30T13:11:07.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a96e3640 1 Processor -- start 2024-03-30T13:11:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a96e3640 1 -- start start 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a96e3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70a41450f0 0x7f70a415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a96e3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a40d08e0 con 0x7f70a4160440 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a96e3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a40d0b50 con 0x7f70a40d04a0 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a96e3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f70a40d0e30 con 0x7f70a41450f0 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70a41450f0 0x7f70a415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70a41450f0 0x7f70a415ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59742/0 (socket says 172.21.15.43:59742) 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a2ffd640 1 -- 172.21.15.43:0/3524166694 learned_addr learned my addr 172.21.15.43:0/3524166694 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a27fc640 1 --1- 172.21.15.43:0/3524166694 >> v1:172.21.15.43:6789/0 conn(0x7f70a4160440 0x7f70a40cfd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34660/0 (socket says 172.21.15.43:34660) 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a2ffd640 1 -- 172.21.15.43:0/3524166694 >> v1:172.21.15.113:6789/0 conn(0x7f70a40d04a0 legacy=0x7f70a406c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a2ffd640 1 -- 172.21.15.43:0/3524166694 >> v1:172.21.15.43:6789/0 conn(0x7f70a4160440 legacy=0x7f70a40cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.539+0000 7f70a2ffd640 1 -- 172.21.15.43:0/3524166694 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a406b700 con 0x7f70a41450f0 2024-03-30T13:11:07.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a2ffd640 1 --2- 172.21.15.43:0/3524166694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a415ff00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f709c009e80 tx=0x7f709c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30041d4dfc301719 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a1ffb640 1 -- 172.21.15.43:0/3524166694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a5780 con 0x7f70a41450f0 2024-03-30T13:11:07.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a1ffb640 1 -- 172.21.15.43:0/3524166694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f709c0a5920 con 0x7f70a41450f0 2024-03-30T13:11:07.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a1ffb640 1 -- 172.21.15.43:0/3524166694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a5cf0 con 0x7f70a41450f0 2024-03-30T13:11:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- 172.21.15.43:0/3524166694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 msgr2=0x7f70a415ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 --2- 172.21.15.43:0/3524166694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a415ff00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f709c009e80 tx=0x7f709c01c9d0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- 172.21.15.43:0/3524166694 shutdown_connections 2024-03-30T13:11:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 --2- 172.21.15.43:0/3524166694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a415ff00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- 172.21.15.43:0/3524166694 >> 172.21.15.43:0/3524166694 conn(0x7f70a40d1070 msgr2=0x7f70a40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- 172.21.15.43:0/3524166694 shutdown_connections 2024-03-30T13:11:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- 172.21.15.43:0/3524166694 wait complete. 2024-03-30T13:11:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 Processor -- start 2024-03-30T13:11:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- start start 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70a40d04a0 0x7f70a415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a4136430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a4160440 0x7f70a4143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f70a406d280 con 0x7f70a41450f0 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f70a406d100 con 0x7f70a4160440 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a96e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f70a406d400 con 0x7f70a40d04a0 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a4136430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a4136430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59750/0 (socket says 172.21.15.43:59750) 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 -- 172.21.15.43:0/4294189521 learned_addr learned my addr 172.21.15.43:0/4294189521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a37fe640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a4160440 0x7f70a4143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a2ffd640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70a40d04a0 0x7f70a415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 -- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70a40d04a0 msgr2=0x7f70a415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70a40d04a0 0x7f70a415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 -- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a4160440 msgr2=0x7f70a4143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a4160440 0x7f70a4143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a27fc640 1 -- 172.21.15.43:0/4294189521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a41609e0 con 0x7f70a41450f0 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a37fe640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a4160440 0x7f70a4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.543+0000 7f70a2ffd640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70a40d04a0 0x7f70a415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f70a27fc640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a4136430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f708c00c370 tx=0x7f708c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f709b7fe640 1 -- 172.21.15.43:0/4294189521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0074e0 con 0x7f70a41450f0 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f709b7fe640 1 -- 172.21.15.43:0/4294189521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f708c007680 con 0x7f70a41450f0 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a4138e50 con 0x7f70a41450f0 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f709b7fe640 1 -- 172.21.15.43:0/4294189521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c004a30 con 0x7f70a41450f0 2024-03-30T13:11:07.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a4139410 con 0x7f70a41450f0 2024-03-30T13:11:07.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a41395e0 con 0x7f70a41450f0 2024-03-30T13:11:07.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f709b7fe640 1 -- 172.21.15.43:0/4294189521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f708c01a020 con 0x7f70a41450f0 2024-03-30T13:11:07.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f709b7fe640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7084067ff0 0x7f708406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f709b7fe640 1 -- 172.21.15.43:0/4294189521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f708c085f80 con 0x7f70a41450f0 2024-03-30T13:11:07.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f70a2ffd640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7084067ff0 0x7f708406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.547+0000 7f70a2ffd640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7084067ff0 0x7f708406a4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f709c0ae000 tx=0x7f709c0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.551+0000 7f709b7fe640 1 -- 172.21.15.43:0/4294189521 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f70a41395e0 con 0x7f70a41450f0 2024-03-30T13:11:07.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.711+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70a406c360 con 0x7f70a41450f0 2024-03-30T13:11:07.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.711+0000 7f709b7fe640 1 -- 172.21.15.43:0/4294189521 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f708c0563f0 con 0x7f70a41450f0 2024-03-30T13:11:07.720 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7084067ff0 msgr2=0x7f708406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7084067ff0 0x7f708406a4b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f709c0ae000 tx=0x7f709c0a7070 comp rx=0 tx=0).stop 2024-03-30T13:11:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 msgr2=0x7f70a4136430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a4136430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f708c00c370 tx=0x7f708c00c830 comp rx=0 tx=0).stop 2024-03-30T13:11:07.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 shutdown_connections 2024-03-30T13:11:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7084067ff0 0x7f708406a4b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f70a4160440 0x7f70a4143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70a41450f0 0x7f70a4136430 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 --2- 172.21.15.43:0/4294189521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f70a40d04a0 0x7f70a415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 >> 172.21.15.43:0/4294189521 conn(0x7f70a40d1070 msgr2=0x7f70a415f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 shutdown_connections 2024-03-30T13:11:07.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.715+0000 7f70a96e3640 1 -- 172.21.15.43:0/4294189521 wait complete. 2024-03-30T13:11:07.737 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88ca67a640 1 Processor -- start 2024-03-30T13:11:07.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88ca67a640 1 -- start start 2024-03-30T13:11:07.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88ca67a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88c406b660 0x7f88c415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88ca67a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c406f9e0 0x7f88c406fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88ca67a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88c406baf0 con 0x7f88c415c390 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88ca67a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f88c40cfcf0 con 0x7f88c406b660 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88ca67a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f88c406bdd0 con 0x7f88c406f9e0 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88c406b660 0x7f88c415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f88c406b660 0x7f88c415c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59760/0 (socket says 172.21.15.43:59760) 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c37fe640 1 -- 172.21.15.43:0/3134939128 learned_addr learned my addr 172.21.15.43:0/3134939128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c8c17640 1 --2- 172.21.15.43:0/3134939128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c406f9e0 0x7f88c406fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c37fe640 1 -- 172.21.15.43:0/3134939128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c406f9e0 msgr2=0x7f88c406fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c37fe640 1 --2- 172.21.15.43:0/3134939128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c406f9e0 0x7f88c406fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c37fe640 1 -- 172.21.15.43:0/3134939128 >> v1:172.21.15.113:6789/0 conn(0x7f88c415c390 legacy=0x7f88c40d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:07.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.843+0000 7f88c37fe640 1 -- 172.21.15.43:0/3134939128 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c406c0b0 con 0x7f88c406b660 2024-03-30T13:11:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88c37fe640 1 --2- 172.21.15.43:0/3134939128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c406b660 0x7f88c415c770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009870 tx=0x7f88b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbfe55c57d85902 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88c2ffd640 1 -- 172.21.15.43:0/3134939128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b1070 con 0x7f88c406b660 2024-03-30T13:11:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88c2ffd640 1 -- 172.21.15.43:0/3134939128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b80ac630 con 0x7f88c406b660 2024-03-30T13:11:07.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88c2ffd640 1 -- 172.21.15.43:0/3134939128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80a8050 con 0x7f88c406b660 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 -- 172.21.15.43:0/3134939128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c406b660 msgr2=0x7f88c415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 --2- 172.21.15.43:0/3134939128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c406b660 0x7f88c415c770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f88b8009870 tx=0x7f88b801c520 comp rx=0 tx=0).stop 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 -- 172.21.15.43:0/3134939128 shutdown_connections 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 --2- 172.21.15.43:0/3134939128 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c406f9e0 0x7f88c406fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 --2- 172.21.15.43:0/3134939128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c406b660 0x7f88c415c770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 -- 172.21.15.43:0/3134939128 >> 172.21.15.43:0/3134939128 conn(0x7f88c40d1070 msgr2=0x7f88c40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 -- 172.21.15.43:0/3134939128 shutdown_connections 2024-03-30T13:11:07.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.847+0000 7f88ca67a640 1 -- 172.21.15.43:0/3134939128 wait complete. 2024-03-30T13:11:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 Processor -- start 2024-03-30T13:11:07.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 -- start start 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c406f9e0 0x7f88c4146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c415c390 0x7f88c415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c415af20 0x7f88c4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f88c406d520 con 0x7f88c415c390 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f88c406d3a0 con 0x7f88c406f9e0 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f88c406d6a0 con 0x7f88c415af20 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c415c390 0x7f88c415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c415c390 0x7f88c415a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59764/0 (socket says 172.21.15.43:59764) 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 -- 172.21.15.43:0/1781152865 learned_addr learned my addr 172.21.15.43:0/1781152865 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 -- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c415af20 msgr2=0x7f88c4155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c415af20 0x7f88c4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 -- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c406f9e0 msgr2=0x7f88c4146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c406f9e0 0x7f88c4146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 -- 172.21.15.43:0/1781152865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c41495d0 con 0x7f88c415c390 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c37fe640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c415c390 0x7f88c415a9e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f88b80af040 tx=0x7f88b80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c0ff9640 1 -- 172.21.15.43:0/1781152865 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80bc070 con 0x7f88c415c390 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c0ff9640 1 -- 172.21.15.43:0/1781152865 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b80b7430 con 0x7f88c415c390 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c41568a0 con 0x7f88c415c390 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88c0ff9640 1 -- 172.21.15.43:0/1781152865 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b80b1040 con 0x7f88c415c390 2024-03-30T13:11:07.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c4156e60 con 0x7f88c415c390 2024-03-30T13:11:07.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.851+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c4068900 con 0x7f88c415c390 2024-03-30T13:11:07.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.855+0000 7f88c0ff9640 1 -- 172.21.15.43:0/1781152865 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f88b80bf070 con 0x7f88c415c390 2024-03-30T13:11:07.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.855+0000 7f88c0ff9640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f88b0067fc0 0x7f88b006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:07.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.855+0000 7f88c0ff9640 1 -- 172.21.15.43:0/1781152865 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f88b811ff00 con 0x7f88c415c390 2024-03-30T13:11:07.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.855+0000 7f88c3fff640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f88b0067fc0 0x7f88b006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:07.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.855+0000 7f88c3fff640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f88b0067fc0 0x7f88b006a480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003cf0 tx=0x7f88b40079c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:07.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:07.859+0000 7f88c0ff9640 1 -- 172.21.15.43:0/1781152865 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f88b8016110 con 0x7f88c415c390 2024-03-30T13:11:08.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.019+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88c40d1f80 con 0x7f88c415c390 2024-03-30T13:11:08.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.019+0000 7f88c0ff9640 1 -- 172.21.15.43:0/1781152865 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f88b80f0400 con 0x7f88c415c390 2024-03-30T13:11:08.028 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f88b0067fc0 msgr2=0x7f88b006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f88b0067fc0 0x7f88b006a480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88b4003cf0 tx=0x7f88b40079c0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c415c390 msgr2=0x7f88c415a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c415c390 0x7f88c415a9e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f88b80af040 tx=0x7f88b80192c0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 shutdown_connections 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f88b0067fc0 0x7f88b006a480 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f88c415af20 0x7f88c4155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88c415c390 0x7f88c415a9e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 --2- 172.21.15.43:0/1781152865 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f88c406f9e0 0x7f88c4146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 >> 172.21.15.43:0/1781152865 conn(0x7f88c40d1070 msgr2=0x7f88c406da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 shutdown_connections 2024-03-30T13:11:08.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.023+0000 7f88ca67a640 1 -- 172.21.15.43:0/1781152865 wait complete. 2024-03-30T13:11:08.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:08.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef84ce640 1 Processor -- start 2024-03-30T13:11:08.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef84ce640 1 -- start start 2024-03-30T13:11:08.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef84ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcef00cf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef84ce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef006ec80 con 0x7fcef00cbe80 2024-03-30T13:11:08.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef84ce640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef006ef60 con 0x7fcef006e780 2024-03-30T13:11:08.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef84ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcef006f240 con 0x7fcef006fd20 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef626a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcef00cbe80 0x7fcef00cc260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34672/0 (socket says 172.21.15.43:34672) 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef626a640 1 -- 172.21.15.43:0/4015528537 learned_addr learned my addr 172.21.15.43:0/4015528537 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774238948 0 0) 0x7fcef006ec80 con 0x7fcef00cbe80 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef6a6b640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcef00cf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcedc003530 con 0x7fcef00cbe80 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3656262033 0 0) 0x7fcedc003530 con 0x7fcef00cbe80 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 msgr2=0x7fcef00cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcef00cf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 >> v1:172.21.15.113:6789/0 conn(0x7fcef006e780 legacy=0x7fcef015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef006f520 con 0x7fcef00cbe80 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fcee0001680 con 0x7fcef00cbe80 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef6a6b640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcef00cf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 >> v1:172.21.15.43:6789/0 conn(0x7fcef00cbe80 legacy=0x7fcef00cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc0039f0 0x7fcedc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.147+0000 7fcef5268640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcedc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5268640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc004d50 0x7fcedc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcef006f240 con 0x7fcedc0039f0 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc00bb10 con 0x7fcedc004d50 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc00bd10 con 0x7fcef006fd20 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef626a640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc0039f0 0x7fcedc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef6a6b640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc004d50 0x7fcedc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5a69640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcedc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef626a640 1 -- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 msgr2=0x7fcedc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef626a640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcedc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef626a640 1 -- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc004d50 msgr2=0x7fcedc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef626a640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcedc004d50 0x7fcedc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef626a640 1 -- 172.21.15.43:0/4015528537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc003530 con 0x7fcedc0039f0 2024-03-30T13:11:08.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5a69640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcef006fd20 0x7fcedc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef626a640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc0039f0 0x7fcedc003e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcee0001ec0 tx=0x7fcee001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc6c996b4a0a5f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0007d40 con 0x7fcedc0039f0 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee00a9070 con 0x7fcedc0039f0 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef5268640 1 -- 172.21.15.43:0/4015528537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee0005d40 con 0x7fcedc0039f0 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef84ce640 1 -- 172.21.15.43:0/4015528537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc0039f0 msgr2=0x7fcedc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef84ce640 1 --2- 172.21.15.43:0/4015528537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcedc0039f0 0x7fcedc003e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcee0001ec0 tx=0x7fcee001cba0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef6a6b640 1 -- 172.21.15.43:0/4015528537 reap_dead start 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef84ce640 1 -- 172.21.15.43:0/4015528537 shutdown_connections 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef84ce640 1 -- 172.21.15.43:0/4015528537 >> 172.21.15.43:0/4015528537 conn(0x7fcef00cdac0 msgr2=0x7fcef00cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef84ce640 1 -- 172.21.15.43:0/4015528537 shutdown_connections 2024-03-30T13:11:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.151+0000 7fcef84ce640 1 -- 172.21.15.43:0/4015528537 wait complete. 2024-03-30T13:11:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 Processor -- start 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 -- start start 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcedc0039f0 0x7fcef00b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef01f55b0 0x7fcef015ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef01f5a30 0x7fcef01604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcef00d0020 con 0x7fcef01f55b0 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcef00cfea0 con 0x7fcef01f5a30 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcef00d01a0 con 0x7fcedc0039f0 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef01f55b0 0x7fcef015ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef01f55b0 0x7fcef015ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59784/0 (socket says 172.21.15.43:59784) 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 -- 172.21.15.43:0/1455669375 learned_addr learned my addr 172.21.15.43:0/1455669375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef626a640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcedc0039f0 0x7fcef00b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef6a6b640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef01f5a30 0x7fcef01604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 -- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcedc0039f0 msgr2=0x7fcef00b52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcedc0039f0 0x7fcef00b52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 -- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef01f5a30 msgr2=0x7fcef01604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef01f5a30 0x7fcef01604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 -- 172.21.15.43:0/1455669375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef013df20 con 0x7fcef01f55b0 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef6a6b640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef01f5a30 0x7fcef01604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef5a69640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef01f55b0 0x7fcef015ff70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fceec000c10 tx=0x7fceec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcee6ffd640 1 -- 172.21.15.43:0/1455669375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec019070 con 0x7fcef01f55b0 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcee6ffd640 1 -- 172.21.15.43:0/1455669375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fceec005270 con 0x7fcef01f55b0 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcee6ffd640 1 -- 172.21.15.43:0/1455669375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceec007030 con 0x7fcef01f55b0 2024-03-30T13:11:08.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef013e200 con 0x7fcef01f55b0 2024-03-30T13:11:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef013e7c0 con 0x7fcef01f55b0 2024-03-30T13:11:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.155+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef0068e60 con 0x7fcef01f55b0 2024-03-30T13:11:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.159+0000 7fcee6ffd640 1 -- 172.21.15.43:0/1455669375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fceec01c070 con 0x7fcef01f55b0 2024-03-30T13:11:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.159+0000 7fcee6ffd640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fced4067ff0 0x7fced406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.159+0000 7fcee6ffd640 1 -- 172.21.15.43:0/1455669375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fceec085f80 con 0x7fcef01f55b0 2024-03-30T13:11:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.159+0000 7fcef626a640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fced4067ff0 0x7fced406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.159+0000 7fcef626a640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fced4067ff0 0x7fced406a4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcee00ab000 tx=0x7fcee00b1040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.163+0000 7fcee6ffd640 1 -- 172.21.15.43:0/1455669375 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fceec055da0 con 0x7fcef01f55b0 2024-03-30T13:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.323+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcef00cef90 con 0x7fcef01f55b0 2024-03-30T13:11:08.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.323+0000 7fcee6ffd640 1 -- 172.21.15.43:0/1455669375 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fceec05a0a0 con 0x7fcef01f55b0 2024-03-30T13:11:08.331 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fced4067ff0 msgr2=0x7fced406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fced4067ff0 0x7fced406a4b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcee00ab000 tx=0x7fcee00b1040 comp rx=0 tx=0).stop 2024-03-30T13:11:08.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef01f55b0 msgr2=0x7fcef015ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef01f55b0 0x7fcef015ff70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fceec000c10 tx=0x7fceec00dc00 comp rx=0 tx=0).stop 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 shutdown_connections 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fced4067ff0 0x7fced406a4b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcef01f5a30 0x7fcef01604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcef01f55b0 0x7fcef015ff70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 --2- 172.21.15.43:0/1455669375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcedc0039f0 0x7fcef00b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 >> 172.21.15.43:0/1455669375 conn(0x7fcef00cdac0 msgr2=0x7fcef00cded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 shutdown_connections 2024-03-30T13:11:08.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.327+0000 7fcef84ce640 1 -- 172.21.15.43:0/1455669375 wait complete. 2024-03-30T13:11:08.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:11:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436cc10640 1 Processor -- start 2024-03-30T13:11:08.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436cc10640 1 -- start start 2024-03-30T13:11:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436cc10640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 0x7f436806dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436cc10640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4368162740 0x7f436806e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436cc10640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4368162cf0 con 0x7f436815f710 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436cc10640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f436815ec30 con 0x7f4368162740 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436cc10640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f436815ef10 con 0x7f43681553b0 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f4365d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f436815f710 0x7f4368162030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34678/0 (socket says 172.21.15.43:34678) 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f4365d9b640 1 -- 172.21.15.43:0/2170104727 learned_addr learned my addr 172.21.15.43:0/2170104727 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436659c640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 0x7f436806dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f4366d9d640 1 --2- 172.21.15.43:0/2170104727 >> v2:172.21.15.113:3300/0 conn(0x7f4368162740 0x7f436806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717215265 0 0) 0x7f4368162cf0 con 0x7f436815f710 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4350003530 con 0x7f436815f710 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 431328847 0 0) 0x7f4350003530 con 0x7f436815f710 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 msgr2=0x7f436806dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 0x7f436806dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 >> v2:172.21.15.113:3300/0 conn(0x7f4368162740 msgr2=0x7f436806e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 --2- 172.21.15.43:0/2170104727 >> v2:172.21.15.113:3300/0 conn(0x7f4368162740 0x7f436806e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436815f1f0 con 0x7f436815f710 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f4358001680 con 0x7f436815f710 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 >> v1:172.21.15.43:6789/0 conn(0x7f436815f710 legacy=0x7f4368162030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f4350003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 0x7f4350004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4350004d50 0x7f43500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f436815ec30 con 0x7f43500039c0 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f436815ef10 con 0x7f4350004d50 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f435000b9d0 con 0x7f43681553b0 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.467+0000 7f436659c640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f4350003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f4366d9d640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4350004d50 0x7f43500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f4365d9b640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 0x7f4350004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436659c640 1 -- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 msgr2=0x7f4350004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436659c640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 0x7f4350004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436659c640 1 -- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4350004d50 msgr2=0x7f43500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436659c640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4350004d50 0x7f43500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436659c640 1 -- 172.21.15.43:0/2170104727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4350003530 con 0x7f43500039c0 2024-03-30T13:11:08.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f4365d9b640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f43681553b0 0x7f4350004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436659c640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f4350003de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f435c009ae0 tx=0x7f435c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47e31aa1fe70f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0096b0 con 0x7f43500039c0 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c0a8430 con 0x7f43500039c0 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.471+0000 7f436559a640 1 -- 172.21.15.43:0/2170104727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0a8820 con 0x7f43500039c0 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- 172.21.15.43:0/2170104727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 msgr2=0x7f4350003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 --2- 172.21.15.43:0/2170104727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f4350003de0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f435c009ae0 tx=0x7f435c01ca70 comp rx=0 tx=0).stop 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f4366d9d640 1 -- 172.21.15.43:0/2170104727 reap_dead start 2024-03-30T13:11:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- 172.21.15.43:0/2170104727 shutdown_connections 2024-03-30T13:11:08.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- 172.21.15.43:0/2170104727 >> 172.21.15.43:0/2170104727 conn(0x7f43680b3d60 msgr2=0x7f43680b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- 172.21.15.43:0/2170104727 shutdown_connections 2024-03-30T13:11:08.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- 172.21.15.43:0/2170104727 wait complete. 2024-03-30T13:11:08.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 Processor -- start 2024-03-30T13:11:08.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- start start 2024-03-30T13:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f43680c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43681fa9d0 0x7f43681fae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436815ab20 0x7f436815afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f43680b2770 con 0x7f43500039c0 2024-03-30T13:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f43680b25f0 con 0x7f43681fa9d0 2024-03-30T13:11:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f43680b28f0 con 0x7f436815ab20 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f43680c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f43680c7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59798/0 (socket says 172.21.15.43:59798) 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 -- 172.21.15.43:0/1998424623 learned_addr learned my addr 172.21.15.43:0/1998424623 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f4366d9d640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436815ab20 0x7f436815afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f4365d9b640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43681fa9d0 0x7f43681fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 -- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436815ab20 msgr2=0x7f436815afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436815ab20 0x7f436815afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 -- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43681fa9d0 msgr2=0x7f43681fae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43681fa9d0 0x7f43681fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 -- 172.21.15.43:0/1998424623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436815b890 con 0x7f43500039c0 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436659c640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f43680c7610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f435c009ae0 tx=0x7f435c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f4356ffd640 1 -- 172.21.15.43:0/1998424623 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0be070 con 0x7f43500039c0 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f4356ffd640 1 -- 172.21.15.43:0/1998424623 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c0a8c60 con 0x7f43500039c0 2024-03-30T13:11:08.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f4356ffd640 1 -- 172.21.15.43:0/1998424623 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c0b9630 con 0x7f43500039c0 2024-03-30T13:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f436815bb70 con 0x7f43500039c0 2024-03-30T13:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.475+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4368156b90 con 0x7f43500039c0 2024-03-30T13:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.479+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4368068890 con 0x7f43500039c0 2024-03-30T13:11:08.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.479+0000 7f4356ffd640 1 -- 172.21.15.43:0/1998424623 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f435c0c2430 con 0x7f43500039c0 2024-03-30T13:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.479+0000 7f4356ffd640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4348067ff0 0x7f434806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.479+0000 7f4356ffd640 1 -- 172.21.15.43:0/1998424623 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f435c120410 con 0x7f43500039c0 2024-03-30T13:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.479+0000 7f4365d9b640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4348067ff0 0x7f434806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.479+0000 7f4365d9b640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4348067ff0 0x7f434806a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4358000f70 tx=0x7f4358009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.487+0000 7f4356ffd640 1 -- 172.21.15.43:0/1998424623 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f435c0ac030 con 0x7f43500039c0 2024-03-30T13:11:08.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.663+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43680b33c0 con 0x7f43500039c0 2024-03-30T13:11:08.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.663+0000 7f4356ffd640 1 -- 172.21.15.43:0/1998424623 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f435c016780 con 0x7f43500039c0 2024-03-30T13:11:08.672 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4348067ff0 msgr2=0x7f434806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4348067ff0 0x7f434806a4b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4358000f70 tx=0x7f4358009040 comp rx=0 tx=0).stop 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 msgr2=0x7f43680c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f43680c7610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f435c009ae0 tx=0x7f435c019810 comp rx=0 tx=0).stop 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 shutdown_connections 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4348067ff0 0x7f434806a4b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f436815ab20 0x7f436815afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f43681fa9d0 0x7f43681fae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 --2- 172.21.15.43:0/1998424623 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43500039c0 0x7f43680c7610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 >> 172.21.15.43:0/1998424623 conn(0x7f43680b3d60 msgr2=0x7f4368161610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 shutdown_connections 2024-03-30T13:11:08.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.667+0000 7f436cc10640 1 -- 172.21.15.43:0/1998424623 wait complete. 2024-03-30T13:11:08.691 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:11:08.691 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:08.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.851+0000 7f027803b640 1 Processor -- start 2024-03-30T13:11:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.851+0000 7f027803b640 1 -- start start 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f027803b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02700ceb60 0x7f02700d0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f027803b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0270162810 0x7f027015e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f027803b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02700b5040 con 0x7f02700ce2d0 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f027803b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0270162380 con 0x7f02700ceb60 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f027803b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f02700b5320 con 0x7f0270162810 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f0275dd7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02700ceb60 0x7f02700d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f02755d6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f02700ce2d0 0x7f02700cef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34688/0 (socket says 172.21.15.43:34688) 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f02755d6640 1 -- 172.21.15.43:0/724228140 learned_addr learned my addr 172.21.15.43:0/724228140 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f02765d8640 1 --2- 172.21.15.43:0/724228140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0270162810 0x7f027015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f0275dd7640 1 -- 172.21.15.43:0/724228140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0270162810 msgr2=0x7f027015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f0275dd7640 1 --2- 172.21.15.43:0/724228140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0270162810 0x7f027015e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f0275dd7640 1 -- 172.21.15.43:0/724228140 >> v1:172.21.15.43:6789/0 conn(0x7f02700ce2d0 legacy=0x7f02700cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f0274dd5640 1 -- 172.21.15.43:0/724228140 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4272302650 0 0) 0x7f02700b5040 con 0x7f02700ce2d0 2024-03-30T13:11:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.855+0000 7f0275dd7640 1 -- 172.21.15.43:0/724228140 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0270162010 con 0x7f02700ceb60 2024-03-30T13:11:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 --2- 172.21.15.43:0/724228140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ceb60 0x7f02700d0730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02600097c0 tx=0x7f026001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39391807c1cdc84f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0274dd5640 1 -- 172.21.15.43:0/724228140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600b0070 con 0x7f02700ceb60 2024-03-30T13:11:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0274dd5640 1 -- 172.21.15.43:0/724228140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02600ab600 con 0x7f02700ceb60 2024-03-30T13:11:08.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0274dd5640 1 -- 172.21.15.43:0/724228140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600ab9d0 con 0x7f02700ceb60 2024-03-30T13:11:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- 172.21.15.43:0/724228140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ceb60 msgr2=0x7f02700d0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 --2- 172.21.15.43:0/724228140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ceb60 0x7f02700d0730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f02600097c0 tx=0x7f026001c410 comp rx=0 tx=0).stop 2024-03-30T13:11:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- 172.21.15.43:0/724228140 shutdown_connections 2024-03-30T13:11:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 --2- 172.21.15.43:0/724228140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0270162810 0x7f027015e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 --2- 172.21.15.43:0/724228140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ceb60 0x7f02700d0730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- 172.21.15.43:0/724228140 >> 172.21.15.43:0/724228140 conn(0x7f02700b3c80 msgr2=0x7f02700b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:08.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- 172.21.15.43:0/724228140 shutdown_connections 2024-03-30T13:11:08.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- 172.21.15.43:0/724228140 wait complete. 2024-03-30T13:11:08.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 Processor -- start 2024-03-30T13:11:08.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- start start 2024-03-30T13:11:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ce2d0 0x7f02700c5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02700ceb60 0x7f0270155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0270162810 0x7f027006de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f027015c3b0 con 0x7f02700ce2d0 2024-03-30T13:11:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f027015c230 con 0x7f0270162810 2024-03-30T13:11:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f027803b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f027016bb20 con 0x7f02700ceb60 2024-03-30T13:11:08.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ce2d0 0x7f02700c5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ce2d0 0x7f02700c5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59816/0 (socket says 172.21.15.43:59816) 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 -- 172.21.15.43:0/3341911879 learned_addr learned my addr 172.21.15.43:0/3341911879 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f02765d8640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0270162810 0x7f027006de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f02755d6640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02700ceb60 0x7f0270155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 -- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02700ceb60 msgr2=0x7f0270155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02700ceb60 0x7f0270155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 -- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0270162810 msgr2=0x7f027006de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0270162810 0x7f027006de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 -- 172.21.15.43:0/3341911879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0270157310 con 0x7f02700ce2d0 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f02765d8640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0270162810 0x7f027006de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f02755d6640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02700ceb60 0x7f0270155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.859+0000 7f0275dd7640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ce2d0 0x7f02700c5010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02600097c0 tx=0x7f026001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02667fc640 1 -- 172.21.15.43:0/3341911879 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600bc070 con 0x7f02700ce2d0 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02667fc640 1 -- 172.21.15.43:0/3341911879 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02600b1040 con 0x7f02700ce2d0 2024-03-30T13:11:08.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0270154d90 con 0x7f02700ce2d0 2024-03-30T13:11:08.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02667fc640 1 -- 172.21.15.43:0/3341911879 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02600b0050 con 0x7f02700ce2d0 2024-03-30T13:11:08.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0270153310 con 0x7f02700ce2d0 2024-03-30T13:11:08.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0270154bc0 con 0x7f02700ce2d0 2024-03-30T13:11:08.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02667fc640 1 -- 172.21.15.43:0/3341911879 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f02600bf070 con 0x7f02700ce2d0 2024-03-30T13:11:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02667fc640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0254067ff0 0x7f025406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02667fc640 1 -- 172.21.15.43:0/3341911879 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0260120510 con 0x7f02700ce2d0 2024-03-30T13:11:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02755d6640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0254067ff0 0x7f025406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:08.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.863+0000 7f02755d6640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0254067ff0 0x7f025406a4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f026c007fd0 tx=0x7f026c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:08.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:08.867+0000 7f02667fc640 1 -- 172.21.15.43:0/3341911879 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0260016090 con 0x7f02700ce2d0 2024-03-30T13:11:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.027+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0270068890 con 0x7f02700ce2d0 2024-03-30T13:11:09.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.027+0000 7f02667fc640 1 -- 172.21.15.43:0/3341911879 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7f02600f08e0 con 0x7f02700ce2d0 2024-03-30T13:11:09.036 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:09.037 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0254067ff0 msgr2=0x7f025406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0254067ff0 0x7f025406a4b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f026c007fd0 tx=0x7f026c009040 comp rx=0 tx=0).stop 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ce2d0 msgr2=0x7f02700c5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ce2d0 0x7f02700c5010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f02600097c0 tx=0x7f026001ac50 comp rx=0 tx=0).stop 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 shutdown_connections 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0254067ff0 0x7f025406a4b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0270162810 0x7f027006de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f02700ceb60 0x7f0270155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 --2- 172.21.15.43:0/3341911879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02700ce2d0 0x7f02700c5010 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 >> 172.21.15.43:0/3341911879 conn(0x7f02700b3c80 msgr2=0x7f02701609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 shutdown_connections 2024-03-30T13:11:09.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.031+0000 7f027803b640 1 -- 172.21.15.43:0/3341911879 wait complete. 2024-03-30T13:11:09.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:09.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:09.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b79e1640 1 Processor -- start 2024-03-30T13:11:09.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b79e1640 1 -- start start 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b79e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90b015f880 0x7f90b00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b79e1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b006f090 0x7f90b00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b79e1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b00ce4d0 con 0x7f90b0070640 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b79e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f90b0070ad0 con 0x7f90b015f880 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b79e1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00cc7f0 con 0x7f90b006f090 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b577d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90b015f880 0x7f90b00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b577d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90b015f880 0x7f90b00b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59822/0 (socket says 172.21.15.43:59822) 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b577d640 1 -- 172.21.15.43:0/1082245241 learned_addr learned my addr 172.21.15.43:0/1082245241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b5f7e640 1 --2- 172.21.15.43:0/1082245241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b006f090 0x7f90b00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b577d640 1 -- 172.21.15.43:0/1082245241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b006f090 msgr2=0x7f90b00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:09.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b577d640 1 --2- 172.21.15.43:0/1082245241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b006f090 0x7f90b00ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b577d640 1 -- 172.21.15.43:0/1082245241 >> v1:172.21.15.113:6789/0 conn(0x7f90b0070640 legacy=0x7f90b006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:09.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.179+0000 7f90b577d640 1 -- 172.21.15.43:0/1082245241 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00ccad0 con 0x7f90b015f880 2024-03-30T13:11:09.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b577d640 1 --2- 172.21.15.43:0/1082245241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00b4ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009870 tx=0x7f90ac01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d653065473bb1c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90a7fff640 1 -- 172.21.15.43:0/1082245241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0b0070 con 0x7f90b015f880 2024-03-30T13:11:09.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90a7fff640 1 -- 172.21.15.43:0/1082245241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac0ab600 con 0x7f90b015f880 2024-03-30T13:11:09.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90a7fff640 1 -- 172.21.15.43:0/1082245241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac0ab9d0 con 0x7f90b015f880 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 -- 172.21.15.43:0/1082245241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 msgr2=0x7f90b00b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 --2- 172.21.15.43:0/1082245241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00b4ff0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009870 tx=0x7f90ac01c400 comp rx=0 tx=0).stop 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 -- 172.21.15.43:0/1082245241 shutdown_connections 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 --2- 172.21.15.43:0/1082245241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b006f090 0x7f90b00ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 --2- 172.21.15.43:0/1082245241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00b4ff0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 -- 172.21.15.43:0/1082245241 >> 172.21.15.43:0/1082245241 conn(0x7f90b00b3cf0 msgr2=0x7f90b00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 -- 172.21.15.43:0/1082245241 shutdown_connections 2024-03-30T13:11:09.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 -- 172.21.15.43:0/1082245241 wait complete. 2024-03-30T13:11:09.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 Processor -- start 2024-03-30T13:11:09.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.183+0000 7f90b79e1640 1 -- start start 2024-03-30T13:11:09.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90b006f090 0x7f90b015dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b0070640 0x7f90b0153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01609d0 con 0x7f90b015f880 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0160850 con 0x7f90b006f090 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f90b01691e0 con 0x7f90b0070640 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59836/0 (socket says 172.21.15.43:59836) 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 -- 172.21.15.43:0/471154663 learned_addr learned my addr 172.21.15.43:0/471154663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b577d640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90b006f090 0x7f90b015dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b4f7c640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b0070640 0x7f90b0153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 -- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b0070640 msgr2=0x7f90b0153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b0070640 0x7f90b0153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 -- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90b006f090 msgr2=0x7f90b015dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90b006f090 0x7f90b015dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 -- 172.21.15.43:0/471154663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00c9940 con 0x7f90b015f880 2024-03-30T13:11:09.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b5f7e640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00c91d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007d40 tx=0x7f90a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90a5ffb640 1 -- 172.21.15.43:0/471154663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8014420 con 0x7f90b015f880 2024-03-30T13:11:09.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90a5ffb640 1 -- 172.21.15.43:0/471154663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a8019070 con 0x7f90b015f880 2024-03-30T13:11:09.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b00c9c20 con 0x7f90b015f880 2024-03-30T13:11:09.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90a5ffb640 1 -- 172.21.15.43:0/471154663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8014820 con 0x7f90b015f880 2024-03-30T13:11:09.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b00ca1e0 con 0x7f90b015f880 2024-03-30T13:11:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0068900 con 0x7f90b015f880 2024-03-30T13:11:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90a5ffb640 1 -- 172.21.15.43:0/471154663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f90a80149c0 con 0x7f90b015f880 2024-03-30T13:11:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.187+0000 7f90a5ffb640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9094067fc0 0x7f909406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.191+0000 7f90a5ffb640 1 -- 172.21.15.43:0/471154663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f90a8087e30 con 0x7f90b015f880 2024-03-30T13:11:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.191+0000 7f90b577d640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9094067fc0 0x7f909406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.191+0000 7f90b577d640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9094067fc0 0x7f909406a480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0ad000 tx=0x7f90ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.195+0000 7f90a5ffb640 1 -- 172.21.15.43:0/471154663 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f90a801c360 con 0x7f90b015f880 2024-03-30T13:11:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f90b00d1790 con 0x7f90b015f880 2024-03-30T13:11:09.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90a5ffb640 1 -- 172.21.15.43:0/471154663 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f90a8058410 con 0x7f90b015f880 2024-03-30T13:11:09.386 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9094067fc0 msgr2=0x7f909406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9094067fc0 0x7f909406a480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0ad000 tx=0x7f90ac008040 comp rx=0 tx=0).stop 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 msgr2=0x7f90b00c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00c91d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007d40 tx=0x7f90a800e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 shutdown_connections 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9094067fc0 0x7f909406a480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90b015f880 0x7f90b00c91d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90b0070640 0x7f90b0153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 --2- 172.21.15.43:0/471154663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90b006f090 0x7f90b015dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 >> 172.21.15.43:0/471154663 conn(0x7f90b00b3cf0 msgr2=0x7f90b0155000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 shutdown_connections 2024-03-30T13:11:09.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.379+0000 7f90b79e1640 1 -- 172.21.15.43:0/471154663 wait complete. 2024-03-30T13:11:09.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.527+0000 7fadde9e6640 1 Processor -- start 2024-03-30T13:11:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.527+0000 7fadde9e6640 1 -- start start 2024-03-30T13:11:09.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadde9e6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadd815f880 0x7fadd80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadde9e6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadd806f090 0x7fadd80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadde9e6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadd80ce4d0 con 0x7fadd8070640 2024-03-30T13:11:09.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadde9e6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fadd8070ad0 con 0x7fadd815f880 2024-03-30T13:11:09.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadde9e6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fadd80cc7f0 con 0x7fadd806f090 2024-03-30T13:11:09.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadd815f880 0x7fadd80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadd815f880 0x7fadd80b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59848/0 (socket says 172.21.15.43:59848) 2024-03-30T13:11:09.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 -- 172.21.15.43:0/178989687 learned_addr learned my addr 172.21.15.43:0/178989687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:09.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7faddcf83640 1 --2- 172.21.15.43:0/178989687 >> v2:172.21.15.113:3300/0 conn(0x7fadd806f090 0x7fadd80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 -- 172.21.15.43:0/178989687 >> v2:172.21.15.113:3300/0 conn(0x7fadd806f090 msgr2=0x7fadd80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:09.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 --2- 172.21.15.43:0/178989687 >> v2:172.21.15.113:3300/0 conn(0x7fadd806f090 0x7fadd80ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 -- 172.21.15.43:0/178989687 >> v1:172.21.15.113:6789/0 conn(0x7fadd8070640 legacy=0x7fadd806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:09.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 -- 172.21.15.43:0/178989687 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd80ccad0 con 0x7fadd815f880 2024-03-30T13:11:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd7fff640 1 --2- 172.21.15.43:0/178989687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd815f880 0x7fadd80b4ff0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fadd80b4860 tx=0x7fadc801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876882ca1f2582d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd6ffd640 1 -- 172.21.15.43:0/178989687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc80b0070 con 0x7fadd815f880 2024-03-30T13:11:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd6ffd640 1 -- 172.21.15.43:0/178989687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadc80ab600 con 0x7fadd815f880 2024-03-30T13:11:09.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.531+0000 7fadd6ffd640 1 -- 172.21.15.43:0/178989687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc80ab9d0 con 0x7fadd815f880 2024-03-30T13:11:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- 172.21.15.43:0/178989687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd815f880 msgr2=0x7fadd80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 --2- 172.21.15.43:0/178989687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd815f880 0x7fadd80b4ff0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fadd80b4860 tx=0x7fadc801c3c0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- 172.21.15.43:0/178989687 shutdown_connections 2024-03-30T13:11:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 --2- 172.21.15.43:0/178989687 >> v2:172.21.15.113:3300/0 conn(0x7fadd806f090 0x7fadd80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 --2- 172.21.15.43:0/178989687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd815f880 0x7fadd80b4ff0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- 172.21.15.43:0/178989687 >> 172.21.15.43:0/178989687 conn(0x7fadd80b3cf0 msgr2=0x7fadd80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- 172.21.15.43:0/178989687 shutdown_connections 2024-03-30T13:11:09.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- 172.21.15.43:0/178989687 wait complete. 2024-03-30T13:11:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 Processor -- start 2024-03-30T13:11:09.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- start start 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd806f090 0x7fadd8156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8070640 0x7fadd80cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd815f880 0x7fadd815d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81609d0 con 0x7fadd8070640 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8160850 con 0x7fadd815f880 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadde9e6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fadd81691e0 con 0x7fadd806f090 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8070640 0x7fadd80cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.535+0000 7fadd77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8070640 0x7fadd80cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59862/0 (socket says 172.21.15.43:59862) 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd77fe640 1 -- 172.21.15.43:0/2775309001 learned_addr learned my addr 172.21.15.43:0/2775309001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd7fff640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd806f090 0x7fadd8156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7faddcf83640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd815f880 0x7fadd815d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd77fe640 1 -- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd806f090 msgr2=0x7fadd8156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd77fe640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd806f090 0x7fadd8156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd77fe640 1 -- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd815f880 msgr2=0x7fadd815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd77fe640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd815f880 0x7fadd815d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd77fe640 1 -- 172.21.15.43:0/2775309001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd81577e0 con 0x7fadd8070640 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd77fe640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8070640 0x7fadd80cbe30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00b3e0 tx=0x7fadcc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd4ff9640 1 -- 172.21.15.43:0/2775309001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc015020 con 0x7fadd8070640 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd4ff9640 1 -- 172.21.15.43:0/2775309001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadcc011850 con 0x7fadd8070640 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd8157220 con 0x7fadd8070640 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd4ff9640 1 -- 172.21.15.43:0/2775309001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc011c40 con 0x7fadd8070640 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd815cd50 con 0x7fadd8070640 2024-03-30T13:11:09.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd8068900 con 0x7fadd8070640 2024-03-30T13:11:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd4ff9640 1 -- 172.21.15.43:0/2775309001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fadcc018020 con 0x7fadd8070640 2024-03-30T13:11:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd4ff9640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fadc4067fc0 0x7fadc406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd4ff9640 1 -- 172.21.15.43:0/2775309001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fadcc0866f0 con 0x7fadd8070640 2024-03-30T13:11:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd7fff640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fadc4067fc0 0x7fadc406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.539+0000 7fadd7fff640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fadc4067fc0 0x7fadc406a480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fadc80ad000 tx=0x7fadc8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.543+0000 7fadd4ff9640 1 -- 172.21.15.43:0/2775309001 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fadcc01f020 con 0x7fadd8070640 2024-03-30T13:11:09.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.703+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fadd80d1790 con 0x7fadd8070640 2024-03-30T13:11:09.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.703+0000 7fadd4ff9640 1 -- 172.21.15.43:0/2775309001 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fadcc0172c0 con 0x7fadd8070640 2024-03-30T13:11:09.711 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fadc4067fc0 msgr2=0x7fadc406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fadc4067fc0 0x7fadc406a480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fadc80ad000 tx=0x7fadc8008040 comp rx=0 tx=0).stop 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8070640 msgr2=0x7fadd80cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8070640 0x7fadd80cbe30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00b3e0 tx=0x7fadcc00b8a0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 shutdown_connections 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fadc4067fc0 0x7fadc406a480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadd815f880 0x7fadd815d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadd8070640 0x7fadd80cbe30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 --2- 172.21.15.43:0/2775309001 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadd806f090 0x7fadd8156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 >> 172.21.15.43:0/2775309001 conn(0x7fadd80b3cf0 msgr2=0x7fadd80b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 shutdown_connections 2024-03-30T13:11:09.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.707+0000 7fadde9e6640 1 -- 172.21.15.43:0/2775309001 wait complete. 2024-03-30T13:11:09.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65bb020640 1 Processor -- start 2024-03-30T13:11:09.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65bb020640 1 -- start start 2024-03-30T13:11:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65bb020640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b40b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65bb020640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65b40700f0 0x7f65b40ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65bb020640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b40b4ba0 con 0x7f65b40b4520 2024-03-30T13:11:09.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65bb020640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f65b4070c30 con 0x7f65b40700f0 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65bb020640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40b4960 con 0x7f65b415afc0 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.831+0000 7f65b95bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65b40700f0 0x7f65b40ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b95bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65b40700f0 0x7f65b40ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59878/0 (socket says 172.21.15.43:59878) 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b95bd640 1 -- 172.21.15.43:0/3146075208 learned_addr learned my addr 172.21.15.43:0/3146075208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b8dbc640 1 --2- 172.21.15.43:0/3146075208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b40b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b95bd640 1 -- 172.21.15.43:0/3146075208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 msgr2=0x7f65b40b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b95bd640 1 --2- 172.21.15.43:0/3146075208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b40b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b95bd640 1 -- 172.21.15.43:0/3146075208 >> v1:172.21.15.113:6789/0 conn(0x7f65b40b4520 legacy=0x7f65b406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:09.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b95bd640 1 -- 172.21.15.43:0/3146075208 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b415ec60 con 0x7f65b40700f0 2024-03-30T13:11:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65b95bd640 1 --2- 172.21.15.43:0/3146075208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40700f0 0x7f65b40ce7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00d150 tx=0x7f65ac01ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a638ca1d1bf529d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65ab7fe640 1 -- 172.21.15.43:0/3146075208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac00b2d0 con 0x7f65b40700f0 2024-03-30T13:11:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65ab7fe640 1 -- 172.21.15.43:0/3146075208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ac00b470 con 0x7f65b40700f0 2024-03-30T13:11:09.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65ab7fe640 1 -- 172.21.15.43:0/3146075208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0a7c70 con 0x7f65b40700f0 2024-03-30T13:11:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 -- 172.21.15.43:0/3146075208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40700f0 msgr2=0x7f65b40ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 --2- 172.21.15.43:0/3146075208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40700f0 0x7f65b40ce7b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f65ac00d150 tx=0x7f65ac01ebc0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 -- 172.21.15.43:0/3146075208 shutdown_connections 2024-03-30T13:11:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 --2- 172.21.15.43:0/3146075208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40700f0 0x7f65b40ce7b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 --2- 172.21.15.43:0/3146075208 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b40b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 -- 172.21.15.43:0/3146075208 >> 172.21.15.43:0/3146075208 conn(0x7f65b40d1070 msgr2=0x7f65b40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 -- 172.21.15.43:0/3146075208 shutdown_connections 2024-03-30T13:11:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.835+0000 7f65bb020640 1 -- 172.21.15.43:0/3146075208 wait complete. 2024-03-30T13:11:09.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 Processor -- start 2024-03-30T13:11:09.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 -- start start 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65b40700f0 0x7f65b4145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40b4520 0x7f65b414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b4159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65b415b3a0 con 0x7f65b40b4520 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65b415b520 con 0x7f65b40700f0 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40cf500 con 0x7f65b415afc0 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40b4520 0x7f65b414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40b4520 0x7f65b414feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59894/0 (socket says 172.21.15.43:59894) 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 -- 172.21.15.43:0/2508743849 learned_addr learned my addr 172.21.15.43:0/2508743849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65b95bd640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b4159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65b8dbc640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65b40700f0 0x7f65b4145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 -- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 msgr2=0x7f65b4159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b4159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 -- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65b40700f0 msgr2=0x7f65b4145060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65b40700f0 0x7f65b4145060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 -- 172.21.15.43:0/2508743849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b41507a0 con 0x7f65b40b4520 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65abfff640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40b4520 0x7f65b414feb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65a400c450 tx=0x7f65a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65a97fa640 1 -- 172.21.15.43:0/2508743849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4012830 con 0x7f65b40b4520 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65a97fa640 1 -- 172.21.15.43:0/2508743849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a4017070 con 0x7f65b40b4520 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b4150a80 con 0x7f65b40b4520 2024-03-30T13:11:09.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65a97fa640 1 -- 172.21.15.43:0/2508743849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4012c30 con 0x7f65b40b4520 2024-03-30T13:11:09.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.839+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b4151040 con 0x7f65b40b4520 2024-03-30T13:11:09.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.843+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b4068900 con 0x7f65b40b4520 2024-03-30T13:11:09.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.843+0000 7f65a97fa640 1 -- 172.21.15.43:0/2508743849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f65a401c460 con 0x7f65b40b4520 2024-03-30T13:11:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.843+0000 7f65a97fa640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f65a0067fc0 0x7f65a006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.843+0000 7f65a97fa640 1 -- 172.21.15.43:0/2508743849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f65a4086570 con 0x7f65b40b4520 2024-03-30T13:11:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.843+0000 7f65b8dbc640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f65a0067fc0 0x7f65a006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:09.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.843+0000 7f65b8dbc640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f65a0067fc0 0x7f65a006a480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65b41f62b0 tx=0x7f65b0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:09.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:09.847+0000 7f65a97fa640 1 -- 172.21.15.43:0/2508743849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f65a4056a70 con 0x7f65b40b4520 2024-03-30T13:11:10.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.003+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f65b40b2560 con 0x7f65b40b4520 2024-03-30T13:11:10.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.007+0000 7f65a97fa640 1 -- 172.21.15.43:0/2508743849 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f65a4056370 con 0x7f65b40b4520 2024-03-30T13:11:10.013 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.007+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f65a0067fc0 msgr2=0x7f65a006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.007+0000 7f65bb020640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f65a0067fc0 0x7f65a006a480 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f65b41f62b0 tx=0x7f65b0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40b4520 msgr2=0x7f65b414feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40b4520 0x7f65b414feb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f65a400c450 tx=0x7f65a400c910 comp rx=0 tx=0).stop 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 shutdown_connections 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f65a0067fc0 0x7f65a006a480 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65b415afc0 0x7f65b4159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65b40b4520 0x7f65b414feb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 --2- 172.21.15.43:0/2508743849 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65b40700f0 0x7f65b4145060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 >> 172.21.15.43:0/2508743849 conn(0x7f65b40d1070 msgr2=0x7f65b406f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 shutdown_connections 2024-03-30T13:11:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.011+0000 7f65bb020640 1 -- 172.21.15.43:0/2508743849 wait complete. 2024-03-30T13:11:10.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:10.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.147+0000 7efc52f00640 1 Processor -- start 2024-03-30T13:11:10.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.147+0000 7efc52f00640 1 -- start start 2024-03-30T13:11:10.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc52f00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 0x7efc4c0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc52f00640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efc4c070640 0x7efc4c06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc52f00640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc4c070aa0 con 0x7efc4c06eec0 2024-03-30T13:11:10.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc52f00640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efc4c0ce4d0 con 0x7efc4c070640 2024-03-30T13:11:10.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc52f00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efc4c0cc7f0 con 0x7efc4c15f880 2024-03-30T13:11:10.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc5149d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efc4c06eec0 0x7efc4c0ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34696/0 (socket says 172.21.15.43:34696) 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc5149d640 1 -- 172.21.15.43:0/625699747 learned_addr learned my addr 172.21.15.43:0/625699747 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4bfff640 1 --2- 172.21.15.43:0/625699747 >> v2:172.21.15.113:3300/0 conn(0x7efc4c070640 0x7efc4c06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc50c9c640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 0x7efc4c0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243398720 0 0) 0x7efc4c070aa0 con 0x7efc4c06eec0 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc34003530 con 0x7efc4c06eec0 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2918048528 0 0) 0x7efc34003530 con 0x7efc4c06eec0 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 msgr2=0x7efc4c0b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 0x7efc4c0b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 >> v2:172.21.15.113:3300/0 conn(0x7efc4c070640 msgr2=0x7efc4c06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 --2- 172.21.15.43:0/625699747 >> v2:172.21.15.113:3300/0 conn(0x7efc4c070640 0x7efc4c06e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c0ccad0 con 0x7efc4c06eec0 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7efc44001680 con 0x7efc4c06eec0 2024-03-30T13:11:10.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 >> v1:172.21.15.43:6789/0 conn(0x7efc4c06eec0 legacy=0x7efc4c0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc340039c0 0x7efc34003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc34004810 0x7efc34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 0x7efc340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c0ce4d0 con 0x7efc34004810 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c0cc7f0 con 0x7efc340039c0 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efc3400ba50 con 0x7efc4c15f880 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4bfff640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc34004810 0x7efc34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc50c9c640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc340039c0 0x7efc34003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc5149d640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 0x7efc340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4bfff640 1 -- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 msgr2=0x7efc340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:10.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4bfff640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c15f880 0x7efc340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4bfff640 1 -- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc340039c0 msgr2=0x7efc34003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:10.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4bfff640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc340039c0 0x7efc34003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.151+0000 7efc4bfff640 1 -- 172.21.15.43:0/625699747 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc34003530 con 0x7efc34004810 2024-03-30T13:11:10.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc4bfff640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc34004810 0x7efc34004c90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc40009ae0 tx=0x7efc4001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d6f78fbed44dae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400096b0 con 0x7efc34004810 2024-03-30T13:11:10.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc400a6490 con 0x7efc34004810 2024-03-30T13:11:10.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc4b7fe640 1 -- 172.21.15.43:0/625699747 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400a6880 con 0x7efc34004810 2024-03-30T13:11:10.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- 172.21.15.43:0/625699747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc34004810 msgr2=0x7efc34004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 --2- 172.21.15.43:0/625699747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc34004810 0x7efc34004c90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc40009ae0 tx=0x7efc4001cab0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc5149d640 1 -- 172.21.15.43:0/625699747 reap_dead start 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- 172.21.15.43:0/625699747 shutdown_connections 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- 172.21.15.43:0/625699747 >> 172.21.15.43:0/625699747 conn(0x7efc4c0b3cf0 msgr2=0x7efc4c0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- 172.21.15.43:0/625699747 shutdown_connections 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- 172.21.15.43:0/625699747 wait complete. 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 Processor -- start 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- start start 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c14c630 0x7efc4c0c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc4c1599b0 0x7efc4c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0c8480 0x7efc4c0ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c0cf650 con 0x7efc4c0c8480 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efc4c0cf4d0 con 0x7efc4c1599b0 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.155+0000 7efc52f00640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efc4c160850 con 0x7efc4c14c630 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0c8480 0x7efc4c0ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0c8480 0x7efc4c0ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59916/0 (socket says 172.21.15.43:59916) 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 -- 172.21.15.43:0/3473828031 learned_addr learned my addr 172.21.15.43:0/3473828031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc4bfff640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc4c1599b0 0x7efc4c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc50c9c640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c14c630 0x7efc4c0c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 -- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c14c630 msgr2=0x7efc4c0c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c14c630 0x7efc4c0c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 -- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc4c1599b0 msgr2=0x7efc4c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc4c1599b0 0x7efc4c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 -- 172.21.15.43:0/3473828031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4c0b7a90 con 0x7efc4c0c8480 2024-03-30T13:11:10.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc4bfff640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc4c1599b0 0x7efc4c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc5149d640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0c8480 0x7efc4c0ca410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc44000c10 tx=0x7efc4400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc497fa640 1 -- 172.21.15.43:0/3473828031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4400c610 con 0x7efc4c0c8480 2024-03-30T13:11:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc497fa640 1 -- 172.21.15.43:0/3473828031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc44018070 con 0x7efc4c0c8480 2024-03-30T13:11:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4c0c7460 con 0x7efc4c0c8480 2024-03-30T13:11:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc497fa640 1 -- 172.21.15.43:0/3473828031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4400c9f0 con 0x7efc4c0c8480 2024-03-30T13:11:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc4c14ca10 con 0x7efc4c0c8480 2024-03-30T13:11:10.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc4c068900 con 0x7efc4c0c8480 2024-03-30T13:11:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc497fa640 1 -- 172.21.15.43:0/3473828031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efc4400cb90 con 0x7efc4c0c8480 2024-03-30T13:11:10.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc497fa640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efc38067fc0 0x7efc3806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc497fa640 1 -- 172.21.15.43:0/3473828031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7efc44086350 con 0x7efc4c0c8480 2024-03-30T13:11:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc50c9c640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efc38067fc0 0x7efc3806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.159+0000 7efc50c9c640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efc38067fc0 0x7efc3806a480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efc3c012350 tx=0x7efc3c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.171+0000 7efc497fa640 1 -- 172.21.15.43:0/3473828031 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efc440567d0 con 0x7efc4c0c8480 2024-03-30T13:11:10.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.331+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7efc4c15a690 con 0x7efc4c0c8480 2024-03-30T13:11:10.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.331+0000 7efc497fa640 1 -- 172.21.15.43:0/3473828031 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7efc440560d0 con 0x7efc4c0c8480 2024-03-30T13:11:10.340 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efc38067fc0 msgr2=0x7efc3806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efc38067fc0 0x7efc3806a480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efc3c012350 tx=0x7efc3c00b040 comp rx=0 tx=0).stop 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0c8480 msgr2=0x7efc4c0ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0c8480 0x7efc4c0ca410 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efc44000c10 tx=0x7efc4400eb90 comp rx=0 tx=0).stop 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 shutdown_connections 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efc38067fc0 0x7efc3806a480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0c8480 0x7efc4c0ca410 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efc4c1599b0 0x7efc4c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 --2- 172.21.15.43:0/3473828031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efc4c14c630 0x7efc4c0c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 >> 172.21.15.43:0/3473828031 conn(0x7efc4c0b3cf0 msgr2=0x7efc4c0b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:10.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 shutdown_connections 2024-03-30T13:11:10.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.335+0000 7efc52f00640 1 -- 172.21.15.43:0/3473828031 wait complete. 2024-03-30T13:11:10.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:11:10.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec8387640 1 Processor -- start 2024-03-30T13:11:10.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec8387640 1 -- start start 2024-03-30T13:11:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec8387640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbec00b5b20 0x7fbec006d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec8387640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbec006dcc0 0x7fbec006e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec8387640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbec015edd0 con 0x7fbec0070490 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec8387640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbec015f0b0 con 0x7fbec006dcc0 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec8387640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbec015f390 con 0x7fbec00b5b20 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec6924640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbec006dcc0 0x7fbec006e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec6123640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbec0070490 0x7fbec015c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34712/0 (socket says 172.21.15.43:34712) 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec6123640 1 -- 172.21.15.43:0/4971043 learned_addr learned my addr 172.21.15.43:0/4971043 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec5922640 1 --2- 172.21.15.43:0/4971043 >> v2:172.21.15.113:3300/0 conn(0x7fbec00b5b20 0x7fbec006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec6924640 1 -- 172.21.15.43:0/4971043 >> v2:172.21.15.113:3300/0 conn(0x7fbec00b5b20 msgr2=0x7fbec006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec6924640 1 --2- 172.21.15.43:0/4971043 >> v2:172.21.15.113:3300/0 conn(0x7fbec00b5b20 0x7fbec006d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec6924640 1 -- 172.21.15.43:0/4971043 >> v1:172.21.15.43:6789/0 conn(0x7fbec0070490 legacy=0x7fbec015c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec5121640 1 -- 172.21.15.43:0/4971043 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 196888634 0 0) 0x7fbec015edd0 con 0x7fbec0070490 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.463+0000 7fbec6924640 1 -- 172.21.15.43:0/4971043 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec015f670 con 0x7fbec006dcc0 2024-03-30T13:11:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec5922640 1 --2- 172.21.15.43:0/4971043 >> v2:172.21.15.113:3300/0 conn(0x7fbec00b5b20 0x7fbec006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec6924640 1 --2- 172.21.15.43:0/4971043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec006dcc0 0x7fbec006e810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbec0073d50 tx=0x7fbeb801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30ebcd88ce8c910 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec5121640 1 -- 172.21.15.43:0/4971043 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb80a66e0 con 0x7fbec006dcc0 2024-03-30T13:11:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec5121640 1 -- 172.21.15.43:0/4971043 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb80a6880 con 0x7fbec006dcc0 2024-03-30T13:11:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec5121640 1 -- 172.21.15.43:0/4971043 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb80a6c50 con 0x7fbec006dcc0 2024-03-30T13:11:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 -- 172.21.15.43:0/4971043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec006dcc0 msgr2=0x7fbec006e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 --2- 172.21.15.43:0/4971043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec006dcc0 0x7fbec006e810 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbec0073d50 tx=0x7fbeb801c810 comp rx=0 tx=0).stop 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 -- 172.21.15.43:0/4971043 shutdown_connections 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 --2- 172.21.15.43:0/4971043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec006dcc0 0x7fbec006e810 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 --2- 172.21.15.43:0/4971043 >> v2:172.21.15.113:3300/0 conn(0x7fbec00b5b20 0x7fbec006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 -- 172.21.15.43:0/4971043 >> 172.21.15.43:0/4971043 conn(0x7fbec0072500 msgr2=0x7fbec0072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 -- 172.21.15.43:0/4971043 shutdown_connections 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 -- 172.21.15.43:0/4971043 wait complete. 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 Processor -- start 2024-03-30T13:11:10.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.467+0000 7fbec8387640 1 -- start start 2024-03-30T13:11:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbec006dcc0 0x7fbec014e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbec0070490 0x7fbec014a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec00b5b20 0x7fbec0164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbec006f8f0 con 0x7fbec00b5b20 2024-03-30T13:11:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbec006f770 con 0x7fbec0070490 2024-03-30T13:11:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbec006fa70 con 0x7fbec006dcc0 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec00b5b20 0x7fbec0164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec00b5b20 0x7fbec0164e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59932/0 (socket says 172.21.15.43:59932) 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 -- 172.21.15.43:0/2129460126 learned_addr learned my addr 172.21.15.43:0/2129460126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec5922640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbec0070490 0x7fbec014a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 -- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbec006dcc0 msgr2=0x7fbec014e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbec006dcc0 0x7fbec014e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 -- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbec0070490 msgr2=0x7fbec014a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbec0070490 0x7fbec014a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 -- 172.21.15.43:0/2129460126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec015d8e0 con 0x7fbec00b5b20 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6924640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec00b5b20 0x7fbec0164e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8019420 tx=0x7fbeb80195b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbeb6ffd640 1 -- 172.21.15.43:0/2129460126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb80be070 con 0x7fbec00b5b20 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbeb6ffd640 1 -- 172.21.15.43:0/2129460126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeb80ae930 con 0x7fbec00b5b20 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec01643a0 con 0x7fbec00b5b20 2024-03-30T13:11:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbeb6ffd640 1 -- 172.21.15.43:0/2129460126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeb80aed20 con 0x7fbec00b5b20 2024-03-30T13:11:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec014a590 con 0x7fbec00b5b20 2024-03-30T13:11:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbec0068890 con 0x7fbec00b5b20 2024-03-30T13:11:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbeb6ffd640 1 -- 172.21.15.43:0/2129460126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbeb80c1070 con 0x7fbec00b5b20 2024-03-30T13:11:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbeb6ffd640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbea4067fc0 0x7fbea406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbeb6ffd640 1 -- 172.21.15.43:0/2129460126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbeb81221a0 con 0x7fbec00b5b20 2024-03-30T13:11:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.471+0000 7fbec6123640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbea4067fc0 0x7fbea406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:10.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.475+0000 7fbec6123640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbea4067fc0 0x7fbea406a480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbebc005890 tx=0x7fbebc007a10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.475+0000 7fbeb6ffd640 1 -- 172.21.15.43:0/2129460126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbeb80c8070 con 0x7fbec00b5b20 2024-03-30T13:11:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbec00b53f0 con 0x7fbec00b5b20 2024-03-30T13:11:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbeb6ffd640 1 -- 172.21.15.43:0/2129460126 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fbeb8016520 con 0x7fbec00b5b20 2024-03-30T13:11:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:11:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbea4067fc0 msgr2=0x7fbea406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbea4067fc0 0x7fbea406a480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbebc005890 tx=0x7fbebc007a10 comp rx=0 tx=0).stop 2024-03-30T13:11:10.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec00b5b20 msgr2=0x7fbec0164e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec00b5b20 0x7fbec0164e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8019420 tx=0x7fbeb80195b0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 shutdown_connections 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbea4067fc0 0x7fbea406a480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbec00b5b20 0x7fbec0164e80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbec0070490 0x7fbec014a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 --2- 172.21.15.43:0/2129460126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbec006dcc0 0x7fbec014e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 >> 172.21.15.43:0/2129460126 conn(0x7fbec0072500 msgr2=0x7fbec006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 shutdown_connections 2024-03-30T13:11:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:10.643+0000 7fbec8387640 1 -- 172.21.15.43:0/2129460126 wait complete. 2024-03-30T13:11:10.667 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-03-30T13:11:10.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:10.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-03-30T13:11:10.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi043.stderr:ceph-fuse[33458]: fuse finished with error 0 and tester_r 0 2024-03-30T13:11:10.690 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:11:16.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:11:16.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:16.792 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-03-30T13:11:16.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:16.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.979+0000 7fc028db6640 1 Processor -- start 2024-03-30T13:11:16.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.979+0000 7fc028db6640 1 -- start start 2024-03-30T13:11:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc028db6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc02415e450 0x7fc0240cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc028db6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0240d0450 0x7fc0240ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc028db6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02415e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:16.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc028db6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc0240b2980 con 0x7fc0240d0450 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc028db6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc0240cf4b0 con 0x7fc02415e450 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc028db6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc0240b1d70 con 0x7fc0240cf0a0 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0240d0450 0x7fc0240ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0240d0450 0x7fc0240ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40958/0 (socket says 172.21.15.43:40958) 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 -- 172.21.15.43:0/3809508261 learned_addr learned my addr 172.21.15.43:0/3809508261 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc022d9d640 1 --2- 172.21.15.43:0/3809508261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02415e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc02259c640 1 --2- 172.21.15.43:0/3809508261 >> v2:172.21.15.113:3300/0 conn(0x7fc02415e450 0x7fc0240cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 -- 172.21.15.43:0/3809508261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 msgr2=0x7fc02415e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 --2- 172.21.15.43:0/3809508261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02415e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 -- 172.21.15.43:0/3809508261 >> v2:172.21.15.113:3300/0 conn(0x7fc02415e450 msgr2=0x7fc0240cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 --2- 172.21.15.43:0/3809508261 >> v2:172.21.15.113:3300/0 conn(0x7fc02415e450 0x7fc0240cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc021d9b640 1 -- 172.21.15.43:0/3809508261 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0240b2050 con 0x7fc0240d0450 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc02259c640 1 --2- 172.21.15.43:0/3809508261 >> v2:172.21.15.113:3300/0 conn(0x7fc02415e450 0x7fc0240cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:16.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.983+0000 7fc022d9d640 1 --2- 172.21.15.43:0/3809508261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc021d9b640 1 --2- 172.21.15.43:0/3809508261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc0240ceb60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc0240cfad0 tx=0x7fc01801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5cdaffa2ecc52 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc02159a640 1 -- 172.21.15.43:0/3809508261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180aa610 con 0x7fc0240d0450 2024-03-30T13:11:16.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc02159a640 1 -- 172.21.15.43:0/3809508261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0180af070 con 0x7fc0240d0450 2024-03-30T13:11:16.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc02159a640 1 -- 172.21.15.43:0/3809508261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180aa9d0 con 0x7fc0240d0450 2024-03-30T13:11:16.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- 172.21.15.43:0/3809508261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 msgr2=0x7fc0240ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 --2- 172.21.15.43:0/3809508261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc0240ceb60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc0240cfad0 tx=0x7fc01801c4f0 comp rx=0 tx=0).stop 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- 172.21.15.43:0/3809508261 shutdown_connections 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 --2- 172.21.15.43:0/3809508261 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02415e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 --2- 172.21.15.43:0/3809508261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc0240ceb60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 --2- 172.21.15.43:0/3809508261 >> v2:172.21.15.113:3300/0 conn(0x7fc02415e450 0x7fc0240cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- 172.21.15.43:0/3809508261 >> 172.21.15.43:0/3809508261 conn(0x7fc0240b3d60 msgr2=0x7fc0240b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- 172.21.15.43:0/3809508261 shutdown_connections 2024-03-30T13:11:16.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- 172.21.15.43:0/3809508261 wait complete. 2024-03-30T13:11:16.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 Processor -- start 2024-03-30T13:11:16.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- start start 2024-03-30T13:11:17.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc02406dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc02415e450 0x7fc02415fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc02415f700 con 0x7fc0240d0450 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc02415f580 con 0x7fc02415e450 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.987+0000 7fc028db6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc02415f880 con 0x7fc0240cf0a0 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc02406dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc02406dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40970/0 (socket says 172.21.15.43:40970) 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 -- 172.21.15.43:0/539807165 learned_addr learned my addr 172.21.15.43:0/539807165 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc022d9d640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc02415e450 0x7fc02415fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc02259c640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 -- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 msgr2=0x7fc02406d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02406d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 -- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc02415e450 msgr2=0x7fc02415fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc02415e450 0x7fc02415fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 -- 172.21.15.43:0/539807165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0241467e0 con 0x7fc0240d0450 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc022d9d640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc02415e450 0x7fc02415fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc02259c640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02406d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc021d9b640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc02406dd00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc01801aba0 tx=0x7fc01801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc012ffd640 1 -- 172.21.15.43:0/539807165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180b3800 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc012ffd640 1 -- 172.21.15.43:0/539807165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0180b39a0 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc024160730 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc012ffd640 1 -- 172.21.15.43:0/539807165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180b3d90 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc024160cf0 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc024068890 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc012ffd640 1 -- 172.21.15.43:0/539807165 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc0180c6390 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc012ffd640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc004067fc0 0x7fc00406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc012ffd640 1 -- 172.21.15.43:0/539807165 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc018121060 con 0x7fc0240d0450 2024-03-30T13:11:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.991+0000 7fc02259c640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc004067fc0 0x7fc00406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:16.995+0000 7fc02259c640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc004067fc0 0x7fc00406a480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc01c008890 tx=0x7fc01c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.003+0000 7fc012ffd640 1 -- 172.21.15.43:0/539807165 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc0180c3060 con 0x7fc0240d0450 2024-03-30T13:11:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.167+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0240c8280 con 0x7fc0240d0450 2024-03-30T13:11:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.167+0000 7fc012ffd640 1 -- 172.21.15.43:0/539807165 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7fc0180a7070 con 0x7fc0240d0450 2024-03-30T13:11:17.176 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:17.176 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc004067fc0 msgr2=0x7fc00406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc004067fc0 0x7fc00406a480 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc01c008890 tx=0x7fc01c008040 comp rx=0 tx=0).stop 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 msgr2=0x7fc02406dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc02406dd00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc01801aba0 tx=0x7fc01801ad30 comp rx=0 tx=0).stop 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 shutdown_connections 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc004067fc0 0x7fc00406a480 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc02415e450 0x7fc02415fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0240d0450 0x7fc02406dd00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 --2- 172.21.15.43:0/539807165 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc0240cf0a0 0x7fc02406d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 >> 172.21.15.43:0/539807165 conn(0x7fc0240b3d60 msgr2=0x7fc024153600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 shutdown_connections 2024-03-30T13:11:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.171+0000 7fc028db6640 1 -- 172.21.15.43:0/539807165 wait complete. 2024-03-30T13:11:17.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:17.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.303+0000 7fc53d012640 1 Processor -- start 2024-03-30T13:11:17.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.303+0000 7fc53d012640 1 -- start start 2024-03-30T13:11:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53d012640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc538072ce0 0x7fc53815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53d012640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5380b5ba0 con 0x7fc538161cb0 2024-03-30T13:11:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53d012640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5380b5e80 con 0x7fc53815f790 2024-03-30T13:11:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53d012640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc5380cee60 con 0x7fc538072ce0 2024-03-30T13:11:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc538072ce0 0x7fc53815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc536d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc538161cb0 0x7fc5380b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50448/0 (socket says 172.21.15.43:50448) 2024-03-30T13:11:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc536d9d640 1 -- 172.21.15.43:0/3344006919 learned_addr learned my addr 172.21.15.43:0/3344006919 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53759e640 1 -- 172.21.15.43:0/3344006919 >> v1:172.21.15.113:6789/0 conn(0x7fc53815f790 legacy=0x7fc5380725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53759e640 1 -- 172.21.15.43:0/3344006919 >> v1:172.21.15.43:6789/0 conn(0x7fc538161cb0 legacy=0x7fc5380b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53759e640 1 -- 172.21.15.43:0/3344006919 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5380b6160 con 0x7fc538072ce0 2024-03-30T13:11:17.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc535d9b640 1 -- 172.21.15.43:0/3344006919 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1030505226 0 0) 0x7fc5380b5ba0 con 0x7fc538161cb0 2024-03-30T13:11:17.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc53759e640 1 --2- 172.21.15.43:0/3344006919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc538072ce0 0x7fc53815ff90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc530009d90 tx=0x7fc53001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc052ad39e91afc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc535d9b640 1 -- 172.21.15.43:0/3344006919 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5300b2070 con 0x7fc538072ce0 2024-03-30T13:11:17.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc535d9b640 1 -- 172.21.15.43:0/3344006919 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5300a5980 con 0x7fc538072ce0 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.307+0000 7fc535d9b640 1 -- 172.21.15.43:0/3344006919 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5300a5d50 con 0x7fc538072ce0 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/3344006919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc538072ce0 msgr2=0x7fc53815ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 --2- 172.21.15.43:0/3344006919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc538072ce0 0x7fc53815ff90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc530009d90 tx=0x7fc53001c8a0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/3344006919 shutdown_connections 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 --2- 172.21.15.43:0/3344006919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc538072ce0 0x7fc53815ff90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/3344006919 >> 172.21.15.43:0/3344006919 conn(0x7fc5380d0f60 msgr2=0x7fc5380d1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/3344006919 shutdown_connections 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/3344006919 wait complete. 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 Processor -- start 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- start start 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc538072ce0 0x7fc5381409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53815f790 0x7fc5381486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc538161cb0 0x7fc538148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380cf750 con 0x7fc53815f790 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc5380cf5d0 con 0x7fc538072ce0 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc5380cf8d0 con 0x7fc538161cb0 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53815f790 0x7fc5381486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53815f790 0x7fc5381486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40996/0 (socket says 172.21.15.43:40996) 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 -- 172.21.15.43:0/2548202837 learned_addr learned my addr 172.21.15.43:0/2548202837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53759e640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc538161cb0 0x7fc538148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc536d9d640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc538072ce0 0x7fc5381409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 -- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc538161cb0 msgr2=0x7fc538148be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc538161cb0 0x7fc538148be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 -- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc538072ce0 msgr2=0x7fc5381409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc538072ce0 0x7fc5381409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 -- 172.21.15.43:0/2548202837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5380ada10 con 0x7fc53815f790 2024-03-30T13:11:17.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc536d9d640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc538072ce0 0x7fc5381409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53759e640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc538161cb0 0x7fc538148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53659c640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53815f790 0x7fc5381486a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00c450 tx=0x7fc52c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc51f7fe640 1 -- 172.21.15.43:0/2548202837 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c012830 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc51f7fe640 1 -- 172.21.15.43:0/2548202837 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52c017070 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5380adcf0 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc51f7fe640 1 -- 172.21.15.43:0/2548202837 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c012c30 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5380ae2b0 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.311+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc538068890 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.315+0000 7fc51f7fe640 1 -- 172.21.15.43:0/2548202837 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc52c01c460 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.315+0000 7fc51f7fe640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc524067fc0 0x7fc52406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.315+0000 7fc51f7fe640 1 -- 172.21.15.43:0/2548202837 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc52c087120 con 0x7fc53815f790 2024-03-30T13:11:17.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.315+0000 7fc536d9d640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc524067fc0 0x7fc52406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.315+0000 7fc536d9d640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc524067fc0 0x7fc52406a480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc528007fd0 tx=0x7fc528009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.323+0000 7fc51f7fe640 1 -- 172.21.15.43:0/2548202837 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc52c057570 con 0x7fc53815f790 2024-03-30T13:11:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.487+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5380738f0 con 0x7fc53815f790 2024-03-30T13:11:17.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc51f7fe640 1 -- 172.21.15.43:0/2548202837 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7fc52c056e70 con 0x7fc53815f790 2024-03-30T13:11:17.498 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:17.498 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc524067fc0 msgr2=0x7fc52406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc524067fc0 0x7fc52406a480 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc528007fd0 tx=0x7fc528009040 comp rx=0 tx=0).stop 2024-03-30T13:11:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53815f790 msgr2=0x7fc5381486a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53815f790 0x7fc5381486a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00c450 tx=0x7fc52c00c910 comp rx=0 tx=0).stop 2024-03-30T13:11:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 shutdown_connections 2024-03-30T13:11:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc524067fc0 0x7fc52406a480 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc538161cb0 0x7fc538148be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53815f790 0x7fc5381486a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 --2- 172.21.15.43:0/2548202837 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc538072ce0 0x7fc5381409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 >> 172.21.15.43:0/2548202837 conn(0x7fc5380d0f60 msgr2=0x7fc5380cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 shutdown_connections 2024-03-30T13:11:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.491+0000 7fc53d012640 1 -- 172.21.15.43:0/2548202837 wait complete. 2024-03-30T13:11:17.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T13:11:17.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f6376c3b640 1 Processor -- start 2024-03-30T13:11:17.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f6376c3b640 1 -- start start 2024-03-30T13:11:17.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f6376c3b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f63700d0bc0 0x7f637015c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f6376c3b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 0x7f637006d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f6376c3b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f637015d190 con 0x7f637006f9e0 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f6376c3b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f637006cb50 con 0x7f63700d0bc0 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f6376c3b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f637006bf40 con 0x7f637006cfe0 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f637006f9e0 0x7f637006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50454/0 (socket says 172.21.15.43:50454) 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 -- 172.21.15.43:0/1981731377 learned_addr learned my addr 172.21.15.43:0/1981731377 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63751d8640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 0x7f637006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636ffff640 1 --2- 172.21.15.43:0/1981731377 >> v2:172.21.15.113:3300/0 conn(0x7f63700d0bc0 0x7f637015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369650156 0 0) 0x7f637015d190 con 0x7f637006f9e0 2024-03-30T13:11:17.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6358003530 con 0x7f637006f9e0 2024-03-30T13:11:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1390755934 0 0) 0x7f6358003530 con 0x7f637006f9e0 2024-03-30T13:11:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 msgr2=0x7f637006d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 0x7f637006d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 >> v2:172.21.15.113:3300/0 conn(0x7f63700d0bc0 msgr2=0x7f637015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 --2- 172.21.15.43:0/1981731377 >> v2:172.21.15.113:3300/0 conn(0x7f63700d0bc0 0x7f637015c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637006c220 con 0x7f637006f9e0 2024-03-30T13:11:17.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f6360001a00 con 0x7f637006f9e0 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 >> v1:172.21.15.43:6789/0 conn(0x7f637006f9e0 legacy=0x7f637006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63580039c0 0x7f6358003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 0x7f6358004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358004d50 0x7f63580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f637006cb50 con 0x7f63580039c0 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f637006bf40 con 0x7f6358004d50 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f635800b9d0 con 0x7f637006cfe0 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63580039c0 0x7f6358003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f636ffff640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 0x7f6358004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63751d8640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358004d50 0x7f63580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 -- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 msgr2=0x7f6358004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f637006cfe0 0x7f6358004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 -- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358004d50 msgr2=0x7f63580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6358004d50 0x7f63580051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.631+0000 7f63749d7640 1 -- 172.21.15.43:0/1981731377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6358003530 con 0x7f63580039c0 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f63749d7640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63580039c0 0x7f6358003de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f636000a890 tx=0x7f636001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f341e3ce88a250d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636000a4e0 con 0x7f63580039c0 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63600ad650 con 0x7f63580039c0 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f636f7fe640 1 -- 172.21.15.43:0/1981731377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63600ada40 con 0x7f63580039c0 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f6376c3b640 1 -- 172.21.15.43:0/1981731377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63580039c0 msgr2=0x7f6358003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f6376c3b640 1 --2- 172.21.15.43:0/1981731377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63580039c0 0x7f6358003de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f636000a890 tx=0x7f636001d3a0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f63751d8640 1 -- 172.21.15.43:0/1981731377 reap_dead start 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f6376c3b640 1 -- 172.21.15.43:0/1981731377 shutdown_connections 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f6376c3b640 1 -- 172.21.15.43:0/1981731377 >> 172.21.15.43:0/1981731377 conn(0x7f63700d1070 msgr2=0x7f63700d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f6376c3b640 1 -- 172.21.15.43:0/1981731377 shutdown_connections 2024-03-30T13:11:17.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.635+0000 7f6376c3b640 1 -- 172.21.15.43:0/1981731377 wait complete. 2024-03-30T13:11:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 Processor -- start 2024-03-30T13:11:17.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 -- start start 2024-03-30T13:11:17.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580039c0 0x7f63700afad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f637014d8c0 0x7f637013bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637013f4b0 0x7f6370072520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6370070750 con 0x7f637013f4b0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f63700705d0 con 0x7f637014d8c0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6370166e90 con 0x7f63580039c0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637013f4b0 0x7f6370072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637013f4b0 0x7f6370072520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41026/0 (socket says 172.21.15.43:41026) 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 -- 172.21.15.43:0/2337449901 learned_addr learned my addr 172.21.15.43:0/2337449901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63749d7640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580039c0 0x7f63700afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f636ffff640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f637014d8c0 0x7f637013bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 -- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580039c0 msgr2=0x7f63700afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580039c0 0x7f63700afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 -- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f637014d8c0 msgr2=0x7f637013bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f637014d8c0 0x7f637013bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 -- 172.21.15.43:0/2337449901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63700b0140 con 0x7f637013f4b0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63749d7640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580039c0 0x7f63700afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f63751d8640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637013f4b0 0x7f6370072520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f637006e180 tx=0x7f6368010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f636d7fa640 1 -- 172.21.15.43:0/2337449901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800f040 con 0x7f637013f4b0 2024-03-30T13:11:17.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f636d7fa640 1 -- 172.21.15.43:0/2337449901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f636800e840 con 0x7f637013f4b0 2024-03-30T13:11:17.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637013e9d0 con 0x7f637013f4b0 2024-03-30T13:11:17.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f636d7fa640 1 -- 172.21.15.43:0/2337449901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636800ec30 con 0x7f637013f4b0 2024-03-30T13:11:17.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637013ef90 con 0x7f637013f4b0 2024-03-30T13:11:17.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.639+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f637013f190 con 0x7f637013f4b0 2024-03-30T13:11:17.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.643+0000 7f636d7fa640 1 -- 172.21.15.43:0/2337449901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f636801c400 con 0x7f637013f4b0 2024-03-30T13:11:17.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.643+0000 7f636d7fa640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f635c067ff0 0x7f635c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:17.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.643+0000 7f636d7fa640 1 -- 172.21.15.43:0/2337449901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6368086cf0 con 0x7f637013f4b0 2024-03-30T13:11:17.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.643+0000 7f63749d7640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f635c067ff0 0x7f635c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:17.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.643+0000 7f63749d7640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f635c067ff0 0x7f635c06a4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63600ad000 tx=0x7f6360001dd0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:17.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.647+0000 7f636d7fa640 1 -- 172.21.15.43:0/2337449901 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f637013f190 con 0x7f637013f4b0 2024-03-30T13:11:17.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:17.835+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6370068900 con 0x7f637013f4b0 2024-03-30T13:11:18.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f636d7fa640 1 -- 172.21.15.43:0/2337449901 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f6368018030 con 0x7f637013f4b0 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f635c067ff0 msgr2=0x7f635c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f635c067ff0 0x7f635c06a4b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f63600ad000 tx=0x7f6360001dd0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637013f4b0 msgr2=0x7f6370072520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637013f4b0 0x7f6370072520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f637006e180 tx=0x7f6368010c20 comp rx=0 tx=0).stop 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 shutdown_connections 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f635c067ff0 0x7f635c06a4b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f637013f4b0 0x7f6370072520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f637014d8c0 0x7f637013bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 --2- 172.21.15.43:0/2337449901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f63580039c0 0x7f63700afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 >> 172.21.15.43:0/2337449901 conn(0x7f63700d1070 msgr2=0x7f63701585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 shutdown_connections 2024-03-30T13:11:18.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.023+0000 7f6376c3b640 1 -- 172.21.15.43:0/2337449901 wait complete. 2024-03-30T13:11:18.051 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-03-30T13:11:18.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T13:11:18.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01ff696640 1 Processor -- start 2024-03-30T13:11:18.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01ff696640 1 -- start start 2024-03-30T13:11:18.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01ff696640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f80ae960 0x7f01f80b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:18.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01ff696640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f806c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01ff696640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f80b58c0 con 0x7f01f815e2f0 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01ff696640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f01f80cd060 con 0x7f01f80ae960 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01ff696640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f01f806b660 con 0x7f01f815e6d0 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fcc31640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f80ae960 0x7f01f80b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fcc31640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f80ae960 0x7f01f80b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41040/0 (socket says 172.21.15.43:41040) 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fcc31640 1 -- 172.21.15.43:0/34031866 learned_addr learned my addr 172.21.15.43:0/34031866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fdc33640 1 --2- 172.21.15.43:0/34031866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f806c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fcc31640 1 -- 172.21.15.43:0/34031866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 msgr2=0x7f01f806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fcc31640 1 --2- 172.21.15.43:0/34031866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f806c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fcc31640 1 -- 172.21.15.43:0/34031866 >> v1:172.21.15.113:6789/0 conn(0x7f01f815e2f0 legacy=0x7f01f80ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:18.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.167+0000 7f01fcc31640 1 -- 172.21.15.43:0/34031866 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f80b5ba0 con 0x7f01f80ae960 2024-03-30T13:11:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01fcc31640 1 --2- 172.21.15.43:0/34031866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f80ae960 0x7f01f80b51e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009870 tx=0x7f01e801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84126c029a9ad33d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01effff640 1 -- 172.21.15.43:0/34031866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80ab610 con 0x7f01f80ae960 2024-03-30T13:11:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01effff640 1 -- 172.21.15.43:0/34031866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e80b0070 con 0x7f01f80ae960 2024-03-30T13:11:18.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01effff640 1 -- 172.21.15.43:0/34031866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80ab9d0 con 0x7f01f80ae960 2024-03-30T13:11:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 -- 172.21.15.43:0/34031866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f80ae960 msgr2=0x7f01f80b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 --2- 172.21.15.43:0/34031866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f80ae960 0x7f01f80b51e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f01e8009870 tx=0x7f01e801c410 comp rx=0 tx=0).stop 2024-03-30T13:11:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 -- 172.21.15.43:0/34031866 shutdown_connections 2024-03-30T13:11:18.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 --2- 172.21.15.43:0/34031866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f806c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 --2- 172.21.15.43:0/34031866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f80ae960 0x7f01f80b51e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 -- 172.21.15.43:0/34031866 >> 172.21.15.43:0/34031866 conn(0x7f01f80d0c90 msgr2=0x7f01f80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:18.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 -- 172.21.15.43:0/34031866 shutdown_connections 2024-03-30T13:11:18.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.171+0000 7f01ff696640 1 -- 172.21.15.43:0/34031866 wait complete. 2024-03-30T13:11:18.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 Processor -- start 2024-03-30T13:11:18.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 -- start start 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f80ae960 0x7f01f806ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f815e2f0 0x7f01f80cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f81533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f01f816b480 con 0x7f01f815e2f0 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f01f816b600 con 0x7f01f80ae960 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f01f816b780 con 0x7f01f815e6d0 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f815e2f0 0x7f01f80cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f815e2f0 0x7f01f80cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41050/0 (socket says 172.21.15.43:41050) 2024-03-30T13:11:18.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 -- 172.21.15.43:0/683378433 learned_addr learned my addr 172.21.15.43:0/683378433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fd432640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f80ae960 0x7f01f806ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fdc33640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f81533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 -- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 msgr2=0x7f01f81533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f81533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 -- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f80ae960 msgr2=0x7f01f806ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f80ae960 0x7f01f806ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 -- 172.21.15.43:0/683378433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f806f410 con 0x7f01f815e2f0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01fcc31640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f815e2f0 0x7f01f80cd2a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f01e801a040 tx=0x7f01e801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01edffb640 1 -- 172.21.15.43:0/683378433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80be5f0 con 0x7f01f815e2f0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01edffb640 1 -- 172.21.15.43:0/683378433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e80be7b0 con 0x7f01f815e2f0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f81585c0 con 0x7f01f815e2f0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01edffb640 1 -- 172.21.15.43:0/683378433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80beba0 con 0x7f01f815e2f0 2024-03-30T13:11:18.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.175+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f8158b80 con 0x7f01f815e2f0 2024-03-30T13:11:18.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.179+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f8068900 con 0x7f01f815e2f0 2024-03-30T13:11:18.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.179+0000 7f01edffb640 1 -- 172.21.15.43:0/683378433 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f01e80c73f0 con 0x7f01f815e2f0 2024-03-30T13:11:18.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.179+0000 7f01edffb640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f01dc067fc0 0x7f01dc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:18.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.179+0000 7f01edffb640 1 -- 172.21.15.43:0/683378433 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f01e81214e0 con 0x7f01f815e2f0 2024-03-30T13:11:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.179+0000 7f01fd432640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f01dc067fc0 0x7f01dc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:18.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.179+0000 7f01fd432640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f01dc067fc0 0x7f01dc06a480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f01f4001650 tx=0x7f01f4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:18.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.187+0000 7f01edffb640 1 -- 172.21.15.43:0/683378433 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f01e80a8050 con 0x7f01f815e2f0 2024-03-30T13:11:18.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:18.383+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f01f806de50 con 0x7f01f815e2f0 2024-03-30T13:11:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.031+0000 7f01edffb640 1 -- 172.21.15.43:0/683378433 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f01f806de50 con 0x7f01f815e2f0 2024-03-30T13:11:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f01dc067fc0 msgr2=0x7f01dc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f01dc067fc0 0x7f01dc06a480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f01f4001650 tx=0x7f01f4009040 comp rx=0 tx=0).stop 2024-03-30T13:11:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f815e2f0 msgr2=0x7f01f80cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f815e2f0 0x7f01f80cd2a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f01e801a040 tx=0x7f01e801ac50 comp rx=0 tx=0).stop 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 shutdown_connections 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f01dc067fc0 0x7f01dc06a480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f01f815e6d0 0x7f01f81533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f815e2f0 0x7f01f80cd2a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 --2- 172.21.15.43:0/683378433 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f01f80ae960 0x7f01f806ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 >> 172.21.15.43:0/683378433 conn(0x7f01f80d0c90 msgr2=0x7f01f815b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 shutdown_connections 2024-03-30T13:11:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.035+0000 7f01ff696640 1 -- 172.21.15.43:0/683378433 wait complete. 2024-03-30T13:11:19.063 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T13:11:19.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:19.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76dea4f640 1 Processor -- start 2024-03-30T13:11:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76dea4f640 1 -- start start 2024-03-30T13:11:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76dea4f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 0x7f76d80cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76dea4f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d80cfb60 con 0x7f76d80cf5e0 2024-03-30T13:11:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76dea4f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76d80cfe40 con 0x7f76d80cd850 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76dea4f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f76d80d0120 con 0x7f76d8142440 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f76d80cf5e0 0x7f76d80cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50460/0 (socket says 172.21.15.43:50460) 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d77fe640 1 -- 172.21.15.43:0/4065292140 learned_addr learned my addr 172.21.15.43:0/4065292140 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 0x7f76d80cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096281970 0 0) 0x7f76d80cfb60 con 0x7f76d80cf5e0 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76c0003530 con 0x7f76d80cf5e0 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2669072673 0 0) 0x7f76c0003530 con 0x7f76d80cf5e0 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 msgr2=0x7f76d80cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 0x7f76d80cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 >> v1:172.21.15.113:6789/0 conn(0x7f76d80cd850 legacy=0x7f76d806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d80d0400 con 0x7f76d80cf5e0 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f76cc001680 con 0x7f76d80cf5e0 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 >> v1:172.21.15.43:6789/0 conn(0x7f76d80cf5e0 legacy=0x7f76d80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c00039f0 0x7f76c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 0x7f76c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004d50 0x7f76c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76d80d0120 con 0x7f76c00039f0 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f76c000ba80 con 0x7f76c0004d50 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f76c000bc80 con 0x7f76d8142440 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c00039f0 0x7f76c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76dcfec640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004d50 0x7f76c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d77fe640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 0x7f76c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d7fff640 1 -- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 msgr2=0x7f76c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 0x7f76c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d7fff640 1 -- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004d50 msgr2=0x7f76c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76c0004d50 0x7f76c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d7fff640 1 -- 172.21.15.43:0/4065292140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c0003530 con 0x7f76c00039f0 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.183+0000 7f76d77fe640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76d8142440 0x7f76c0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c00039f0 0x7f76c0003e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009ab0 tx=0x7f76c801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a29fcb9aca38a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c80096b0 con 0x7f76c00039f0 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76c80a6510 con 0x7f76c00039f0 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76d6ffd640 1 -- 172.21.15.43:0/4065292140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c80a6900 con 0x7f76c00039f0 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 -- 172.21.15.43:0/4065292140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c00039f0 msgr2=0x7f76c0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 --2- 172.21.15.43:0/4065292140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76c00039f0 0x7f76c0003e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f76c8009ab0 tx=0x7f76c801ca20 comp rx=0 tx=0).stop 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dcfec640 1 -- 172.21.15.43:0/4065292140 reap_dead start 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 -- 172.21.15.43:0/4065292140 shutdown_connections 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 -- 172.21.15.43:0/4065292140 >> 172.21.15.43:0/4065292140 conn(0x7f76d80b2b30 msgr2=0x7f76d80d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 -- 172.21.15.43:0/4065292140 shutdown_connections 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 -- 172.21.15.43:0/4065292140 wait complete. 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 Processor -- start 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.187+0000 7f76dea4f640 1 -- start start 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c00039f0 0x7f76d81f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d81583b0 0x7f76d8158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d813e8a0 0x7f76d813ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f76d806cb40 con 0x7f76d813e8a0 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f76d80ccc70 con 0x7f76d81583b0 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f76d806ccc0 con 0x7f76c00039f0 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d813e8a0 0x7f76d813ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d813e8a0 0x7f76d813ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41060/0 (socket says 172.21.15.43:41060) 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 -- 172.21.15.43:0/4235694327 learned_addr learned my addr 172.21.15.43:0/4235694327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d77fe640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d81583b0 0x7f76d8158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c00039f0 0x7f76d81f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 -- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c00039f0 msgr2=0x7f76d81f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c00039f0 0x7f76d81f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 -- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d81583b0 msgr2=0x7f76d8158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d81583b0 0x7f76d8158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 -- 172.21.15.43:0/4235694327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d813f540 con 0x7f76d813e8a0 2024-03-30T13:11:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dcfec640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d813e8a0 0x7f76d813ed20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76d80b5dd0 tx=0x7f76d000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d4ff9640 1 -- 172.21.15.43:0/4235694327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003b90 con 0x7f76d813e8a0 2024-03-30T13:11:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d4ff9640 1 -- 172.21.15.43:0/4235694327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d0018070 con 0x7f76d813e8a0 2024-03-30T13:11:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d813f820 con 0x7f76d813e8a0 2024-03-30T13:11:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d4ff9640 1 -- 172.21.15.43:0/4235694327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d00050d0 con 0x7f76d813e8a0 2024-03-30T13:11:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d813fde0 con 0x7f76d813e8a0 2024-03-30T13:11:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76d8068890 con 0x7f76d813e8a0 2024-03-30T13:11:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d4ff9640 1 -- 172.21.15.43:0/4235694327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f76d0005270 con 0x7f76d813e8a0 2024-03-30T13:11:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d4ff9640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f76c4067ff0 0x7f76c406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d4ff9640 1 -- 172.21.15.43:0/4235694327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f76d0086470 con 0x7f76d813e8a0 2024-03-30T13:11:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.191+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f76c4067ff0 0x7f76c406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.195+0000 7f76d7fff640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f76c4067ff0 0x7f76c406a4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76c80af000 tx=0x7f76c80046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.199+0000 7f76d4ff9640 1 -- 172.21.15.43:0/4235694327 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f76d0056970 con 0x7f76d813e8a0 2024-03-30T13:11:19.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.395+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76d80b00f0 con 0x7f76d813e8a0 2024-03-30T13:11:19.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.395+0000 7f76d4ff9640 1 -- 172.21.15.43:0/4235694327 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f76d0056270 con 0x7f76d813e8a0 2024-03-30T13:11:19.404 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:19.404 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:11:19.404 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f76c4067ff0 msgr2=0x7f76c406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f76c4067ff0 0x7f76c406a4b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f76c80af000 tx=0x7f76c80046d0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d813e8a0 msgr2=0x7f76d813ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d813e8a0 0x7f76d813ed20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f76d80b5dd0 tx=0x7f76d000e9e0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 shutdown_connections 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f76c4067ff0 0x7f76c406a4b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d813e8a0 0x7f76d813ed20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f76d81583b0 0x7f76d8158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 --2- 172.21.15.43:0/4235694327 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f76c00039f0 0x7f76d81f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 >> 172.21.15.43:0/4235694327 conn(0x7f76d80b2b30 msgr2=0x7f76d80cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:19.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 shutdown_connections 2024-03-30T13:11:19.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.399+0000 7f76dea4f640 1 -- 172.21.15.43:0/4235694327 wait complete. 2024-03-30T13:11:19.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9d53f9640 1 Processor -- start 2024-03-30T13:11:19.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9d53f9640 1 -- start start 2024-03-30T13:11:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9d53f9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd9d0164ff0 0x7fd9d01673e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9d53f9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d0160a90 con 0x7fd9d0145670 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9d53f9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d00cfce0 con 0x7fd9d00d18e0 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9d53f9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d00d1e30 con 0x7fd9d0164ff0 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9ceffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd9d0145670 0x7fd9d015fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50462/0 (socket says 172.21.15.43:50462) 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9ceffd640 1 -- 172.21.15.43:0/176611330 learned_addr learned my addr 172.21.15.43:0/176611330 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cf7fe640 1 --2- 172.21.15.43:0/176611330 >> v2:172.21.15.113:3300/0 conn(0x7fd9d0164ff0 0x7fd9d01673e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310685590 0 0) 0x7fd9d0160a90 con 0x7fd9d0145670 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9b8003530 con 0x7fd9d0145670 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 189974209 0 0) 0x7fd9b8003530 con 0x7fd9d0145670 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 >> v2:172.21.15.113:3300/0 conn(0x7fd9d0164ff0 msgr2=0x7fd9d01673e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 --2- 172.21.15.43:0/176611330 >> v2:172.21.15.113:3300/0 conn(0x7fd9d0164ff0 0x7fd9d01673e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 >> v1:172.21.15.113:6789/0 conn(0x7fd9d00d18e0 legacy=0x7fd9d0160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d006f430 con 0x7fd9d0145670 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fd9c0001680 con 0x7fd9d0145670 2024-03-30T13:11:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 >> v1:172.21.15.43:6789/0 conn(0x7fd9d0145670 legacy=0x7fd9d015fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80039c0 0x7fd9b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b8004810 0x7fd9b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d00d1e30 con 0x7fd9b80051d0 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b800c070 con 0x7fd9b8004810 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b800c270 con 0x7fd9b80039c0 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cf7fe640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9ceffd640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80039c0 0x7fd9b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9ce7fc640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b8004810 0x7fd9b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cf7fe640 1 -- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80039c0 msgr2=0x7fd9b8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cf7fe640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9b80039c0 0x7fd9b8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cf7fe640 1 -- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b8004810 msgr2=0x7fd9b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cf7fe640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9b8004810 0x7fd9b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.535+0000 7fd9cf7fe640 1 -- 172.21.15.43:0/176611330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b8003530 con 0x7fd9b80051d0 2024-03-30T13:11:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9cf7fe640 1 -- 172.21.15.43:0/176611330 reap_dead start 2024-03-30T13:11:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9cf7fe640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9b800b9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009f90 tx=0x7fd9c801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63edea6ba1e4758 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8009c10 con 0x7fd9b80051d0 2024-03-30T13:11:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c80a5c90 con 0x7fd9b80051d0 2024-03-30T13:11:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9cdffb640 1 -- 172.21.15.43:0/176611330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8003be0 con 0x7fd9b80051d0 2024-03-30T13:11:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 -- 172.21.15.43:0/176611330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 msgr2=0x7fd9b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9b800b9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009f90 tx=0x7fd9c801cc80 comp rx=0 tx=0).stop 2024-03-30T13:11:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 -- 172.21.15.43:0/176611330 shutdown_connections 2024-03-30T13:11:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/176611330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9b800b9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 -- 172.21.15.43:0/176611330 >> 172.21.15.43:0/176611330 conn(0x7fd9d00b3c80 msgr2=0x7fd9d00b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 -- 172.21.15.43:0/176611330 shutdown_connections 2024-03-30T13:11:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 -- 172.21.15.43:0/176611330 wait complete. 2024-03-30T13:11:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 Processor -- start 2024-03-30T13:11:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.539+0000 7fd9d53f9640 1 -- start start 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9d00c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d00b7610 0x7fd9d01414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d006c770 0x7fd9d006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d0145d10 con 0x7fd9b80051d0 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d006f9f0 con 0x7fd9d006c770 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d0145e90 con 0x7fd9d00b7610 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9d00c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9d00c9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41068/0 (socket says 172.21.15.43:41068) 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 -- 172.21.15.43:0/2474627058 learned_addr learned my addr 172.21.15.43:0/2474627058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ce7fc640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d00b7610 0x7fd9d01414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9cf7fe640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d006c770 0x7fd9d006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 -- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d00b7610 msgr2=0x7fd9d01414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d00b7610 0x7fd9d01414c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 -- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d006c770 msgr2=0x7fd9d006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d006c770 0x7fd9d006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 -- 172.21.15.43:0/2474627058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d00c9720 con 0x7fd9b80051d0 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9cf7fe640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d006c770 0x7fd9d006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ceffd640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9d00c9140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e780 tx=0x7fd9c000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9b77fe640 1 -- 172.21.15.43:0/2474627058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000c6c0 con 0x7fd9b80051d0 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9b77fe640 1 -- 172.21.15.43:0/2474627058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c0018070 con 0x7fd9b80051d0 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d00c9a00 con 0x7fd9b80051d0 2024-03-30T13:11:19.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9b77fe640 1 -- 172.21.15.43:0/2474627058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c000ca60 con 0x7fd9b80051d0 2024-03-30T13:11:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d00ca2a0 con 0x7fd9b80051d0 2024-03-30T13:11:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d0068890 con 0x7fd9b80051d0 2024-03-30T13:11:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9b77fe640 1 -- 172.21.15.43:0/2474627058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd9c000cc00 con 0x7fd9b80051d0 2024-03-30T13:11:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9b77fe640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9bc067fc0 0x7fd9bc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9b77fe640 1 -- 172.21.15.43:0/2474627058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd9c0086670 con 0x7fd9b80051d0 2024-03-30T13:11:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.543+0000 7fd9ce7fc640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9bc067fc0 0x7fd9bc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.547+0000 7fd9ce7fc640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9bc067fc0 0x7fd9bc06a480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9d006d8d0 tx=0x7fd9c4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.551+0000 7fd9b77fe640 1 -- 172.21.15.43:0/2474627058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd9c0056ae0 con 0x7fd9b80051d0 2024-03-30T13:11:19.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.715+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9d00ca470 con 0x7fd9b80051d0 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.715+0000 7fd9b77fe640 1 -- 172.21.15.43:0/2474627058 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7fd9c00563e0 con 0x7fd9b80051d0 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:19.723 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9bc067fc0 msgr2=0x7fd9bc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9bc067fc0 0x7fd9bc06a480 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd9d006d8d0 tx=0x7fd9c4009040 comp rx=0 tx=0).stop 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 msgr2=0x7fd9d00c9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9d00c9140 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd9c000e780 tx=0x7fd9c000eb80 comp rx=0 tx=0).stop 2024-03-30T13:11:19.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 shutdown_connections 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9bc067fc0 0x7fd9bc06a480 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9d006c770 0x7fd9d006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9d00b7610 0x7fd9d01414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 --2- 172.21.15.43:0/2474627058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9b80051d0 0x7fd9d00c9140 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 >> 172.21.15.43:0/2474627058 conn(0x7fd9d00b3c80 msgr2=0x7fd9d0141e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 shutdown_connections 2024-03-30T13:11:19.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.719+0000 7fd9d53f9640 1 -- 172.21.15.43:0/2474627058 wait complete. 2024-03-30T13:11:19.743 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T13:11:19.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.859+0000 7f2ad8678640 1 Processor -- start 2024-03-30T13:11:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.859+0000 7f2ad8678640 1 -- start start 2024-03-30T13:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad8678640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad00b3720 0x7f2ad015f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad8678640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ad006f950 0x7f2ad00b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad8678640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ad015f870 con 0x7f2ad0161ea0 2024-03-30T13:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad8678640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad006c670 con 0x7f2ad006f950 2024-03-30T13:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad8678640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad00b4040 con 0x7f2ad00b3720 2024-03-30T13:11:19.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad6c15640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2ad0161ea0 0x7f2ad0162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50470/0 (socket says 172.21.15.43:50470) 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad6c15640 1 -- 172.21.15.43:0/2848844031 learned_addr learned my addr 172.21.15.43:0/2848844031 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad5c13640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ad006f950 0x7f2ad00b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad6414640 1 --2- 172.21.15.43:0/2848844031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad00b3720 0x7f2ad015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad5c13640 1 -- 172.21.15.43:0/2848844031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad00b3720 msgr2=0x7f2ad015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad5c13640 1 --2- 172.21.15.43:0/2848844031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad00b3720 0x7f2ad015f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad5c13640 1 -- 172.21.15.43:0/2848844031 >> v1:172.21.15.43:6789/0 conn(0x7f2ad0161ea0 legacy=0x7f2ad0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad5412640 1 -- 172.21.15.43:0/2848844031 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2853597296 0 0) 0x7f2ac8001120 con 0x7f2ad0161ea0 2024-03-30T13:11:19.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad5c13640 1 -- 172.21.15.43:0/2848844031 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad006dc30 con 0x7f2ad006f950 2024-03-30T13:11:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.863+0000 7f2ad5c13640 1 --2- 172.21.15.43:0/2848844031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad006f950 0x7f2ad00b3b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40097c0 tx=0x7f2ac401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9d353b8daf85bc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad5412640 1 -- 172.21.15.43:0/2848844031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40a7070 con 0x7f2ad006f950 2024-03-30T13:11:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad5412640 1 -- 172.21.15.43:0/2848844031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac40b0070 con 0x7f2ad006f950 2024-03-30T13:11:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad5412640 1 -- 172.21.15.43:0/2848844031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40ab890 con 0x7f2ad006f950 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- 172.21.15.43:0/2848844031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad006f950 msgr2=0x7f2ad00b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 --2- 172.21.15.43:0/2848844031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad006f950 0x7f2ad00b3b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40097c0 tx=0x7f2ac401c470 comp rx=0 tx=0).stop 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- 172.21.15.43:0/2848844031 shutdown_connections 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 --2- 172.21.15.43:0/2848844031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad006f950 0x7f2ad00b3b00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 --2- 172.21.15.43:0/2848844031 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad00b3720 0x7f2ad015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- 172.21.15.43:0/2848844031 >> 172.21.15.43:0/2848844031 conn(0x7f2ad00b2b30 msgr2=0x7f2ad00d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- 172.21.15.43:0/2848844031 shutdown_connections 2024-03-30T13:11:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- 172.21.15.43:0/2848844031 wait complete. 2024-03-30T13:11:19.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 Processor -- start 2024-03-30T13:11:19.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- start start 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad006f950 0x7f2ad015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ad00b3720 0x7f2ad0149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0161ea0 0x7f2ad014a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad006ecb0 con 0x7f2ad0161ea0 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad006e790 con 0x7f2ad00b3720 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad8678640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad006ee30 con 0x7f2ad006f950 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0161ea0 0x7f2ad014a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0161ea0 0x7f2ad014a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41088/0 (socket says 172.21.15.43:41088) 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 -- 172.21.15.43:0/796929060 learned_addr learned my addr 172.21.15.43:0/796929060 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6414640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad006f950 0x7f2ad015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad5c13640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ad00b3720 0x7f2ad0149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 -- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad006f950 msgr2=0x7f2ad015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad006f950 0x7f2ad015dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 -- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ad00b3720 msgr2=0x7f2ad0149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ad00b3720 0x7f2ad0149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.867+0000 7f2ad6c15640 1 -- 172.21.15.43:0/796929060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad0162560 con 0x7f2ad0161ea0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2ad6c15640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0161ea0 0x7f2ad014a5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800c7b0 tx=0x7f2ac800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2aceffd640 1 -- 172.21.15.43:0/796929060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8018070 con 0x7f2ad0161ea0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2aceffd640 1 -- 172.21.15.43:0/796929060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac80070d0 con 0x7f2ad0161ea0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad015d120 con 0x7f2ad0161ea0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2aceffd640 1 -- 172.21.15.43:0/796929060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac80074c0 con 0x7f2ad0161ea0 2024-03-30T13:11:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad015d6e0 con 0x7f2ad0161ea0 2024-03-30T13:11:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad0068890 con 0x7f2ad0161ea0 2024-03-30T13:11:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2aceffd640 1 -- 172.21.15.43:0/796929060 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2ac801b070 con 0x7f2ad0161ea0 2024-03-30T13:11:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2aceffd640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2abc067ff0 0x7f2abc06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2aceffd640 1 -- 172.21.15.43:0/796929060 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2ac8085b60 con 0x7f2ad0161ea0 2024-03-30T13:11:19.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2ad6414640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2abc067ff0 0x7f2abc06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:19.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.871+0000 7f2ad6414640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2abc067ff0 0x7f2abc06a4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0003fd0 tx=0x7f2ac00039c0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:19.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:19.875+0000 7f2aceffd640 1 -- 172.21.15.43:0/796929060 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2ad015f870 con 0x7f2ad0161ea0 2024-03-30T13:11:20.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.099+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ad00678a0 con 0x7f2abc067ff0 2024-03-30T13:11:20.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.123+0000 7f2aceffd640 1 -- 172.21.15.43:0/796929060 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f2ad00678a0 con 0x7f2abc067ff0 2024-03-30T13:11:20.135 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:20.138 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:11:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2abc067ff0 msgr2=0x7f2abc06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2abc067ff0 0x7f2abc06a4b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0003fd0 tx=0x7f2ac00039c0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0161ea0 msgr2=0x7f2ad014a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0161ea0 0x7f2ad014a5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2ac800c7b0 tx=0x7f2ac800cc70 comp rx=0 tx=0).stop 2024-03-30T13:11:20.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 shutdown_connections 2024-03-30T13:11:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2abc067ff0 0x7f2abc06a4b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ad0161ea0 0x7f2ad014a5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2ad00b3720 0x7f2ad0149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 --2- 172.21.15.43:0/796929060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2ad006f950 0x7f2ad015dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 >> 172.21.15.43:0/796929060 conn(0x7f2ad00b2b30 msgr2=0x7f2ad0160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 shutdown_connections 2024-03-30T13:11:20.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.131+0000 7f2ad8678640 1 -- 172.21.15.43:0/796929060 wait complete. 2024-03-30T13:11:20.155 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-30T13:11:19.844205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236252,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738468,"statfs":{"total":773094113280,"available":772852191232,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005122"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857033+0000","last_change":"2024-03-30T13:10:54.944419+0000","last_active":"2024-03-30T13:11:01.857033+0000","last_peered":"2024-03-30T13:11:01.857033+0000","last_clean":"2024-03-30T13:11:01.857033+0000","last_became_active":"2024-03-30T13:10:54.943176+0000","last_became_peered":"2024-03-30T13:10:54.943176+0000","last_unstale":"2024-03-30T13:11:01.857033+0000","last_undegraded":"2024-03-30T13:11:01.857033+0000","last_fullsized":"2024-03-30T13:11:01.857033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983961+0000","last_change":"2024-03-30T13:10:55.957609+0000","last_active":"2024-03-30T13:11:00.983961+0000","last_peered":"2024-03-30T13:11:00.983961+0000","last_clean":"2024-03-30T13:11:00.983961+0000","last_became_active":"2024-03-30T13:10:55.956870+0000","last_became_peered":"2024-03-30T13:10:55.956870+0000","last_unstale":"2024-03-30T13:11:00.983961+0000","last_undegraded":"2024-03-30T13:11:00.983961+0000","last_fullsized":"2024-03-30T13:11:00.983961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:04:28.315124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858409+0000","last_change":"2024-03-30T13:11:00.037601+0000","last_active":"2024-03-30T13:11:01.858409+0000","last_peered":"2024-03-30T13:11:01.858409+0000","last_clean":"2024-03-30T13:11:01.858409+0000","last_became_active":"2024-03-30T13:11:00.032050+0000","last_became_peered":"2024-03-30T13:11:00.032050+0000","last_unstale":"2024-03-30T13:11:01.858409+0000","last_undegraded":"2024-03-30T13:11:01.858409+0000","last_fullsized":"2024-03-30T13:11:01.858409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:47.447707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.984087+0000","last_change":"2024-03-30T13:10:54.940096+0000","last_active":"2024-03-30T13:11:00.984087+0000","last_peered":"2024-03-30T13:11:00.984087+0000","last_clean":"2024-03-30T13:11:00.984087+0000","last_became_active":"2024-03-30T13:10:54.939368+0000","last_became_peered":"2024-03-30T13:10:54.939368+0000","last_unstale":"2024-03-30T13:11:00.984087+0000","last_undegraded":"2024-03-30T13:11:00.984087+0000","last_fullsized":"2024-03-30T13:11:00.984087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:09:39.904016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981759+0000","last_change":"2024-03-30T13:10:55.979018+0000","last_active":"2024-03-30T13:11:00.981759+0000","last_peered":"2024-03-30T13:11:00.981759+0000","last_clean":"2024-03-30T13:11:00.981759+0000","last_became_active":"2024-03-30T13:10:55.975807+0000","last_became_peered":"2024-03-30T13:10:55.975807+0000","last_unstale":"2024-03-30T13:11:00.981759+0000","last_undegraded":"2024-03-30T13:11:00.981759+0000","last_fullsized":"2024-03-30T13:11:00.981759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:35:35.881832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857637+0000","last_change":"2024-03-30T13:11:00.020209+0000","last_active":"2024-03-30T13:11:01.857637+0000","last_peered":"2024-03-30T13:11:01.857637+0000","last_clean":"2024-03-30T13:11:01.857637+0000","last_became_active":"2024-03-30T13:11:00.019627+0000","last_became_peered":"2024-03-30T13:11:00.019627+0000","last_unstale":"2024-03-30T13:11:01.857637+0000","last_undegraded":"2024-03-30T13:11:01.857637+0000","last_fullsized":"2024-03-30T13:11:01.857637+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:31.595903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856064+0000","last_change":"2024-03-30T13:10:54.945298+0000","last_active":"2024-03-30T13:11:01.856064+0000","last_peered":"2024-03-30T13:11:01.856064+0000","last_clean":"2024-03-30T13:11:01.856064+0000","last_became_active":"2024-03-30T13:10:54.942725+0000","last_became_peered":"2024-03-30T13:10:54.942725+0000","last_unstale":"2024-03-30T13:11:01.856064+0000","last_undegraded":"2024-03-30T13:11:01.856064+0000","last_fullsized":"2024-03-30T13:11:01.856064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978049+0000","last_change":"2024-03-30T13:10:55.973824+0000","last_active":"2024-03-30T13:11:00.978049+0000","last_peered":"2024-03-30T13:11:00.978049+0000","last_clean":"2024-03-30T13:11:00.978049+0000","last_became_active":"2024-03-30T13:10:55.969728+0000","last_became_peered":"2024-03-30T13:10:55.969728+0000","last_unstale":"2024-03-30T13:11:00.978049+0000","last_undegraded":"2024-03-30T13:11:00.978049+0000","last_fullsized":"2024-03-30T13:11:00.978049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858055+0000","last_change":"2024-03-30T13:11:00.015698+0000","last_active":"2024-03-30T13:11:01.858055+0000","last_peered":"2024-03-30T13:11:01.858055+0000","last_clean":"2024-03-30T13:11:01.858055+0000","last_became_active":"2024-03-30T13:11:00.015263+0000","last_became_peered":"2024-03-30T13:11:00.015263+0000","last_unstale":"2024-03-30T13:11:01.858055+0000","last_undegraded":"2024-03-30T13:11:01.858055+0000","last_fullsized":"2024-03-30T13:11:01.858055+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856071+0000","last_change":"2024-03-30T13:10:54.941048+0000","last_active":"2024-03-30T13:11:01.856071+0000","last_peered":"2024-03-30T13:11:01.856071+0000","last_clean":"2024-03-30T13:11:01.856071+0000","last_became_active":"2024-03-30T13:10:54.940543+0000","last_became_peered":"2024-03-30T13:10:54.940543+0000","last_unstale":"2024-03-30T13:11:01.856071+0000","last_undegraded":"2024-03-30T13:11:01.856071+0000","last_fullsized":"2024-03-30T13:11:01.856071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855446+0000","last_change":"2024-03-30T13:10:55.965085+0000","last_active":"2024-03-30T13:11:01.855446+0000","last_peered":"2024-03-30T13:11:01.855446+0000","last_clean":"2024-03-30T13:11:01.855446+0000","last_became_active":"2024-03-30T13:10:55.964573+0000","last_became_peered":"2024-03-30T13:10:55.964573+0000","last_unstale":"2024-03-30T13:11:01.855446+0000","last_undegraded":"2024-03-30T13:11:01.855446+0000","last_fullsized":"2024-03-30T13:11:01.855446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:56:01.049552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857815+0000","last_change":"2024-03-30T13:11:00.020015+0000","last_active":"2024-03-30T13:11:01.857815+0000","last_peered":"2024-03-30T13:11:01.857815+0000","last_clean":"2024-03-30T13:11:01.857815+0000","last_became_active":"2024-03-30T13:11:00.019563+0000","last_became_peered":"2024-03-30T13:11:00.019563+0000","last_unstale":"2024-03-30T13:11:01.857815+0000","last_undegraded":"2024-03-30T13:11:01.857815+0000","last_fullsized":"2024-03-30T13:11:01.857815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:44.852627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854082+0000","last_change":"2024-03-30T13:10:55.972256+0000","last_active":"2024-03-30T13:11:01.854082+0000","last_peered":"2024-03-30T13:11:01.854082+0000","last_clean":"2024-03-30T13:11:01.854082+0000","last_became_active":"2024-03-30T13:10:55.971884+0000","last_became_peered":"2024-03-30T13:10:55.971884+0000","last_unstale":"2024-03-30T13:11:01.854082+0000","last_undegraded":"2024-03-30T13:11:01.854082+0000","last_fullsized":"2024-03-30T13:11:01.854082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:47:49.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854680+0000","last_change":"2024-03-30T13:10:54.932797+0000","last_active":"2024-03-30T13:11:01.854680+0000","last_peered":"2024-03-30T13:11:01.854680+0000","last_clean":"2024-03-30T13:11:01.854680+0000","last_became_active":"2024-03-30T13:10:54.932392+0000","last_became_peered":"2024-03-30T13:10:54.932392+0000","last_unstale":"2024-03-30T13:11:01.854680+0000","last_undegraded":"2024-03-30T13:11:01.854680+0000","last_fullsized":"2024-03-30T13:11:01.854680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855304+0000","last_change":"2024-03-30T13:11:00.005113+0000","last_active":"2024-03-30T13:11:01.855304+0000","last_peered":"2024-03-30T13:11:01.855304+0000","last_clean":"2024-03-30T13:11:01.855304+0000","last_became_active":"2024-03-30T13:11:00.004701+0000","last_became_peered":"2024-03-30T13:11:00.004701+0000","last_unstale":"2024-03-30T13:11:01.855304+0000","last_undegraded":"2024-03-30T13:11:01.855304+0000","last_fullsized":"2024-03-30T13:11:01.855304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:12:46.188048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859494+0000","last_change":"2024-03-30T13:10:55.993733+0000","last_active":"2024-03-30T13:11:01.859494+0000","last_peered":"2024-03-30T13:11:01.859494+0000","last_clean":"2024-03-30T13:11:01.859494+0000","last_became_active":"2024-03-30T13:10:55.993012+0000","last_became_peered":"2024-03-30T13:10:55.993012+0000","last_unstale":"2024-03-30T13:11:01.859494+0000","last_undegraded":"2024-03-30T13:11:01.859494+0000","last_fullsized":"2024-03-30T13:11:01.859494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980295+0000","last_change":"2024-03-30T13:10:54.947616+0000","last_active":"2024-03-30T13:11:00.980295+0000","last_peered":"2024-03-30T13:11:00.980295+0000","last_clean":"2024-03-30T13:11:00.980295+0000","last_became_active":"2024-03-30T13:10:54.946695+0000","last_became_peered":"2024-03-30T13:10:54.946695+0000","last_unstale":"2024-03-30T13:11:00.980295+0000","last_undegraded":"2024-03-30T13:11:00.980295+0000","last_fullsized":"2024-03-30T13:11:00.980295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855501+0000","last_change":"2024-03-30T13:11:00.019814+0000","last_active":"2024-03-30T13:11:01.855501+0000","last_peered":"2024-03-30T13:11:01.855501+0000","last_clean":"2024-03-30T13:11:01.855501+0000","last_became_active":"2024-03-30T13:11:00.019359+0000","last_became_peered":"2024-03-30T13:11:00.019359+0000","last_unstale":"2024-03-30T13:11:01.855501+0000","last_undegraded":"2024-03-30T13:11:01.855501+0000","last_fullsized":"2024-03-30T13:11:01.855501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:40:01.204781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855194+0000","last_change":"2024-03-30T13:10:56.004578+0000","last_active":"2024-03-30T13:11:01.855194+0000","last_peered":"2024-03-30T13:11:01.855194+0000","last_clean":"2024-03-30T13:11:01.855194+0000","last_became_active":"2024-03-30T13:10:56.003914+0000","last_became_peered":"2024-03-30T13:10:56.003914+0000","last_unstale":"2024-03-30T13:11:01.855194+0000","last_undegraded":"2024-03-30T13:11:01.855194+0000","last_fullsized":"2024-03-30T13:11:01.855194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856085+0000","last_change":"2024-03-30T13:10:54.948212+0000","last_active":"2024-03-30T13:11:01.856085+0000","last_peered":"2024-03-30T13:11:01.856085+0000","last_clean":"2024-03-30T13:11:01.856085+0000","last_became_active":"2024-03-30T13:10:54.947336+0000","last_became_peered":"2024-03-30T13:10:54.947336+0000","last_unstale":"2024-03-30T13:11:01.856085+0000","last_undegraded":"2024-03-30T13:11:01.856085+0000","last_fullsized":"2024-03-30T13:11:01.856085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859925+0000","last_change":"2024-03-30T13:11:00.051626+0000","last_active":"2024-03-30T13:11:01.859925+0000","last_peered":"2024-03-30T13:11:01.859925+0000","last_clean":"2024-03-30T13:11:01.859925+0000","last_became_active":"2024-03-30T13:11:00.050798+0000","last_became_peered":"2024-03-30T13:11:00.050798+0000","last_unstale":"2024-03-30T13:11:01.859925+0000","last_undegraded":"2024-03-30T13:11:01.859925+0000","last_fullsized":"2024-03-30T13:11:01.859925+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:15.648737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853752+0000","last_change":"2024-03-30T13:10:55.965723+0000","last_active":"2024-03-30T13:11:01.853752+0000","last_peered":"2024-03-30T13:11:01.853752+0000","last_clean":"2024-03-30T13:11:01.853752+0000","last_became_active":"2024-03-30T13:10:55.965433+0000","last_became_peered":"2024-03-30T13:10:55.965433+0000","last_unstale":"2024-03-30T13:11:01.853752+0000","last_undegraded":"2024-03-30T13:11:01.853752+0000","last_fullsized":"2024-03-30T13:11:01.853752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854998+0000","last_change":"2024-03-30T13:10:54.940390+0000","last_active":"2024-03-30T13:11:01.854998+0000","last_peered":"2024-03-30T13:11:01.854998+0000","last_clean":"2024-03-30T13:11:01.854998+0000","last_became_active":"2024-03-30T13:10:54.939980+0000","last_became_peered":"2024-03-30T13:10:54.939980+0000","last_unstale":"2024-03-30T13:11:01.854998+0000","last_undegraded":"2024-03-30T13:11:01.854998+0000","last_fullsized":"2024-03-30T13:11:01.854998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858329+0000","last_change":"2024-03-30T13:11:00.019922+0000","last_active":"2024-03-30T13:11:01.858329+0000","last_peered":"2024-03-30T13:11:01.858329+0000","last_clean":"2024-03-30T13:11:01.858329+0000","last_became_active":"2024-03-30T13:11:00.019491+0000","last_became_peered":"2024-03-30T13:11:00.019491+0000","last_unstale":"2024-03-30T13:11:01.858329+0000","last_undegraded":"2024-03-30T13:11:01.858329+0000","last_fullsized":"2024-03-30T13:11:01.858329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856375+0000","last_change":"2024-03-30T13:10:55.968355+0000","last_active":"2024-03-30T13:11:01.856375+0000","last_peered":"2024-03-30T13:11:01.856375+0000","last_clean":"2024-03-30T13:11:01.856375+0000","last_became_active":"2024-03-30T13:10:55.965410+0000","last_became_peered":"2024-03-30T13:10:55.965410+0000","last_unstale":"2024-03-30T13:11:01.856375+0000","last_undegraded":"2024-03-30T13:11:01.856375+0000","last_fullsized":"2024-03-30T13:11:01.856375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:47:03.240883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855058+0000","last_change":"2024-03-30T13:10:54.944101+0000","last_active":"2024-03-30T13:11:01.855058+0000","last_peered":"2024-03-30T13:11:01.855058+0000","last_clean":"2024-03-30T13:11:01.855058+0000","last_became_active":"2024-03-30T13:10:54.943588+0000","last_became_peered":"2024-03-30T13:10:54.943588+0000","last_unstale":"2024-03-30T13:11:01.855058+0000","last_undegraded":"2024-03-30T13:11:01.855058+0000","last_fullsized":"2024-03-30T13:11:01.855058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857849+0000","last_change":"2024-03-30T13:11:00.009582+0000","last_active":"2024-03-30T13:11:01.857849+0000","last_peered":"2024-03-30T13:11:01.857849+0000","last_clean":"2024-03-30T13:11:01.857849+0000","last_became_active":"2024-03-30T13:11:00.009033+0000","last_became_peered":"2024-03-30T13:11:00.009033+0000","last_unstale":"2024-03-30T13:11:01.857849+0000","last_undegraded":"2024-03-30T13:11:01.857849+0000","last_fullsized":"2024-03-30T13:11:01.857849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856879+0000","last_change":"2024-03-30T13:10:56.002735+0000","last_active":"2024-03-30T13:11:01.856879+0000","last_peered":"2024-03-30T13:11:01.856879+0000","last_clean":"2024-03-30T13:11:01.856879+0000","last_became_active":"2024-03-30T13:10:56.001973+0000","last_became_peered":"2024-03-30T13:10:56.001973+0000","last_unstale":"2024-03-30T13:11:01.856879+0000","last_undegraded":"2024-03-30T13:11:01.856879+0000","last_fullsized":"2024-03-30T13:11:01.856879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856685+0000","last_change":"2024-03-30T13:10:54.947650+0000","last_active":"2024-03-30T13:11:01.856685+0000","last_peered":"2024-03-30T13:11:01.856685+0000","last_clean":"2024-03-30T13:11:01.856685+0000","last_became_active":"2024-03-30T13:10:54.946378+0000","last_became_peered":"2024-03-30T13:10:54.946378+0000","last_unstale":"2024-03-30T13:11:01.856685+0000","last_undegraded":"2024-03-30T13:11:01.856685+0000","last_fullsized":"2024-03-30T13:11:01.856685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855829+0000","last_change":"2024-03-30T13:11:00.031425+0000","last_active":"2024-03-30T13:11:01.855829+0000","last_peered":"2024-03-30T13:11:01.855829+0000","last_clean":"2024-03-30T13:11:01.855829+0000","last_became_active":"2024-03-30T13:11:00.030689+0000","last_became_peered":"2024-03-30T13:11:00.030689+0000","last_unstale":"2024-03-30T13:11:01.855829+0000","last_undegraded":"2024-03-30T13:11:01.855829+0000","last_fullsized":"2024-03-30T13:11:01.855829+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:50.405876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981455+0000","last_change":"2024-03-30T13:10:55.991469+0000","last_active":"2024-03-30T13:11:00.981455+0000","last_peered":"2024-03-30T13:11:00.981455+0000","last_clean":"2024-03-30T13:11:00.981455+0000","last_became_active":"2024-03-30T13:10:55.990596+0000","last_became_peered":"2024-03-30T13:10:55.990596+0000","last_unstale":"2024-03-30T13:11:00.981455+0000","last_undegraded":"2024-03-30T13:11:00.981455+0000","last_fullsized":"2024-03-30T13:11:00.981455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856131+0000","last_change":"2024-03-30T13:10:54.945048+0000","last_active":"2024-03-30T13:11:01.856131+0000","last_peered":"2024-03-30T13:11:01.856131+0000","last_clean":"2024-03-30T13:11:01.856131+0000","last_became_active":"2024-03-30T13:10:54.944552+0000","last_became_peered":"2024-03-30T13:10:54.944552+0000","last_unstale":"2024-03-30T13:11:01.856131+0000","last_undegraded":"2024-03-30T13:11:01.856131+0000","last_fullsized":"2024-03-30T13:11:01.856131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:08.826146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980755+0000","last_change":"2024-03-30T13:11:00.038523+0000","last_active":"2024-03-30T13:11:00.980755+0000","last_peered":"2024-03-30T13:11:00.980755+0000","last_clean":"2024-03-30T13:11:00.980755+0000","last_became_active":"2024-03-30T13:11:00.037276+0000","last_became_peered":"2024-03-30T13:11:00.037276+0000","last_unstale":"2024-03-30T13:11:00.980755+0000","last_undegraded":"2024-03-30T13:11:00.980755+0000","last_fullsized":"2024-03-30T13:11:00.980755+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858113+0000","last_change":"2024-03-30T13:10:55.992831+0000","last_active":"2024-03-30T13:11:01.858113+0000","last_peered":"2024-03-30T13:11:01.858113+0000","last_clean":"2024-03-30T13:11:01.858113+0000","last_became_active":"2024-03-30T13:10:55.992521+0000","last_became_peered":"2024-03-30T13:10:55.992521+0000","last_unstale":"2024-03-30T13:11:01.858113+0000","last_undegraded":"2024-03-30T13:11:01.858113+0000","last_fullsized":"2024-03-30T13:11:01.858113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979972+0000","last_change":"2024-03-30T13:10:54.952220+0000","last_active":"2024-03-30T13:11:00.979972+0000","last_peered":"2024-03-30T13:11:00.979972+0000","last_clean":"2024-03-30T13:11:00.979972+0000","last_became_active":"2024-03-30T13:10:54.951484+0000","last_became_peered":"2024-03-30T13:10:54.951484+0000","last_unstale":"2024-03-30T13:11:00.979972+0000","last_undegraded":"2024-03-30T13:11:00.979972+0000","last_fullsized":"2024-03-30T13:11:00.979972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:11:24.184946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979274+0000","last_change":"2024-03-30T13:11:00.043402+0000","last_active":"2024-03-30T13:11:00.979274+0000","last_peered":"2024-03-30T13:11:00.979274+0000","last_clean":"2024-03-30T13:11:00.979274+0000","last_became_active":"2024-03-30T13:11:00.040341+0000","last_became_peered":"2024-03-30T13:11:00.040341+0000","last_unstale":"2024-03-30T13:11:00.979274+0000","last_undegraded":"2024-03-30T13:11:00.979274+0000","last_fullsized":"2024-03-30T13:11:00.979274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855368+0000","last_change":"2024-03-30T13:10:55.958140+0000","last_active":"2024-03-30T13:11:01.855368+0000","last_peered":"2024-03-30T13:11:01.855368+0000","last_clean":"2024-03-30T13:11:01.855368+0000","last_became_active":"2024-03-30T13:10:55.957860+0000","last_became_peered":"2024-03-30T13:10:55.957860+0000","last_unstale":"2024-03-30T13:11:01.855368+0000","last_undegraded":"2024-03-30T13:11:01.855368+0000","last_fullsized":"2024-03-30T13:11:01.855368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:54.477311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854336+0000","last_change":"2024-03-30T13:10:54.945433+0000","last_active":"2024-03-30T13:11:01.854336+0000","last_peered":"2024-03-30T13:11:01.854336+0000","last_clean":"2024-03-30T13:11:01.854336+0000","last_became_active":"2024-03-30T13:10:54.945133+0000","last_became_peered":"2024-03-30T13:10:54.945133+0000","last_unstale":"2024-03-30T13:11:01.854336+0000","last_undegraded":"2024-03-30T13:11:01.854336+0000","last_fullsized":"2024-03-30T13:11:01.854336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:59.514214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980453+0000","last_change":"2024-03-30T13:11:00.044030+0000","last_active":"2024-03-30T13:11:00.980453+0000","last_peered":"2024-03-30T13:11:00.980453+0000","last_clean":"2024-03-30T13:11:00.980453+0000","last_became_active":"2024-03-30T13:11:00.037702+0000","last_became_peered":"2024-03-30T13:11:00.037702+0000","last_unstale":"2024-03-30T13:11:00.980453+0000","last_undegraded":"2024-03-30T13:11:00.980453+0000","last_fullsized":"2024-03-30T13:11:00.980453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979645+0000","last_change":"2024-03-30T13:10:55.962616+0000","last_active":"2024-03-30T13:11:00.979645+0000","last_peered":"2024-03-30T13:11:00.979645+0000","last_clean":"2024-03-30T13:11:00.979645+0000","last_became_active":"2024-03-30T13:10:55.961327+0000","last_became_peered":"2024-03-30T13:10:55.961327+0000","last_unstale":"2024-03-30T13:11:00.979645+0000","last_undegraded":"2024-03-30T13:11:00.979645+0000","last_fullsized":"2024-03-30T13:11:00.979645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:17.511817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858373+0000","last_change":"2024-03-30T13:10:54.942851+0000","last_active":"2024-03-30T13:11:01.858373+0000","last_peered":"2024-03-30T13:11:01.858373+0000","last_clean":"2024-03-30T13:11:01.858373+0000","last_became_active":"2024-03-30T13:10:54.939805+0000","last_became_peered":"2024-03-30T13:10:54.939805+0000","last_unstale":"2024-03-30T13:11:01.858373+0000","last_undegraded":"2024-03-30T13:11:01.858373+0000","last_fullsized":"2024-03-30T13:11:01.858373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:24.569683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982246+0000","last_change":"2024-03-30T13:11:00.040718+0000","last_active":"2024-03-30T13:11:00.982246+0000","last_peered":"2024-03-30T13:11:00.982246+0000","last_clean":"2024-03-30T13:11:00.982246+0000","last_became_active":"2024-03-30T13:11:00.039372+0000","last_became_peered":"2024-03-30T13:11:00.039372+0000","last_unstale":"2024-03-30T13:11:00.982246+0000","last_undegraded":"2024-03-30T13:11:00.982246+0000","last_fullsized":"2024-03-30T13:11:00.982246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:26.483027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979060+0000","last_change":"2024-03-30T13:10:55.984056+0000","last_active":"2024-03-30T13:11:00.979060+0000","last_peered":"2024-03-30T13:11:00.979060+0000","last_clean":"2024-03-30T13:11:00.979060+0000","last_became_active":"2024-03-30T13:10:55.974450+0000","last_became_peered":"2024-03-30T13:10:55.974450+0000","last_unstale":"2024-03-30T13:11:00.979060+0000","last_undegraded":"2024-03-30T13:11:00.979060+0000","last_fullsized":"2024-03-30T13:11:00.979060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853918+0000","last_change":"2024-03-30T13:10:54.942183+0000","last_active":"2024-03-30T13:11:01.853918+0000","last_peered":"2024-03-30T13:11:01.853918+0000","last_clean":"2024-03-30T13:11:01.853918+0000","last_became_active":"2024-03-30T13:10:54.941923+0000","last_became_peered":"2024-03-30T13:10:54.941923+0000","last_unstale":"2024-03-30T13:11:01.853918+0000","last_undegraded":"2024-03-30T13:11:01.853918+0000","last_fullsized":"2024-03-30T13:11:01.853918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:39.803698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983242+0000","last_change":"2024-03-30T13:11:00.024121+0000","last_active":"2024-03-30T13:11:00.983242+0000","last_peered":"2024-03-30T13:11:00.983242+0000","last_clean":"2024-03-30T13:11:00.983242+0000","last_became_active":"2024-03-30T13:11:00.022137+0000","last_became_peered":"2024-03-30T13:11:00.022137+0000","last_unstale":"2024-03-30T13:11:00.983242+0000","last_undegraded":"2024-03-30T13:11:00.983242+0000","last_fullsized":"2024-03-30T13:11:00.983242+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:08:16.219711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856025+0000","last_change":"2024-03-30T13:10:55.978563+0000","last_active":"2024-03-30T13:11:01.856025+0000","last_peered":"2024-03-30T13:11:01.856025+0000","last_clean":"2024-03-30T13:11:01.856025+0000","last_became_active":"2024-03-30T13:10:55.973372+0000","last_became_peered":"2024-03-30T13:10:55.973372+0000","last_unstale":"2024-03-30T13:11:01.856025+0000","last_undegraded":"2024-03-30T13:11:01.856025+0000","last_fullsized":"2024-03-30T13:11:01.856025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854645+0000","last_change":"2024-03-30T13:10:54.932742+0000","last_active":"2024-03-30T13:11:01.854645+0000","last_peered":"2024-03-30T13:11:01.854645+0000","last_clean":"2024-03-30T13:11:01.854645+0000","last_became_active":"2024-03-30T13:10:54.932326+0000","last_became_peered":"2024-03-30T13:10:54.932326+0000","last_unstale":"2024-03-30T13:11:01.854645+0000","last_undegraded":"2024-03-30T13:11:01.854645+0000","last_fullsized":"2024-03-30T13:11:01.854645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977500+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:11:00.977500+0000","last_peered":"2024-03-30T13:11:00.977500+0000","last_clean":"2024-03-30T13:11:00.977500+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:11:00.977500+0000","last_undegraded":"2024-03-30T13:11:00.977500+0000","last_fullsized":"2024-03-30T13:11:00.977500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856097+0000","last_change":"2024-03-30T13:10:59.992352+0000","last_active":"2024-03-30T13:11:01.856097+0000","last_peered":"2024-03-30T13:11:01.856097+0000","last_clean":"2024-03-30T13:11:01.856097+0000","last_became_active":"2024-03-30T13:10:59.991915+0000","last_became_peered":"2024-03-30T13:10:59.991915+0000","last_unstale":"2024-03-30T13:11:01.856097+0000","last_undegraded":"2024-03-30T13:11:01.856097+0000","last_fullsized":"2024-03-30T13:11:01.856097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:13:07.419696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855684+0000","last_change":"2024-03-30T13:10:55.971288+0000","last_active":"2024-03-30T13:11:01.855684+0000","last_peered":"2024-03-30T13:11:01.855684+0000","last_clean":"2024-03-30T13:11:01.855684+0000","last_became_active":"2024-03-30T13:10:55.970980+0000","last_became_peered":"2024-03-30T13:10:55.970980+0000","last_unstale":"2024-03-30T13:11:01.855684+0000","last_undegraded":"2024-03-30T13:11:01.855684+0000","last_fullsized":"2024-03-30T13:11:01.855684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:57.143218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980646+0000","last_change":"2024-03-30T13:10:54.946952+0000","last_active":"2024-03-30T13:11:00.980646+0000","last_peered":"2024-03-30T13:11:00.980646+0000","last_clean":"2024-03-30T13:11:00.980646+0000","last_became_active":"2024-03-30T13:10:54.946235+0000","last_became_peered":"2024-03-30T13:10:54.946235+0000","last_unstale":"2024-03-30T13:11:00.980646+0000","last_undegraded":"2024-03-30T13:11:00.980646+0000","last_fullsized":"2024-03-30T13:11:00.980646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857845+0000","last_change":"2024-03-30T13:11:00.017769+0000","last_active":"2024-03-30T13:11:01.857845+0000","last_peered":"2024-03-30T13:11:01.857845+0000","last_clean":"2024-03-30T13:11:01.857845+0000","last_became_active":"2024-03-30T13:11:00.017317+0000","last_became_peered":"2024-03-30T13:11:00.017317+0000","last_unstale":"2024-03-30T13:11:01.857845+0000","last_undegraded":"2024-03-30T13:11:01.857845+0000","last_fullsized":"2024-03-30T13:11:01.857845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:01.211799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.963559+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.963354+0000","last_became_peered":"2024-03-30T13:10:55.963354+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981372+0000","last_change":"2024-03-30T13:10:54.942041+0000","last_active":"2024-03-30T13:11:00.981372+0000","last_peered":"2024-03-30T13:11:00.981372+0000","last_clean":"2024-03-30T13:11:00.981372+0000","last_became_active":"2024-03-30T13:10:54.940995+0000","last_became_peered":"2024-03-30T13:10:54.940995+0000","last_unstale":"2024-03-30T13:11:00.981372+0000","last_undegraded":"2024-03-30T13:11:00.981372+0000","last_fullsized":"2024-03-30T13:11:00.981372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:54:21.415837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982091+0000","last_change":"2024-03-30T13:11:00.010520+0000","last_active":"2024-03-30T13:11:00.982091+0000","last_peered":"2024-03-30T13:11:00.982091+0000","last_clean":"2024-03-30T13:11:00.982091+0000","last_became_active":"2024-03-30T13:11:00.009785+0000","last_became_peered":"2024-03-30T13:11:00.009785+0000","last_unstale":"2024-03-30T13:11:00.982091+0000","last_undegraded":"2024-03-30T13:11:00.982091+0000","last_fullsized":"2024-03-30T13:11:00.982091+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:13:10.451556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856333+0000","last_change":"2024-03-30T13:10:55.979699+0000","last_active":"2024-03-30T13:11:01.856333+0000","last_peered":"2024-03-30T13:11:01.856333+0000","last_clean":"2024-03-30T13:11:01.856333+0000","last_became_active":"2024-03-30T13:10:55.979427+0000","last_became_peered":"2024-03-30T13:10:55.979427+0000","last_unstale":"2024-03-30T13:11:01.856333+0000","last_undegraded":"2024-03-30T13:11:01.856333+0000","last_fullsized":"2024-03-30T13:11:01.856333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856586+0000","last_change":"2024-03-30T13:10:54.945823+0000","last_active":"2024-03-30T13:11:01.856586+0000","last_peered":"2024-03-30T13:11:01.856586+0000","last_clean":"2024-03-30T13:11:01.856586+0000","last_became_active":"2024-03-30T13:10:54.944624+0000","last_became_peered":"2024-03-30T13:10:54.944624+0000","last_unstale":"2024-03-30T13:11:01.856586+0000","last_undegraded":"2024-03-30T13:11:01.856586+0000","last_fullsized":"2024-03-30T13:11:01.856586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853761+0000","last_change":"2024-03-30T13:11:00.002695+0000","last_active":"2024-03-30T13:11:01.853761+0000","last_peered":"2024-03-30T13:11:01.853761+0000","last_clean":"2024-03-30T13:11:01.853761+0000","last_became_active":"2024-03-30T13:11:00.002251+0000","last_became_peered":"2024-03-30T13:11:00.002251+0000","last_unstale":"2024-03-30T13:11:01.853761+0000","last_undegraded":"2024-03-30T13:11:01.853761+0000","last_fullsized":"2024-03-30T13:11:01.853761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:23:48.725109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858389+0000","last_change":"2024-03-30T13:10:55.966115+0000","last_active":"2024-03-30T13:11:01.858389+0000","last_peered":"2024-03-30T13:11:01.858389+0000","last_clean":"2024-03-30T13:11:01.858389+0000","last_became_active":"2024-03-30T13:10:55.965541+0000","last_became_peered":"2024-03-30T13:10:55.965541+0000","last_unstale":"2024-03-30T13:11:01.858389+0000","last_undegraded":"2024-03-30T13:11:01.858389+0000","last_fullsized":"2024-03-30T13:11:01.858389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856413+0000","last_change":"2024-03-30T13:10:54.945660+0000","last_active":"2024-03-30T13:11:01.856413+0000","last_peered":"2024-03-30T13:11:01.856413+0000","last_clean":"2024-03-30T13:11:01.856413+0000","last_became_active":"2024-03-30T13:10:54.944892+0000","last_became_peered":"2024-03-30T13:10:54.944892+0000","last_unstale":"2024-03-30T13:11:01.856413+0000","last_undegraded":"2024-03-30T13:11:01.856413+0000","last_fullsized":"2024-03-30T13:11:01.856413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977650+0000","last_change":"2024-03-30T13:11:00.014434+0000","last_active":"2024-03-30T13:11:00.977650+0000","last_peered":"2024-03-30T13:11:00.977650+0000","last_clean":"2024-03-30T13:11:00.977650+0000","last_became_active":"2024-03-30T13:11:00.013995+0000","last_became_peered":"2024-03-30T13:11:00.013995+0000","last_unstale":"2024-03-30T13:11:00.977650+0000","last_undegraded":"2024-03-30T13:11:00.977650+0000","last_fullsized":"2024-03-30T13:11:00.977650+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:13.896277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853679+0000","last_change":"2024-03-30T13:10:55.960324+0000","last_active":"2024-03-30T13:11:01.853679+0000","last_peered":"2024-03-30T13:11:01.853679+0000","last_clean":"2024-03-30T13:11:01.853679+0000","last_became_active":"2024-03-30T13:10:55.958877+0000","last_became_peered":"2024-03-30T13:10:55.958877+0000","last_unstale":"2024-03-30T13:11:01.853679+0000","last_undegraded":"2024-03-30T13:11:01.853679+0000","last_fullsized":"2024-03-30T13:11:01.853679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855512+0000","last_change":"2024-03-30T13:10:54.943074+0000","last_active":"2024-03-30T13:11:01.855512+0000","last_peered":"2024-03-30T13:11:01.855512+0000","last_clean":"2024-03-30T13:11:01.855512+0000","last_became_active":"2024-03-30T13:10:54.941870+0000","last_became_peered":"2024-03-30T13:10:54.941870+0000","last_unstale":"2024-03-30T13:11:01.855512+0000","last_undegraded":"2024-03-30T13:11:01.855512+0000","last_fullsized":"2024-03-30T13:11:01.855512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983714+0000","last_change":"2024-03-30T13:11:00.023257+0000","last_active":"2024-03-30T13:11:00.983714+0000","last_peered":"2024-03-30T13:11:00.983714+0000","last_clean":"2024-03-30T13:11:00.983714+0000","last_became_active":"2024-03-30T13:11:00.020654+0000","last_became_peered":"2024-03-30T13:11:00.020654+0000","last_unstale":"2024-03-30T13:11:00.983714+0000","last_undegraded":"2024-03-30T13:11:00.983714+0000","last_fullsized":"2024-03-30T13:11:00.983714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:16:15.446866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977511+0000","last_change":"2024-03-30T13:10:55.970389+0000","last_active":"2024-03-30T13:11:00.977511+0000","last_peered":"2024-03-30T13:11:00.977511+0000","last_clean":"2024-03-30T13:11:00.977511+0000","last_became_active":"2024-03-30T13:10:55.969686+0000","last_became_peered":"2024-03-30T13:10:55.969686+0000","last_unstale":"2024-03-30T13:11:00.977511+0000","last_undegraded":"2024-03-30T13:11:00.977511+0000","last_fullsized":"2024-03-30T13:11:00.977511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855488+0000","last_change":"2024-03-30T13:10:54.945121+0000","last_active":"2024-03-30T13:11:01.855488+0000","last_peered":"2024-03-30T13:11:01.855488+0000","last_clean":"2024-03-30T13:11:01.855488+0000","last_became_active":"2024-03-30T13:10:54.944583+0000","last_became_peered":"2024-03-30T13:10:54.944583+0000","last_unstale":"2024-03-30T13:11:01.855488+0000","last_undegraded":"2024-03-30T13:11:01.855488+0000","last_fullsized":"2024-03-30T13:11:01.855488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981873+0000","last_change":"2024-03-30T13:11:00.023347+0000","last_active":"2024-03-30T13:11:00.981873+0000","last_peered":"2024-03-30T13:11:00.981873+0000","last_clean":"2024-03-30T13:11:00.981873+0000","last_became_active":"2024-03-30T13:11:00.021082+0000","last_became_peered":"2024-03-30T13:11:00.021082+0000","last_unstale":"2024-03-30T13:11:00.981873+0000","last_undegraded":"2024-03-30T13:11:00.981873+0000","last_fullsized":"2024-03-30T13:11:00.981873+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:39.699905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854454+0000","last_change":"2024-03-30T13:10:55.964652+0000","last_active":"2024-03-30T13:11:01.854454+0000","last_peered":"2024-03-30T13:11:01.854454+0000","last_clean":"2024-03-30T13:11:01.854454+0000","last_became_active":"2024-03-30T13:10:55.964363+0000","last_became_peered":"2024-03-30T13:10:55.964363+0000","last_unstale":"2024-03-30T13:11:01.854454+0000","last_undegraded":"2024-03-30T13:11:01.854454+0000","last_fullsized":"2024-03-30T13:11:01.854454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:17.023431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854282+0000","last_change":"2024-03-30T13:10:54.936477+0000","last_active":"2024-03-30T13:11:01.854282+0000","last_peered":"2024-03-30T13:11:01.854282+0000","last_clean":"2024-03-30T13:11:01.854282+0000","last_became_active":"2024-03-30T13:10:54.935689+0000","last_became_peered":"2024-03-30T13:10:54.935689+0000","last_unstale":"2024-03-30T13:11:01.854282+0000","last_undegraded":"2024-03-30T13:11:01.854282+0000","last_fullsized":"2024-03-30T13:11:01.854282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:28.044884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980099+0000","last_change":"2024-03-30T13:11:00.044614+0000","last_active":"2024-03-30T13:11:00.980099+0000","last_peered":"2024-03-30T13:11:00.980099+0000","last_clean":"2024-03-30T13:11:00.980099+0000","last_became_active":"2024-03-30T13:11:00.039909+0000","last_became_peered":"2024-03-30T13:11:00.039909+0000","last_unstale":"2024-03-30T13:11:00.980099+0000","last_undegraded":"2024-03-30T13:11:00.980099+0000","last_fullsized":"2024-03-30T13:11:00.980099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979980+0000","last_change":"2024-03-30T13:10:55.985354+0000","last_active":"2024-03-30T13:11:00.979980+0000","last_peered":"2024-03-30T13:11:00.979980+0000","last_clean":"2024-03-30T13:11:00.979980+0000","last_became_active":"2024-03-30T13:10:55.974918+0000","last_became_peered":"2024-03-30T13:10:55.974918+0000","last_unstale":"2024-03-30T13:11:00.979980+0000","last_undegraded":"2024-03-30T13:11:00.979980+0000","last_fullsized":"2024-03-30T13:11:00.979980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:29.402254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859297+0000","last_change":"2024-03-30T13:10:54.955637+0000","last_active":"2024-03-30T13:11:01.859297+0000","last_peered":"2024-03-30T13:11:01.859297+0000","last_clean":"2024-03-30T13:11:01.859297+0000","last_became_active":"2024-03-30T13:10:54.949281+0000","last_became_peered":"2024-03-30T13:10:54.949281+0000","last_unstale":"2024-03-30T13:11:01.859297+0000","last_undegraded":"2024-03-30T13:11:01.859297+0000","last_fullsized":"2024-03-30T13:11:01.859297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855707+0000","last_change":"2024-03-30T13:11:00.015834+0000","last_active":"2024-03-30T13:11:01.855707+0000","last_peered":"2024-03-30T13:11:01.855707+0000","last_clean":"2024-03-30T13:11:01.855707+0000","last_became_active":"2024-03-30T13:11:00.015344+0000","last_became_peered":"2024-03-30T13:11:00.015344+0000","last_unstale":"2024-03-30T13:11:01.855707+0000","last_undegraded":"2024-03-30T13:11:01.855707+0000","last_fullsized":"2024-03-30T13:11:01.855707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:43.288754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855890+0000","last_change":"2024-03-30T13:10:55.993807+0000","last_active":"2024-03-30T13:11:01.855890+0000","last_peered":"2024-03-30T13:11:01.855890+0000","last_clean":"2024-03-30T13:11:01.855890+0000","last_became_active":"2024-03-30T13:10:55.993589+0000","last_became_peered":"2024-03-30T13:10:55.993589+0000","last_unstale":"2024-03-30T13:11:01.855890+0000","last_undegraded":"2024-03-30T13:11:01.855890+0000","last_fullsized":"2024-03-30T13:11:01.855890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:57.150236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:06.622780+0000","last_change":"2024-03-30T13:10:54.939893+0000","last_active":"2024-03-30T13:11:06.622780+0000","last_peered":"2024-03-30T13:11:06.622780+0000","last_clean":"2024-03-30T13:11:06.622780+0000","last_became_active":"2024-03-30T13:10:54.939427+0000","last_became_peered":"2024-03-30T13:10:54.939427+0000","last_unstale":"2024-03-30T13:11:06.622780+0000","last_undegraded":"2024-03-30T13:11:06.622780+0000","last_fullsized":"2024-03-30T13:11:06.622780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:07.195072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855852+0000","last_change":"2024-03-30T13:11:00.013757+0000","last_active":"2024-03-30T13:11:01.855852+0000","last_peered":"2024-03-30T13:11:01.855852+0000","last_clean":"2024-03-30T13:11:01.855852+0000","last_became_active":"2024-03-30T13:11:00.012615+0000","last_became_peered":"2024-03-30T13:11:00.012615+0000","last_unstale":"2024-03-30T13:11:01.855852+0000","last_undegraded":"2024-03-30T13:11:01.855852+0000","last_fullsized":"2024-03-30T13:11:01.855852+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:02:00.559037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853971+0000","last_change":"2024-03-30T13:10:55.957241+0000","last_active":"2024-03-30T13:11:01.853971+0000","last_peered":"2024-03-30T13:11:01.853971+0000","last_clean":"2024-03-30T13:11:01.853971+0000","last_became_active":"2024-03-30T13:10:55.956872+0000","last_became_peered":"2024-03-30T13:10:55.956872+0000","last_unstale":"2024-03-30T13:11:01.853971+0000","last_undegraded":"2024-03-30T13:11:01.853971+0000","last_fullsized":"2024-03-30T13:11:01.853971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:00.663013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859629+0000","last_change":"2024-03-30T13:10:54.947319+0000","last_active":"2024-03-30T13:11:01.859629+0000","last_peered":"2024-03-30T13:11:01.859629+0000","last_clean":"2024-03-30T13:11:01.859629+0000","last_became_active":"2024-03-30T13:10:54.946638+0000","last_became_peered":"2024-03-30T13:10:54.946638+0000","last_unstale":"2024-03-30T13:11:01.859629+0000","last_undegraded":"2024-03-30T13:11:01.859629+0000","last_fullsized":"2024-03-30T13:11:01.859629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858359+0000","last_change":"2024-03-30T13:11:00.032095+0000","last_active":"2024-03-30T13:11:01.858359+0000","last_peered":"2024-03-30T13:11:01.858359+0000","last_clean":"2024-03-30T13:11:01.858359+0000","last_became_active":"2024-03-30T13:11:00.031651+0000","last_became_peered":"2024-03-30T13:11:00.031651+0000","last_unstale":"2024-03-30T13:11:01.858359+0000","last_undegraded":"2024-03-30T13:11:01.858359+0000","last_fullsized":"2024-03-30T13:11:01.858359+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981894+0000","last_change":"2024-03-30T13:10:55.963310+0000","last_active":"2024-03-30T13:11:00.981894+0000","last_peered":"2024-03-30T13:11:00.981894+0000","last_clean":"2024-03-30T13:11:00.981894+0000","last_became_active":"2024-03-30T13:10:55.962608+0000","last_became_peered":"2024-03-30T13:10:55.962608+0000","last_unstale":"2024-03-30T13:11:00.981894+0000","last_undegraded":"2024-03-30T13:11:00.981894+0000","last_fullsized":"2024-03-30T13:11:00.981894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:53.368510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855969+0000","last_change":"2024-03-30T13:10:54.937448+0000","last_active":"2024-03-30T13:11:01.855969+0000","last_peered":"2024-03-30T13:11:01.855969+0000","last_clean":"2024-03-30T13:11:01.855969+0000","last_became_active":"2024-03-30T13:10:54.936836+0000","last_became_peered":"2024-03-30T13:10:54.936836+0000","last_unstale":"2024-03-30T13:11:01.855969+0000","last_undegraded":"2024-03-30T13:11:01.855969+0000","last_fullsized":"2024-03-30T13:11:01.855969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858228+0000","last_change":"2024-03-30T13:11:00.039950+0000","last_active":"2024-03-30T13:11:01.858228+0000","last_peered":"2024-03-30T13:11:01.858228+0000","last_clean":"2024-03-30T13:11:01.858228+0000","last_became_active":"2024-03-30T13:11:00.038871+0000","last_became_peered":"2024-03-30T13:11:00.038871+0000","last_unstale":"2024-03-30T13:11:01.858228+0000","last_undegraded":"2024-03-30T13:11:01.858228+0000","last_fullsized":"2024-03-30T13:11:01.858228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:16.001906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858336+0000","last_change":"2024-03-30T13:10:55.962143+0000","last_active":"2024-03-30T13:11:01.858336+0000","last_peered":"2024-03-30T13:11:01.858336+0000","last_clean":"2024-03-30T13:11:01.858336+0000","last_became_active":"2024-03-30T13:10:55.961186+0000","last_became_peered":"2024-03-30T13:10:55.961186+0000","last_unstale":"2024-03-30T13:11:01.858336+0000","last_undegraded":"2024-03-30T13:11:01.858336+0000","last_fullsized":"2024-03-30T13:11:01.858336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978652+0000","last_change":"2024-03-30T13:10:54.940086+0000","last_active":"2024-03-30T13:11:00.978652+0000","last_peered":"2024-03-30T13:11:00.978652+0000","last_clean":"2024-03-30T13:11:00.978652+0000","last_became_active":"2024-03-30T13:10:54.939206+0000","last_became_peered":"2024-03-30T13:10:54.939206+0000","last_unstale":"2024-03-30T13:11:00.978652+0000","last_undegraded":"2024-03-30T13:11:00.978652+0000","last_fullsized":"2024-03-30T13:11:00.978652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855931+0000","last_change":"2024-03-30T13:11:00.034220+0000","last_active":"2024-03-30T13:11:01.855931+0000","last_peered":"2024-03-30T13:11:01.855931+0000","last_clean":"2024-03-30T13:11:01.855931+0000","last_became_active":"2024-03-30T13:11:00.033211+0000","last_became_peered":"2024-03-30T13:11:00.033211+0000","last_unstale":"2024-03-30T13:11:01.855931+0000","last_undegraded":"2024-03-30T13:11:01.855931+0000","last_fullsized":"2024-03-30T13:11:01.855931+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:22.895535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856526+0000","last_change":"2024-03-30T13:11:00.028139+0000","last_active":"2024-03-30T13:11:01.856526+0000","last_peered":"2024-03-30T13:11:01.856526+0000","last_clean":"2024-03-30T13:11:01.856526+0000","last_became_active":"2024-03-30T13:11:00.027230+0000","last_became_peered":"2024-03-30T13:11:00.027230+0000","last_unstale":"2024-03-30T13:11:01.856526+0000","last_undegraded":"2024-03-30T13:11:01.856526+0000","last_fullsized":"2024-03-30T13:11:01.856526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:04.724576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855891+0000","last_change":"2024-03-30T13:10:55.996825+0000","last_active":"2024-03-30T13:11:01.855891+0000","last_peered":"2024-03-30T13:11:01.855891+0000","last_clean":"2024-03-30T13:11:01.855891+0000","last_became_active":"2024-03-30T13:10:55.977957+0000","last_became_peered":"2024-03-30T13:10:55.977957+0000","last_unstale":"2024-03-30T13:11:01.855891+0000","last_undegraded":"2024-03-30T13:11:01.855891+0000","last_fullsized":"2024-03-30T13:11:01.855891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853813+0000","last_change":"2024-03-30T13:10:54.930843+0000","last_active":"2024-03-30T13:11:01.853813+0000","last_peered":"2024-03-30T13:11:01.853813+0000","last_clean":"2024-03-30T13:11:01.853813+0000","last_became_active":"2024-03-30T13:10:54.930056+0000","last_became_peered":"2024-03-30T13:10:54.930056+0000","last_unstale":"2024-03-30T13:11:01.853813+0000","last_undegraded":"2024-03-30T13:11:01.853813+0000","last_fullsized":"2024-03-30T13:11:01.853813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854932+0000","last_change":"2024-03-30T13:10:59.991481+0000","last_active":"2024-03-30T13:11:01.854932+0000","last_peered":"2024-03-30T13:11:01.854932+0000","last_clean":"2024-03-30T13:11:01.854932+0000","last_became_active":"2024-03-30T13:10:59.991071+0000","last_became_peered":"2024-03-30T13:10:59.991071+0000","last_unstale":"2024-03-30T13:11:01.854932+0000","last_undegraded":"2024-03-30T13:11:01.854932+0000","last_fullsized":"2024-03-30T13:11:01.854932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:29:50.731633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979852+0000","last_change":"2024-03-30T13:10:55.994215+0000","last_active":"2024-03-30T13:11:00.979852+0000","last_peered":"2024-03-30T13:11:00.979852+0000","last_clean":"2024-03-30T13:11:00.979852+0000","last_became_active":"2024-03-30T13:10:55.993553+0000","last_became_peered":"2024-03-30T13:10:55.993553+0000","last_unstale":"2024-03-30T13:11:00.979852+0000","last_undegraded":"2024-03-30T13:11:00.979852+0000","last_fullsized":"2024-03-30T13:11:00.979852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:35.609380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979309+0000","last_change":"2024-03-30T13:10:54.941904+0000","last_active":"2024-03-30T13:11:00.979309+0000","last_peered":"2024-03-30T13:11:00.979309+0000","last_clean":"2024-03-30T13:11:00.979309+0000","last_became_active":"2024-03-30T13:10:54.941223+0000","last_became_peered":"2024-03-30T13:10:54.941223+0000","last_unstale":"2024-03-30T13:11:00.979309+0000","last_undegraded":"2024-03-30T13:11:00.979309+0000","last_fullsized":"2024-03-30T13:11:00.979309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854339+0000","last_change":"2024-03-30T13:10:54.943752+0000","last_active":"2024-03-30T13:11:01.854339+0000","last_peered":"2024-03-30T13:11:01.854339+0000","last_clean":"2024-03-30T13:11:01.854339+0000","last_became_active":"2024-03-30T13:10:54.943525+0000","last_became_peered":"2024-03-30T13:10:54.943525+0000","last_unstale":"2024-03-30T13:11:01.854339+0000","last_undegraded":"2024-03-30T13:11:01.854339+0000","last_fullsized":"2024-03-30T13:11:01.854339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:47.689419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855653+0000","last_change":"2024-03-30T13:10:55.996261+0000","last_active":"2024-03-30T13:11:01.855653+0000","last_peered":"2024-03-30T13:11:01.855653+0000","last_clean":"2024-03-30T13:11:01.855653+0000","last_became_active":"2024-03-30T13:10:55.979906+0000","last_became_peered":"2024-03-30T13:10:55.979906+0000","last_unstale":"2024-03-30T13:11:01.855653+0000","last_undegraded":"2024-03-30T13:11:01.855653+0000","last_fullsized":"2024-03-30T13:11:01.855653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:11.587174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856525+0000","last_change":"2024-03-30T13:11:00.031762+0000","last_active":"2024-03-30T13:11:01.856525+0000","last_peered":"2024-03-30T13:11:01.856525+0000","last_clean":"2024-03-30T13:11:01.856525+0000","last_became_active":"2024-03-30T13:11:00.030859+0000","last_became_peered":"2024-03-30T13:11:00.030859+0000","last_unstale":"2024-03-30T13:11:01.856525+0000","last_undegraded":"2024-03-30T13:11:01.856525+0000","last_fullsized":"2024-03-30T13:11:01.856525+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:21.084995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982601+0000","last_change":"2024-03-30T13:10:54.944088+0000","last_active":"2024-03-30T13:11:00.982601+0000","last_peered":"2024-03-30T13:11:00.982601+0000","last_clean":"2024-03-30T13:11:00.982601+0000","last_became_active":"2024-03-30T13:10:54.943275+0000","last_became_peered":"2024-03-30T13:10:54.943275+0000","last_unstale":"2024-03-30T13:11:00.982601+0000","last_undegraded":"2024-03-30T13:11:00.982601+0000","last_fullsized":"2024-03-30T13:11:00.982601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:35:57.616293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977803+0000","last_change":"2024-03-30T13:10:55.970451+0000","last_active":"2024-03-30T13:11:00.977803+0000","last_peered":"2024-03-30T13:11:00.977803+0000","last_clean":"2024-03-30T13:11:00.977803+0000","last_became_active":"2024-03-30T13:10:55.969774+0000","last_became_peered":"2024-03-30T13:10:55.969774+0000","last_unstale":"2024-03-30T13:11:00.977803+0000","last_undegraded":"2024-03-30T13:11:00.977803+0000","last_fullsized":"2024-03-30T13:11:00.977803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859408+0000","last_change":"2024-03-30T13:11:00.039666+0000","last_active":"2024-03-30T13:11:01.859408+0000","last_peered":"2024-03-30T13:11:01.859408+0000","last_clean":"2024-03-30T13:11:01.859408+0000","last_became_active":"2024-03-30T13:11:00.038738+0000","last_became_peered":"2024-03-30T13:11:00.038738+0000","last_unstale":"2024-03-30T13:11:01.859408+0000","last_undegraded":"2024-03-30T13:11:01.859408+0000","last_fullsized":"2024-03-30T13:11:01.859408+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:53.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857699+0000","last_change":"2024-03-30T13:10:55.985031+0000","last_active":"2024-03-30T13:11:01.857699+0000","last_peered":"2024-03-30T13:11:01.857699+0000","last_clean":"2024-03-30T13:11:01.857699+0000","last_became_active":"2024-03-30T13:10:55.984727+0000","last_became_peered":"2024-03-30T13:10:55.984727+0000","last_unstale":"2024-03-30T13:11:01.857699+0000","last_undegraded":"2024-03-30T13:11:01.857699+0000","last_fullsized":"2024-03-30T13:11:01.857699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858193+0000","last_change":"2024-03-30T13:10:55.977928+0000","last_active":"2024-03-30T13:11:01.858193+0000","last_peered":"2024-03-30T13:11:01.858193+0000","last_clean":"2024-03-30T13:11:01.858193+0000","last_became_active":"2024-03-30T13:10:55.977153+0000","last_became_peered":"2024-03-30T13:10:55.977153+0000","last_unstale":"2024-03-30T13:11:01.858193+0000","last_undegraded":"2024-03-30T13:11:01.858193+0000","last_fullsized":"2024-03-30T13:11:01.858193+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853982+0000","last_change":"2024-03-30T13:10:55.965525+0000","last_active":"2024-03-30T13:11:01.853982+0000","last_peered":"2024-03-30T13:11:01.853982+0000","last_clean":"2024-03-30T13:11:01.853982+0000","last_became_active":"2024-03-30T13:10:55.965066+0000","last_became_peered":"2024-03-30T13:10:55.965066+0000","last_unstale":"2024-03-30T13:11:01.853982+0000","last_undegraded":"2024-03-30T13:11:01.853982+0000","last_fullsized":"2024-03-30T13:11:01.853982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982361+0000","last_change":"2024-03-30T13:10:55.961817+0000","last_active":"2024-03-30T13:11:00.982361+0000","last_peered":"2024-03-30T13:11:00.982361+0000","last_clean":"2024-03-30T13:11:00.982361+0000","last_became_active":"2024-03-30T13:10:55.960939+0000","last_became_peered":"2024-03-30T13:10:55.960939+0000","last_unstale":"2024-03-30T13:11:00.982361+0000","last_undegraded":"2024-03-30T13:11:00.982361+0000","last_fullsized":"2024-03-30T13:11:00.982361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:30:53.806860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980400+0000","last_change":"2024-03-30T13:10:55.991524+0000","last_active":"2024-03-30T13:11:00.980400+0000","last_peered":"2024-03-30T13:11:00.980400+0000","last_clean":"2024-03-30T13:11:00.980400+0000","last_became_active":"2024-03-30T13:10:55.990783+0000","last_became_peered":"2024-03-30T13:10:55.990783+0000","last_unstale":"2024-03-30T13:11:00.980400+0000","last_undegraded":"2024-03-30T13:11:00.980400+0000","last_fullsized":"2024-03-30T13:11:00.980400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:09.834713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981092+0000","last_change":"2024-03-30T13:10:55.985903+0000","last_active":"2024-03-30T13:11:00.981092+0000","last_peered":"2024-03-30T13:11:00.981092+0000","last_clean":"2024-03-30T13:11:00.981092+0000","last_became_active":"2024-03-30T13:10:55.985086+0000","last_became_peered":"2024-03-30T13:10:55.985086+0000","last_unstale":"2024-03-30T13:11:00.981092+0000","last_undegraded":"2024-03-30T13:11:00.981092+0000","last_fullsized":"2024-03-30T13:11:00.981092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858231+0000","last_change":"2024-03-30T13:10:55.961871+0000","last_active":"2024-03-30T13:11:01.858231+0000","last_peered":"2024-03-30T13:11:01.858231+0000","last_clean":"2024-03-30T13:11:01.858231+0000","last_became_active":"2024-03-30T13:10:55.961589+0000","last_became_peered":"2024-03-30T13:10:55.961589+0000","last_unstale":"2024-03-30T13:11:01.858231+0000","last_undegraded":"2024-03-30T13:11:01.858231+0000","last_fullsized":"2024-03-30T13:11:01.858231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981592+0000","last_change":"2024-03-30T13:10:55.979356+0000","last_active":"2024-03-30T13:11:00.981592+0000","last_peered":"2024-03-30T13:11:00.981592+0000","last_clean":"2024-03-30T13:11:00.981592+0000","last_became_active":"2024-03-30T13:10:55.978691+0000","last_became_peered":"2024-03-30T13:10:55.978691+0000","last_unstale":"2024-03-30T13:11:00.981592+0000","last_undegraded":"2024-03-30T13:11:00.981592+0000","last_fullsized":"2024-03-30T13:11:00.981592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:26:40.192186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853812+0000","last_change":"2024-03-30T13:10:55.965264+0000","last_active":"2024-03-30T13:11:01.853812+0000","last_peered":"2024-03-30T13:11:01.853812+0000","last_clean":"2024-03-30T13:11:01.853812+0000","last_became_active":"2024-03-30T13:10:55.964763+0000","last_became_peered":"2024-03-30T13:10:55.964763+0000","last_unstale":"2024-03-30T13:11:01.853812+0000","last_undegraded":"2024-03-30T13:11:01.853812+0000","last_fullsized":"2024-03-30T13:11:01.853812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980777+0000","last_change":"2024-03-30T13:10:55.968931+0000","last_active":"2024-03-30T13:11:00.980777+0000","last_peered":"2024-03-30T13:11:00.980777+0000","last_clean":"2024-03-30T13:11:00.980777+0000","last_became_active":"2024-03-30T13:10:55.968082+0000","last_became_peered":"2024-03-30T13:10:55.968082+0000","last_unstale":"2024-03-30T13:11:00.980777+0000","last_undegraded":"2024-03-30T13:11:00.980777+0000","last_fullsized":"2024-03-30T13:11:00.980777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:06:36.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977480+0000","last_change":"2024-03-30T13:10:55.971076+0000","last_active":"2024-03-30T13:11:00.977480+0000","last_peered":"2024-03-30T13:11:00.977480+0000","last_clean":"2024-03-30T13:11:00.977480+0000","last_became_active":"2024-03-30T13:10:55.970755+0000","last_became_peered":"2024-03-30T13:10:55.970755+0000","last_unstale":"2024-03-30T13:11:00.977480+0000","last_undegraded":"2024-03-30T13:11:00.977480+0000","last_fullsized":"2024-03-30T13:11:00.977480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855370+0000","last_change":"2024-03-30T13:10:55.996858+0000","last_active":"2024-03-30T13:11:01.855370+0000","last_peered":"2024-03-30T13:11:01.855370+0000","last_clean":"2024-03-30T13:11:01.855370+0000","last_became_active":"2024-03-30T13:10:55.980721+0000","last_became_peered":"2024-03-30T13:10:55.980721+0000","last_unstale":"2024-03-30T13:11:01.855370+0000","last_undegraded":"2024-03-30T13:11:01.855370+0000","last_fullsized":"2024-03-30T13:11:01.855370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:49.462756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855814+0000","last_change":"2024-03-30T13:10:55.976030+0000","last_active":"2024-03-30T13:11:01.855814+0000","last_peered":"2024-03-30T13:11:01.855814+0000","last_clean":"2024-03-30T13:11:01.855814+0000","last_became_active":"2024-03-30T13:10:55.975744+0000","last_became_peered":"2024-03-30T13:10:55.975744+0000","last_unstale":"2024-03-30T13:11:01.855814+0000","last_undegraded":"2024-03-30T13:11:01.855814+0000","last_fullsized":"2024-03-30T13:11:01.855814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.998425+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.998137+0000","last_became_peered":"2024-03-30T13:10:55.998137+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:11:09.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859740+0000","last_change":"2024-03-30T13:10:55.961591+0000","last_active":"2024-03-30T13:11:01.859740+0000","last_peered":"2024-03-30T13:11:01.859740+0000","last_clean":"2024-03-30T13:11:01.859740+0000","last_became_active":"2024-03-30T13:10:55.960885+0000","last_became_peered":"2024-03-30T13:10:55.960885+0000","last_unstale":"2024-03-30T13:11:01.859740+0000","last_undegraded":"2024-03-30T13:11:01.859740+0000","last_fullsized":"2024-03-30T13:11:01.859740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855050+0000","last_change":"2024-03-30T13:10:55.964798+0000","last_active":"2024-03-30T13:11:01.855050+0000","last_peered":"2024-03-30T13:11:01.855050+0000","last_clean":"2024-03-30T13:11:01.855050+0000","last_became_active":"2024-03-30T13:10:55.964471+0000","last_became_peered":"2024-03-30T13:10:55.964471+0000","last_unstale":"2024-03-30T13:11:01.855050+0000","last_undegraded":"2024-03-30T13:11:01.855050+0000","last_fullsized":"2024-03-30T13:11:01.855050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:41.557413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853680+0000","last_change":"2024-03-30T13:10:55.959954+0000","last_active":"2024-03-30T13:11:01.853680+0000","last_peered":"2024-03-30T13:11:01.853680+0000","last_clean":"2024-03-30T13:11:01.853680+0000","last_became_active":"2024-03-30T13:10:55.958856+0000","last_became_peered":"2024-03-30T13:10:55.958856+0000","last_unstale":"2024-03-30T13:11:01.853680+0000","last_undegraded":"2024-03-30T13:11:01.853680+0000","last_fullsized":"2024-03-30T13:11:01.853680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980372+0000","last_change":"2024-03-30T13:10:55.957900+0000","last_active":"2024-03-30T13:11:00.980372+0000","last_peered":"2024-03-30T13:11:00.980372+0000","last_clean":"2024-03-30T13:11:00.980372+0000","last_became_active":"2024-03-30T13:10:55.957071+0000","last_became_peered":"2024-03-30T13:10:55.957071+0000","last_unstale":"2024-03-30T13:11:00.980372+0000","last_undegraded":"2024-03-30T13:11:00.980372+0000","last_fullsized":"2024-03-30T13:11:00.980372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:16:27.043781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979715+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.979715+0000","last_peered":"2024-03-30T13:11:00.979715+0000","last_clean":"2024-03-30T13:11:00.979715+0000","last_became_active":"2024-03-30T13:10:55.990646+0000","last_became_peered":"2024-03-30T13:10:55.990646+0000","last_unstale":"2024-03-30T13:11:00.979715+0000","last_undegraded":"2024-03-30T13:11:00.979715+0000","last_fullsized":"2024-03-30T13:11:00.979715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981245+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.981245+0000","last_peered":"2024-03-30T13:11:00.981245+0000","last_clean":"2024-03-30T13:11:00.981245+0000","last_became_active":"2024-03-30T13:10:55.990666+0000","last_became_peered":"2024-03-30T13:10:55.990666+0000","last_unstale":"2024-03-30T13:11:00.981245+0000","last_undegraded":"2024-03-30T13:11:00.981245+0000","last_fullsized":"2024-03-30T13:11:00.981245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855715+0000","last_change":"2024-03-30T13:10:55.975002+0000","last_active":"2024-03-30T13:11:01.855715+0000","last_peered":"2024-03-30T13:11:01.855715+0000","last_clean":"2024-03-30T13:11:01.855715+0000","last_became_active":"2024-03-30T13:10:55.974620+0000","last_became_peered":"2024-03-30T13:10:55.974620+0000","last_unstale":"2024-03-30T13:11:01.855715+0000","last_undegraded":"2024-03-30T13:11:01.855715+0000","last_fullsized":"2024-03-30T13:11:01.855715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:33.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855796+0000","last_change":"2024-03-30T13:10:55.962912+0000","last_active":"2024-03-30T13:11:01.855796+0000","last_peered":"2024-03-30T13:11:01.855796+0000","last_clean":"2024-03-30T13:11:01.855796+0000","last_became_active":"2024-03-30T13:10:55.962404+0000","last_became_peered":"2024-03-30T13:10:55.962404+0000","last_unstale":"2024-03-30T13:11:01.855796+0000","last_undegraded":"2024-03-30T13:11:01.855796+0000","last_fullsized":"2024-03-30T13:11:01.855796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855883+0000","last_change":"2024-03-30T13:10:55.962709+0000","last_active":"2024-03-30T13:11:01.855883+0000","last_peered":"2024-03-30T13:11:01.855883+0000","last_clean":"2024-03-30T13:11:01.855883+0000","last_became_active":"2024-03-30T13:10:55.962426+0000","last_became_peered":"2024-03-30T13:10:55.962426+0000","last_unstale":"2024-03-30T13:11:01.855883+0000","last_undegraded":"2024-03-30T13:11:01.855883+0000","last_fullsized":"2024-03-30T13:11:01.855883+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978582+0000","last_change":"2024-03-30T13:10:55.973390+0000","last_active":"2024-03-30T13:11:00.978582+0000","last_peered":"2024-03-30T13:11:00.978582+0000","last_clean":"2024-03-30T13:11:00.978582+0000","last_became_active":"2024-03-30T13:10:55.969915+0000","last_became_peered":"2024-03-30T13:10:55.969915+0000","last_unstale":"2024-03-30T13:11:00.978582+0000","last_undegraded":"2024-03-30T13:11:00.978582+0000","last_fullsized":"2024-03-30T13:11:00.978582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979479+0000","last_change":"2024-03-30T13:10:55.969396+0000","last_active":"2024-03-30T13:11:00.979479+0000","last_peered":"2024-03-30T13:11:00.979479+0000","last_clean":"2024-03-30T13:11:00.979479+0000","last_became_active":"2024-03-30T13:10:55.965044+0000","last_became_peered":"2024-03-30T13:10:55.965044+0000","last_unstale":"2024-03-30T13:11:00.979479+0000","last_undegraded":"2024-03-30T13:11:00.979479+0000","last_fullsized":"2024-03-30T13:11:00.979479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:34:33.758969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855400+0000","last_change":"2024-03-30T13:10:55.974744+0000","last_active":"2024-03-30T13:11:01.855400+0000","last_peered":"2024-03-30T13:11:01.855400+0000","last_clean":"2024-03-30T13:11:01.855400+0000","last_became_active":"2024-03-30T13:10:55.974421+0000","last_became_peered":"2024-03-30T13:10:55.974421+0000","last_unstale":"2024-03-30T13:11:01.855400+0000","last_undegraded":"2024-03-30T13:11:01.855400+0000","last_fullsized":"2024-03-30T13:11:01.855400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:40.791064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857905+0000","last_change":"2024-03-30T13:10:55.998371+0000","last_active":"2024-03-30T13:11:01.857905+0000","last_peered":"2024-03-30T13:11:01.857905+0000","last_clean":"2024-03-30T13:11:01.857905+0000","last_became_active":"2024-03-30T13:10:55.997851+0000","last_became_peered":"2024-03-30T13:10:55.997851+0000","last_unstale":"2024-03-30T13:11:01.857905+0000","last_undegraded":"2024-03-30T13:11:01.857905+0000","last_fullsized":"2024-03-30T13:11:01.857905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:46.705515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855480+0000","last_change":"2024-03-30T13:10:55.967579+0000","last_active":"2024-03-30T13:11:01.855480+0000","last_peered":"2024-03-30T13:11:01.855480+0000","last_clean":"2024-03-30T13:11:01.855480+0000","last_became_active":"2024-03-30T13:10:55.967284+0000","last_became_peered":"2024-03-30T13:10:55.967284+0000","last_unstale":"2024-03-30T13:11:01.855480+0000","last_undegraded":"2024-03-30T13:11:01.855480+0000","last_fullsized":"2024-03-30T13:11:01.855480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:39.989019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858659+0000","last_change":"2024-03-30T13:10:55.998028+0000","last_active":"2024-03-30T13:11:01.858659+0000","last_peered":"2024-03-30T13:11:01.858659+0000","last_clean":"2024-03-30T13:11:01.858659+0000","last_became_active":"2024-03-30T13:10:55.997628+0000","last_became_peered":"2024-03-30T13:10:55.997628+0000","last_unstale":"2024-03-30T13:11:01.858659+0000","last_undegraded":"2024-03-30T13:11:01.858659+0000","last_fullsized":"2024-03-30T13:11:01.858659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982130+0000","last_change":"2024-03-30T13:10:55.983619+0000","last_active":"2024-03-30T13:11:00.982130+0000","last_peered":"2024-03-30T13:11:00.982130+0000","last_clean":"2024-03-30T13:11:00.982130+0000","last_became_active":"2024-03-30T13:10:55.982947+0000","last_became_peered":"2024-03-30T13:10:55.982947+0000","last_unstale":"2024-03-30T13:11:00.982130+0000","last_undegraded":"2024-03-30T13:11:00.982130+0000","last_fullsized":"2024-03-30T13:11:00.982130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:00.291625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983477+0000","last_change":"2024-03-30T13:10:55.992614+0000","last_active":"2024-03-30T13:11:00.983477+0000","last_peered":"2024-03-30T13:11:00.983477+0000","last_clean":"2024-03-30T13:11:00.983477+0000","last_became_active":"2024-03-30T13:10:55.991559+0000","last_became_peered":"2024-03-30T13:10:55.991559+0000","last_unstale":"2024-03-30T13:11:00.983477+0000","last_undegraded":"2024-03-30T13:11:00.983477+0000","last_fullsized":"2024-03-30T13:11:00.983477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:28:13.766039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856251+0000","last_change":"2024-03-30T13:10:55.960973+0000","last_active":"2024-03-30T13:11:01.856251+0000","last_peered":"2024-03-30T13:11:01.856251+0000","last_clean":"2024-03-30T13:11:01.856251+0000","last_became_active":"2024-03-30T13:10:55.960665+0000","last_became_peered":"2024-03-30T13:10:55.960665+0000","last_unstale":"2024-03-30T13:11:01.856251+0000","last_undegraded":"2024-03-30T13:11:01.856251+0000","last_fullsized":"2024-03-30T13:11:01.856251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:20.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:20.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f36671ce640 1 Processor -- start 2024-03-30T13:11:20.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f36671ce640 1 -- start start 2024-03-30T13:11:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f36671ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3660145e10 0x7f36600cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f36671ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 0x7f366015fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f36671ce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36600d06c0 con 0x7f3660160a90 2024-03-30T13:11:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f36671ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f36600d09a0 con 0x7f3660145e10 2024-03-30T13:11:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f36671ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f36600d0c80 con 0x7f36600d0220 2024-03-30T13:11:20.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f366576b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3660160a90 0x7f36600b6e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50486/0 (socket says 172.21.15.43:50486) 2024-03-30T13:11:20.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f366576b640 1 -- 172.21.15.43:0/1795302650 learned_addr learned my addr 172.21.15.43:0/1795302650 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f3664f6a640 1 --2- 172.21.15.43:0/1795302650 >> v2:172.21.15.113:3300/0 conn(0x7f3660145e10 0x7f36600cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365ffff640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 0x7f366015fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168831936 0 0) 0x7f36600d06c0 con 0x7f3660160a90 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3648003530 con 0x7f3660160a90 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2084648484 0 0) 0x7f3648003530 con 0x7f3660160a90 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 msgr2=0x7f366015fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 0x7f366015fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 >> v2:172.21.15.113:3300/0 conn(0x7f3660145e10 msgr2=0x7f36600cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:20.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 --2- 172.21.15.43:0/1795302650 >> v2:172.21.15.113:3300/0 conn(0x7f3660145e10 0x7f36600cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36600d0f60 con 0x7f3660160a90 2024-03-30T13:11:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f3658001a00 con 0x7f3660160a90 2024-03-30T13:11:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 >> v1:172.21.15.43:6789/0 conn(0x7f3660160a90 legacy=0x7f36600b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f3648003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 0x7f3648004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3648004d50 0x7f36480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36600d09a0 con 0x7f36480039c0 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36600d0c80 con 0x7f3648004d50 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f364800b9d0 con 0x7f36600d0220 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f3664f6a640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f3648003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f366576b640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3648004d50 0x7f36480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f365ffff640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 0x7f3648004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f3664f6a640 1 -- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 msgr2=0x7f3648004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f3664f6a640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36600d0220 0x7f3648004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f3664f6a640 1 -- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3648004d50 msgr2=0x7f36480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f3664f6a640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3648004d50 0x7f36480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f3664f6a640 1 -- 172.21.15.43:0/1795302650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3648003530 con 0x7f36480039c0 2024-03-30T13:11:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.319+0000 7f366576b640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3648004d50 0x7f36480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f3664f6a640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f3648003de0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ae0 tx=0x7f365401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a820c03d5bdf269 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540b2070 con 0x7f36480039c0 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36540a5590 con 0x7f36480039c0 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f365f7fe640 1 -- 172.21.15.43:0/1795302650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540096b0 con 0x7f36480039c0 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 -- 172.21.15.43:0/1795302650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 msgr2=0x7f3648003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 --2- 172.21.15.43:0/1795302650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f3648003de0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ae0 tx=0x7f365401cab0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f366576b640 1 -- 172.21.15.43:0/1795302650 reap_dead start 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 -- 172.21.15.43:0/1795302650 shutdown_connections 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 -- 172.21.15.43:0/1795302650 >> 172.21.15.43:0/1795302650 conn(0x7f36600ceb90 msgr2=0x7f36600cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 -- 172.21.15.43:0/1795302650 shutdown_connections 2024-03-30T13:11:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 -- 172.21.15.43:0/1795302650 wait complete. 2024-03-30T13:11:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 Processor -- start 2024-03-30T13:11:20.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.323+0000 7f36671ce640 1 -- start start 2024-03-30T13:11:20.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f36601ed380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366006ffa0 0x7f36601fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36601f9b60 0x7f36601f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36601600f0 con 0x7f36480039c0 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f366015ff70 con 0x7f36601f9b60 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3660160270 con 0x7f366006ffa0 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f36601ed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f36601ed380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41110/0 (socket says 172.21.15.43:41110) 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 -- 172.21.15.43:0/3296906347 learned_addr learned my addr 172.21.15.43:0/3296906347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 -- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366006ffa0 msgr2=0x7f36601fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366006ffa0 0x7f36601fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 -- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36601f9b60 msgr2=0x7f36601f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36601f9b60 0x7f36601f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 -- 172.21.15.43:0/3296906347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36601eb990 con 0x7f36480039c0 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f3664f6a640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f36601ed380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ae0 tx=0x7f3648005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:20.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f365d7fa640 1 -- 172.21.15.43:0/3296906347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36540b2040 con 0x7f36480039c0 2024-03-30T13:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f365d7fa640 1 -- 172.21.15.43:0/3296906347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365400f000 con 0x7f36480039c0 2024-03-30T13:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36601ebc70 con 0x7f36480039c0 2024-03-30T13:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f365d7fa640 1 -- 172.21.15.43:0/3296906347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654004020 con 0x7f36480039c0 2024-03-30T13:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36601ec230 con 0x7f36480039c0 2024-03-30T13:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3660068890 con 0x7f36480039c0 2024-03-30T13:11:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.327+0000 7f365d7fa640 1 -- 172.21.15.43:0/3296906347 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f36540c0020 con 0x7f36480039c0 2024-03-30T13:11:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.331+0000 7f365d7fa640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f364c067fc0 0x7f364c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.331+0000 7f365d7fa640 1 -- 172.21.15.43:0/3296906347 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f36541204f0 con 0x7f36480039c0 2024-03-30T13:11:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.331+0000 7f365ffff640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f364c067fc0 0x7f364c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.331+0000 7f365ffff640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f364c067fc0 0x7f364c06a480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f36601ed0a0 tx=0x7f3650008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:20.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.335+0000 7f365d7fa640 1 -- 172.21.15.43:0/3296906347 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f36540c7020 con 0x7f36480039c0 2024-03-30T13:11:20.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.491+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f36600b5940 con 0x7f364c067fc0 2024-03-30T13:11:20.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.515+0000 7f365d7fa640 1 -- 172.21.15.43:0/3296906347 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332530 (secure 0 0 0) 0x7f36600b5940 con 0x7f364c067fc0 2024-03-30T13:11:20.527 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:20.529 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:11:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f364c067fc0 msgr2=0x7f364c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f364c067fc0 0x7f364c06a480 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f36601ed0a0 tx=0x7f3650008040 comp rx=0 tx=0).stop 2024-03-30T13:11:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 msgr2=0x7f36601ed380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f36601ed380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3654009ae0 tx=0x7f3648005840 comp rx=0 tx=0).stop 2024-03-30T13:11:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 shutdown_connections 2024-03-30T13:11:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f364c067fc0 0x7f364c06a480 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36601f9b60 0x7f36601f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f366006ffa0 0x7f36601fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 --2- 172.21.15.43:0/3296906347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36480039c0 0x7f36601ed380 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 >> 172.21.15.43:0/3296906347 conn(0x7f36600ceb90 msgr2=0x7f36600d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 shutdown_connections 2024-03-30T13:11:20.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.519+0000 7f36671ce640 1 -- 172.21.15.43:0/3296906347 wait complete. 2024-03-30T13:11:20.545 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-03-30T13:11:19.844205+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236252,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738468,"statfs":{"total":773094113280,"available":772852191232,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005122"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857033+0000","last_change":"2024-03-30T13:10:54.944419+0000","last_active":"2024-03-30T13:11:01.857033+0000","last_peered":"2024-03-30T13:11:01.857033+0000","last_clean":"2024-03-30T13:11:01.857033+0000","last_became_active":"2024-03-30T13:10:54.943176+0000","last_became_peered":"2024-03-30T13:10:54.943176+0000","last_unstale":"2024-03-30T13:11:01.857033+0000","last_undegraded":"2024-03-30T13:11:01.857033+0000","last_fullsized":"2024-03-30T13:11:01.857033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983961+0000","last_change":"2024-03-30T13:10:55.957609+0000","last_active":"2024-03-30T13:11:00.983961+0000","last_peered":"2024-03-30T13:11:00.983961+0000","last_clean":"2024-03-30T13:11:00.983961+0000","last_became_active":"2024-03-30T13:10:55.956870+0000","last_became_peered":"2024-03-30T13:10:55.956870+0000","last_unstale":"2024-03-30T13:11:00.983961+0000","last_undegraded":"2024-03-30T13:11:00.983961+0000","last_fullsized":"2024-03-30T13:11:00.983961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:04:28.315124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858409+0000","last_change":"2024-03-30T13:11:00.037601+0000","last_active":"2024-03-30T13:11:01.858409+0000","last_peered":"2024-03-30T13:11:01.858409+0000","last_clean":"2024-03-30T13:11:01.858409+0000","last_became_active":"2024-03-30T13:11:00.032050+0000","last_became_peered":"2024-03-30T13:11:00.032050+0000","last_unstale":"2024-03-30T13:11:01.858409+0000","last_undegraded":"2024-03-30T13:11:01.858409+0000","last_fullsized":"2024-03-30T13:11:01.858409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:47.447707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.984087+0000","last_change":"2024-03-30T13:10:54.940096+0000","last_active":"2024-03-30T13:11:00.984087+0000","last_peered":"2024-03-30T13:11:00.984087+0000","last_clean":"2024-03-30T13:11:00.984087+0000","last_became_active":"2024-03-30T13:10:54.939368+0000","last_became_peered":"2024-03-30T13:10:54.939368+0000","last_unstale":"2024-03-30T13:11:00.984087+0000","last_undegraded":"2024-03-30T13:11:00.984087+0000","last_fullsized":"2024-03-30T13:11:00.984087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:09:39.904016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981759+0000","last_change":"2024-03-30T13:10:55.979018+0000","last_active":"2024-03-30T13:11:00.981759+0000","last_peered":"2024-03-30T13:11:00.981759+0000","last_clean":"2024-03-30T13:11:00.981759+0000","last_became_active":"2024-03-30T13:10:55.975807+0000","last_became_peered":"2024-03-30T13:10:55.975807+0000","last_unstale":"2024-03-30T13:11:00.981759+0000","last_undegraded":"2024-03-30T13:11:00.981759+0000","last_fullsized":"2024-03-30T13:11:00.981759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:35:35.881832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857637+0000","last_change":"2024-03-30T13:11:00.020209+0000","last_active":"2024-03-30T13:11:01.857637+0000","last_peered":"2024-03-30T13:11:01.857637+0000","last_clean":"2024-03-30T13:11:01.857637+0000","last_became_active":"2024-03-30T13:11:00.019627+0000","last_became_peered":"2024-03-30T13:11:00.019627+0000","last_unstale":"2024-03-30T13:11:01.857637+0000","last_undegraded":"2024-03-30T13:11:01.857637+0000","last_fullsized":"2024-03-30T13:11:01.857637+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:31.595903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856064+0000","last_change":"2024-03-30T13:10:54.945298+0000","last_active":"2024-03-30T13:11:01.856064+0000","last_peered":"2024-03-30T13:11:01.856064+0000","last_clean":"2024-03-30T13:11:01.856064+0000","last_became_active":"2024-03-30T13:10:54.942725+0000","last_became_peered":"2024-03-30T13:10:54.942725+0000","last_unstale":"2024-03-30T13:11:01.856064+0000","last_undegraded":"2024-03-30T13:11:01.856064+0000","last_fullsized":"2024-03-30T13:11:01.856064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978049+0000","last_change":"2024-03-30T13:10:55.973824+0000","last_active":"2024-03-30T13:11:00.978049+0000","last_peered":"2024-03-30T13:11:00.978049+0000","last_clean":"2024-03-30T13:11:00.978049+0000","last_became_active":"2024-03-30T13:10:55.969728+0000","last_became_peered":"2024-03-30T13:10:55.969728+0000","last_unstale":"2024-03-30T13:11:00.978049+0000","last_undegraded":"2024-03-30T13:11:00.978049+0000","last_fullsized":"2024-03-30T13:11:00.978049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858055+0000","last_change":"2024-03-30T13:11:00.015698+0000","last_active":"2024-03-30T13:11:01.858055+0000","last_peered":"2024-03-30T13:11:01.858055+0000","last_clean":"2024-03-30T13:11:01.858055+0000","last_became_active":"2024-03-30T13:11:00.015263+0000","last_became_peered":"2024-03-30T13:11:00.015263+0000","last_unstale":"2024-03-30T13:11:01.858055+0000","last_undegraded":"2024-03-30T13:11:01.858055+0000","last_fullsized":"2024-03-30T13:11:01.858055+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856071+0000","last_change":"2024-03-30T13:10:54.941048+0000","last_active":"2024-03-30T13:11:01.856071+0000","last_peered":"2024-03-30T13:11:01.856071+0000","last_clean":"2024-03-30T13:11:01.856071+0000","last_became_active":"2024-03-30T13:10:54.940543+0000","last_became_peered":"2024-03-30T13:10:54.940543+0000","last_unstale":"2024-03-30T13:11:01.856071+0000","last_undegraded":"2024-03-30T13:11:01.856071+0000","last_fullsized":"2024-03-30T13:11:01.856071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855446+0000","last_change":"2024-03-30T13:10:55.965085+0000","last_active":"2024-03-30T13:11:01.855446+0000","last_peered":"2024-03-30T13:11:01.855446+0000","last_clean":"2024-03-30T13:11:01.855446+0000","last_became_active":"2024-03-30T13:10:55.964573+0000","last_became_peered":"2024-03-30T13:10:55.964573+0000","last_unstale":"2024-03-30T13:11:01.855446+0000","last_undegraded":"2024-03-30T13:11:01.855446+0000","last_fullsized":"2024-03-30T13:11:01.855446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:56:01.049552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857815+0000","last_change":"2024-03-30T13:11:00.020015+0000","last_active":"2024-03-30T13:11:01.857815+0000","last_peered":"2024-03-30T13:11:01.857815+0000","last_clean":"2024-03-30T13:11:01.857815+0000","last_became_active":"2024-03-30T13:11:00.019563+0000","last_became_peered":"2024-03-30T13:11:00.019563+0000","last_unstale":"2024-03-30T13:11:01.857815+0000","last_undegraded":"2024-03-30T13:11:01.857815+0000","last_fullsized":"2024-03-30T13:11:01.857815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:44.852627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854082+0000","last_change":"2024-03-30T13:10:55.972256+0000","last_active":"2024-03-30T13:11:01.854082+0000","last_peered":"2024-03-30T13:11:01.854082+0000","last_clean":"2024-03-30T13:11:01.854082+0000","last_became_active":"2024-03-30T13:10:55.971884+0000","last_became_peered":"2024-03-30T13:10:55.971884+0000","last_unstale":"2024-03-30T13:11:01.854082+0000","last_undegraded":"2024-03-30T13:11:01.854082+0000","last_fullsized":"2024-03-30T13:11:01.854082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:47:49.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854680+0000","last_change":"2024-03-30T13:10:54.932797+0000","last_active":"2024-03-30T13:11:01.854680+0000","last_peered":"2024-03-30T13:11:01.854680+0000","last_clean":"2024-03-30T13:11:01.854680+0000","last_became_active":"2024-03-30T13:10:54.932392+0000","last_became_peered":"2024-03-30T13:10:54.932392+0000","last_unstale":"2024-03-30T13:11:01.854680+0000","last_undegraded":"2024-03-30T13:11:01.854680+0000","last_fullsized":"2024-03-30T13:11:01.854680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855304+0000","last_change":"2024-03-30T13:11:00.005113+0000","last_active":"2024-03-30T13:11:01.855304+0000","last_peered":"2024-03-30T13:11:01.855304+0000","last_clean":"2024-03-30T13:11:01.855304+0000","last_became_active":"2024-03-30T13:11:00.004701+0000","last_became_peered":"2024-03-30T13:11:00.004701+0000","last_unstale":"2024-03-30T13:11:01.855304+0000","last_undegraded":"2024-03-30T13:11:01.855304+0000","last_fullsized":"2024-03-30T13:11:01.855304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:12:46.188048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859494+0000","last_change":"2024-03-30T13:10:55.993733+0000","last_active":"2024-03-30T13:11:01.859494+0000","last_peered":"2024-03-30T13:11:01.859494+0000","last_clean":"2024-03-30T13:11:01.859494+0000","last_became_active":"2024-03-30T13:10:55.993012+0000","last_became_peered":"2024-03-30T13:10:55.993012+0000","last_unstale":"2024-03-30T13:11:01.859494+0000","last_undegraded":"2024-03-30T13:11:01.859494+0000","last_fullsized":"2024-03-30T13:11:01.859494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980295+0000","last_change":"2024-03-30T13:10:54.947616+0000","last_active":"2024-03-30T13:11:00.980295+0000","last_peered":"2024-03-30T13:11:00.980295+0000","last_clean":"2024-03-30T13:11:00.980295+0000","last_became_active":"2024-03-30T13:10:54.946695+0000","last_became_peered":"2024-03-30T13:10:54.946695+0000","last_unstale":"2024-03-30T13:11:00.980295+0000","last_undegraded":"2024-03-30T13:11:00.980295+0000","last_fullsized":"2024-03-30T13:11:00.980295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855501+0000","last_change":"2024-03-30T13:11:00.019814+0000","last_active":"2024-03-30T13:11:01.855501+0000","last_peered":"2024-03-30T13:11:01.855501+0000","last_clean":"2024-03-30T13:11:01.855501+0000","last_became_active":"2024-03-30T13:11:00.019359+0000","last_became_peered":"2024-03-30T13:11:00.019359+0000","last_unstale":"2024-03-30T13:11:01.855501+0000","last_undegraded":"2024-03-30T13:11:01.855501+0000","last_fullsized":"2024-03-30T13:11:01.855501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:40:01.204781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855194+0000","last_change":"2024-03-30T13:10:56.004578+0000","last_active":"2024-03-30T13:11:01.855194+0000","last_peered":"2024-03-30T13:11:01.855194+0000","last_clean":"2024-03-30T13:11:01.855194+0000","last_became_active":"2024-03-30T13:10:56.003914+0000","last_became_peered":"2024-03-30T13:10:56.003914+0000","last_unstale":"2024-03-30T13:11:01.855194+0000","last_undegraded":"2024-03-30T13:11:01.855194+0000","last_fullsized":"2024-03-30T13:11:01.855194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856085+0000","last_change":"2024-03-30T13:10:54.948212+0000","last_active":"2024-03-30T13:11:01.856085+0000","last_peered":"2024-03-30T13:11:01.856085+0000","last_clean":"2024-03-30T13:11:01.856085+0000","last_became_active":"2024-03-30T13:10:54.947336+0000","last_became_peered":"2024-03-30T13:10:54.947336+0000","last_unstale":"2024-03-30T13:11:01.856085+0000","last_undegraded":"2024-03-30T13:11:01.856085+0000","last_fullsized":"2024-03-30T13:11:01.856085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859925+0000","last_change":"2024-03-30T13:11:00.051626+0000","last_active":"2024-03-30T13:11:01.859925+0000","last_peered":"2024-03-30T13:11:01.859925+0000","last_clean":"2024-03-30T13:11:01.859925+0000","last_became_active":"2024-03-30T13:11:00.050798+0000","last_became_peered":"2024-03-30T13:11:00.050798+0000","last_unstale":"2024-03-30T13:11:01.859925+0000","last_undegraded":"2024-03-30T13:11:01.859925+0000","last_fullsized":"2024-03-30T13:11:01.859925+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:15.648737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853752+0000","last_change":"2024-03-30T13:10:55.965723+0000","last_active":"2024-03-30T13:11:01.853752+0000","last_peered":"2024-03-30T13:11:01.853752+0000","last_clean":"2024-03-30T13:11:01.853752+0000","last_became_active":"2024-03-30T13:10:55.965433+0000","last_became_peered":"2024-03-30T13:10:55.965433+0000","last_unstale":"2024-03-30T13:11:01.853752+0000","last_undegraded":"2024-03-30T13:11:01.853752+0000","last_fullsized":"2024-03-30T13:11:01.853752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854998+0000","last_change":"2024-03-30T13:10:54.940390+0000","last_active":"2024-03-30T13:11:01.854998+0000","last_peered":"2024-03-30T13:11:01.854998+0000","last_clean":"2024-03-30T13:11:01.854998+0000","last_became_active":"2024-03-30T13:10:54.939980+0000","last_became_peered":"2024-03-30T13:10:54.939980+0000","last_unstale":"2024-03-30T13:11:01.854998+0000","last_undegraded":"2024-03-30T13:11:01.854998+0000","last_fullsized":"2024-03-30T13:11:01.854998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858329+0000","last_change":"2024-03-30T13:11:00.019922+0000","last_active":"2024-03-30T13:11:01.858329+0000","last_peered":"2024-03-30T13:11:01.858329+0000","last_clean":"2024-03-30T13:11:01.858329+0000","last_became_active":"2024-03-30T13:11:00.019491+0000","last_became_peered":"2024-03-30T13:11:00.019491+0000","last_unstale":"2024-03-30T13:11:01.858329+0000","last_undegraded":"2024-03-30T13:11:01.858329+0000","last_fullsized":"2024-03-30T13:11:01.858329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856375+0000","last_change":"2024-03-30T13:10:55.968355+0000","last_active":"2024-03-30T13:11:01.856375+0000","last_peered":"2024-03-30T13:11:01.856375+0000","last_clean":"2024-03-30T13:11:01.856375+0000","last_became_active":"2024-03-30T13:10:55.965410+0000","last_became_peered":"2024-03-30T13:10:55.965410+0000","last_unstale":"2024-03-30T13:11:01.856375+0000","last_undegraded":"2024-03-30T13:11:01.856375+0000","last_fullsized":"2024-03-30T13:11:01.856375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:47:03.240883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855058+0000","last_change":"2024-03-30T13:10:54.944101+0000","last_active":"2024-03-30T13:11:01.855058+0000","last_peered":"2024-03-30T13:11:01.855058+0000","last_clean":"2024-03-30T13:11:01.855058+0000","last_became_active":"2024-03-30T13:10:54.943588+0000","last_became_peered":"2024-03-30T13:10:54.943588+0000","last_unstale":"2024-03-30T13:11:01.855058+0000","last_undegraded":"2024-03-30T13:11:01.855058+0000","last_fullsized":"2024-03-30T13:11:01.855058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857849+0000","last_change":"2024-03-30T13:11:00.009582+0000","last_active":"2024-03-30T13:11:01.857849+0000","last_peered":"2024-03-30T13:11:01.857849+0000","last_clean":"2024-03-30T13:11:01.857849+0000","last_became_active":"2024-03-30T13:11:00.009033+0000","last_became_peered":"2024-03-30T13:11:00.009033+0000","last_unstale":"2024-03-30T13:11:01.857849+0000","last_undegraded":"2024-03-30T13:11:01.857849+0000","last_fullsized":"2024-03-30T13:11:01.857849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856879+0000","last_change":"2024-03-30T13:10:56.002735+0000","last_active":"2024-03-30T13:11:01.856879+0000","last_peered":"2024-03-30T13:11:01.856879+0000","last_clean":"2024-03-30T13:11:01.856879+0000","last_became_active":"2024-03-30T13:10:56.001973+0000","last_became_peered":"2024-03-30T13:10:56.001973+0000","last_unstale":"2024-03-30T13:11:01.856879+0000","last_undegraded":"2024-03-30T13:11:01.856879+0000","last_fullsized":"2024-03-30T13:11:01.856879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856685+0000","last_change":"2024-03-30T13:10:54.947650+0000","last_active":"2024-03-30T13:11:01.856685+0000","last_peered":"2024-03-30T13:11:01.856685+0000","last_clean":"2024-03-30T13:11:01.856685+0000","last_became_active":"2024-03-30T13:10:54.946378+0000","last_became_peered":"2024-03-30T13:10:54.946378+0000","last_unstale":"2024-03-30T13:11:01.856685+0000","last_undegraded":"2024-03-30T13:11:01.856685+0000","last_fullsized":"2024-03-30T13:11:01.856685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855829+0000","last_change":"2024-03-30T13:11:00.031425+0000","last_active":"2024-03-30T13:11:01.855829+0000","last_peered":"2024-03-30T13:11:01.855829+0000","last_clean":"2024-03-30T13:11:01.855829+0000","last_became_active":"2024-03-30T13:11:00.030689+0000","last_became_peered":"2024-03-30T13:11:00.030689+0000","last_unstale":"2024-03-30T13:11:01.855829+0000","last_undegraded":"2024-03-30T13:11:01.855829+0000","last_fullsized":"2024-03-30T13:11:01.855829+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:50.405876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981455+0000","last_change":"2024-03-30T13:10:55.991469+0000","last_active":"2024-03-30T13:11:00.981455+0000","last_peered":"2024-03-30T13:11:00.981455+0000","last_clean":"2024-03-30T13:11:00.981455+0000","last_became_active":"2024-03-30T13:10:55.990596+0000","last_became_peered":"2024-03-30T13:10:55.990596+0000","last_unstale":"2024-03-30T13:11:00.981455+0000","last_undegraded":"2024-03-30T13:11:00.981455+0000","last_fullsized":"2024-03-30T13:11:00.981455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856131+0000","last_change":"2024-03-30T13:10:54.945048+0000","last_active":"2024-03-30T13:11:01.856131+0000","last_peered":"2024-03-30T13:11:01.856131+0000","last_clean":"2024-03-30T13:11:01.856131+0000","last_became_active":"2024-03-30T13:10:54.944552+0000","last_became_peered":"2024-03-30T13:10:54.944552+0000","last_unstale":"2024-03-30T13:11:01.856131+0000","last_undegraded":"2024-03-30T13:11:01.856131+0000","last_fullsized":"2024-03-30T13:11:01.856131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:08.826146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980755+0000","last_change":"2024-03-30T13:11:00.038523+0000","last_active":"2024-03-30T13:11:00.980755+0000","last_peered":"2024-03-30T13:11:00.980755+0000","last_clean":"2024-03-30T13:11:00.980755+0000","last_became_active":"2024-03-30T13:11:00.037276+0000","last_became_peered":"2024-03-30T13:11:00.037276+0000","last_unstale":"2024-03-30T13:11:00.980755+0000","last_undegraded":"2024-03-30T13:11:00.980755+0000","last_fullsized":"2024-03-30T13:11:00.980755+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858113+0000","last_change":"2024-03-30T13:10:55.992831+0000","last_active":"2024-03-30T13:11:01.858113+0000","last_peered":"2024-03-30T13:11:01.858113+0000","last_clean":"2024-03-30T13:11:01.858113+0000","last_became_active":"2024-03-30T13:10:55.992521+0000","last_became_peered":"2024-03-30T13:10:55.992521+0000","last_unstale":"2024-03-30T13:11:01.858113+0000","last_undegraded":"2024-03-30T13:11:01.858113+0000","last_fullsized":"2024-03-30T13:11:01.858113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979972+0000","last_change":"2024-03-30T13:10:54.952220+0000","last_active":"2024-03-30T13:11:00.979972+0000","last_peered":"2024-03-30T13:11:00.979972+0000","last_clean":"2024-03-30T13:11:00.979972+0000","last_became_active":"2024-03-30T13:10:54.951484+0000","last_became_peered":"2024-03-30T13:10:54.951484+0000","last_unstale":"2024-03-30T13:11:00.979972+0000","last_undegraded":"2024-03-30T13:11:00.979972+0000","last_fullsized":"2024-03-30T13:11:00.979972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:11:24.184946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979274+0000","last_change":"2024-03-30T13:11:00.043402+0000","last_active":"2024-03-30T13:11:00.979274+0000","last_peered":"2024-03-30T13:11:00.979274+0000","last_clean":"2024-03-30T13:11:00.979274+0000","last_became_active":"2024-03-30T13:11:00.040341+0000","last_became_peered":"2024-03-30T13:11:00.040341+0000","last_unstale":"2024-03-30T13:11:00.979274+0000","last_undegraded":"2024-03-30T13:11:00.979274+0000","last_fullsized":"2024-03-30T13:11:00.979274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855368+0000","last_change":"2024-03-30T13:10:55.958140+0000","last_active":"2024-03-30T13:11:01.855368+0000","last_peered":"2024-03-30T13:11:01.855368+0000","last_clean":"2024-03-30T13:11:01.855368+0000","last_became_active":"2024-03-30T13:10:55.957860+0000","last_became_peered":"2024-03-30T13:10:55.957860+0000","last_unstale":"2024-03-30T13:11:01.855368+0000","last_undegraded":"2024-03-30T13:11:01.855368+0000","last_fullsized":"2024-03-30T13:11:01.855368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:54.477311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854336+0000","last_change":"2024-03-30T13:10:54.945433+0000","last_active":"2024-03-30T13:11:01.854336+0000","last_peered":"2024-03-30T13:11:01.854336+0000","last_clean":"2024-03-30T13:11:01.854336+0000","last_became_active":"2024-03-30T13:10:54.945133+0000","last_became_peered":"2024-03-30T13:10:54.945133+0000","last_unstale":"2024-03-30T13:11:01.854336+0000","last_undegraded":"2024-03-30T13:11:01.854336+0000","last_fullsized":"2024-03-30T13:11:01.854336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:59.514214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980453+0000","last_change":"2024-03-30T13:11:00.044030+0000","last_active":"2024-03-30T13:11:00.980453+0000","last_peered":"2024-03-30T13:11:00.980453+0000","last_clean":"2024-03-30T13:11:00.980453+0000","last_became_active":"2024-03-30T13:11:00.037702+0000","last_became_peered":"2024-03-30T13:11:00.037702+0000","last_unstale":"2024-03-30T13:11:00.980453+0000","last_undegraded":"2024-03-30T13:11:00.980453+0000","last_fullsized":"2024-03-30T13:11:00.980453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979645+0000","last_change":"2024-03-30T13:10:55.962616+0000","last_active":"2024-03-30T13:11:00.979645+0000","last_peered":"2024-03-30T13:11:00.979645+0000","last_clean":"2024-03-30T13:11:00.979645+0000","last_became_active":"2024-03-30T13:10:55.961327+0000","last_became_peered":"2024-03-30T13:10:55.961327+0000","last_unstale":"2024-03-30T13:11:00.979645+0000","last_undegraded":"2024-03-30T13:11:00.979645+0000","last_fullsized":"2024-03-30T13:11:00.979645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:17.511817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858373+0000","last_change":"2024-03-30T13:10:54.942851+0000","last_active":"2024-03-30T13:11:01.858373+0000","last_peered":"2024-03-30T13:11:01.858373+0000","last_clean":"2024-03-30T13:11:01.858373+0000","last_became_active":"2024-03-30T13:10:54.939805+0000","last_became_peered":"2024-03-30T13:10:54.939805+0000","last_unstale":"2024-03-30T13:11:01.858373+0000","last_undegraded":"2024-03-30T13:11:01.858373+0000","last_fullsized":"2024-03-30T13:11:01.858373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:24.569683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982246+0000","last_change":"2024-03-30T13:11:00.040718+0000","last_active":"2024-03-30T13:11:00.982246+0000","last_peered":"2024-03-30T13:11:00.982246+0000","last_clean":"2024-03-30T13:11:00.982246+0000","last_became_active":"2024-03-30T13:11:00.039372+0000","last_became_peered":"2024-03-30T13:11:00.039372+0000","last_unstale":"2024-03-30T13:11:00.982246+0000","last_undegraded":"2024-03-30T13:11:00.982246+0000","last_fullsized":"2024-03-30T13:11:00.982246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:26.483027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979060+0000","last_change":"2024-03-30T13:10:55.984056+0000","last_active":"2024-03-30T13:11:00.979060+0000","last_peered":"2024-03-30T13:11:00.979060+0000","last_clean":"2024-03-30T13:11:00.979060+0000","last_became_active":"2024-03-30T13:10:55.974450+0000","last_became_peered":"2024-03-30T13:10:55.974450+0000","last_unstale":"2024-03-30T13:11:00.979060+0000","last_undegraded":"2024-03-30T13:11:00.979060+0000","last_fullsized":"2024-03-30T13:11:00.979060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853918+0000","last_change":"2024-03-30T13:10:54.942183+0000","last_active":"2024-03-30T13:11:01.853918+0000","last_peered":"2024-03-30T13:11:01.853918+0000","last_clean":"2024-03-30T13:11:01.853918+0000","last_became_active":"2024-03-30T13:10:54.941923+0000","last_became_peered":"2024-03-30T13:10:54.941923+0000","last_unstale":"2024-03-30T13:11:01.853918+0000","last_undegraded":"2024-03-30T13:11:01.853918+0000","last_fullsized":"2024-03-30T13:11:01.853918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:39.803698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983242+0000","last_change":"2024-03-30T13:11:00.024121+0000","last_active":"2024-03-30T13:11:00.983242+0000","last_peered":"2024-03-30T13:11:00.983242+0000","last_clean":"2024-03-30T13:11:00.983242+0000","last_became_active":"2024-03-30T13:11:00.022137+0000","last_became_peered":"2024-03-30T13:11:00.022137+0000","last_unstale":"2024-03-30T13:11:00.983242+0000","last_undegraded":"2024-03-30T13:11:00.983242+0000","last_fullsized":"2024-03-30T13:11:00.983242+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:08:16.219711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856025+0000","last_change":"2024-03-30T13:10:55.978563+0000","last_active":"2024-03-30T13:11:01.856025+0000","last_peered":"2024-03-30T13:11:01.856025+0000","last_clean":"2024-03-30T13:11:01.856025+0000","last_became_active":"2024-03-30T13:10:55.973372+0000","last_became_peered":"2024-03-30T13:10:55.973372+0000","last_unstale":"2024-03-30T13:11:01.856025+0000","last_undegraded":"2024-03-30T13:11:01.856025+0000","last_fullsized":"2024-03-30T13:11:01.856025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854645+0000","last_change":"2024-03-30T13:10:54.932742+0000","last_active":"2024-03-30T13:11:01.854645+0000","last_peered":"2024-03-30T13:11:01.854645+0000","last_clean":"2024-03-30T13:11:01.854645+0000","last_became_active":"2024-03-30T13:10:54.932326+0000","last_became_peered":"2024-03-30T13:10:54.932326+0000","last_unstale":"2024-03-30T13:11:01.854645+0000","last_undegraded":"2024-03-30T13:11:01.854645+0000","last_fullsized":"2024-03-30T13:11:01.854645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977500+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:11:00.977500+0000","last_peered":"2024-03-30T13:11:00.977500+0000","last_clean":"2024-03-30T13:11:00.977500+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:11:00.977500+0000","last_undegraded":"2024-03-30T13:11:00.977500+0000","last_fullsized":"2024-03-30T13:11:00.977500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856097+0000","last_change":"2024-03-30T13:10:59.992352+0000","last_active":"2024-03-30T13:11:01.856097+0000","last_peered":"2024-03-30T13:11:01.856097+0000","last_clean":"2024-03-30T13:11:01.856097+0000","last_became_active":"2024-03-30T13:10:59.991915+0000","last_became_peered":"2024-03-30T13:10:59.991915+0000","last_unstale":"2024-03-30T13:11:01.856097+0000","last_undegraded":"2024-03-30T13:11:01.856097+0000","last_fullsized":"2024-03-30T13:11:01.856097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:13:07.419696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855684+0000","last_change":"2024-03-30T13:10:55.971288+0000","last_active":"2024-03-30T13:11:01.855684+0000","last_peered":"2024-03-30T13:11:01.855684+0000","last_clean":"2024-03-30T13:11:01.855684+0000","last_became_active":"2024-03-30T13:10:55.970980+0000","last_became_peered":"2024-03-30T13:10:55.970980+0000","last_unstale":"2024-03-30T13:11:01.855684+0000","last_undegraded":"2024-03-30T13:11:01.855684+0000","last_fullsized":"2024-03-30T13:11:01.855684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:57.143218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980646+0000","last_change":"2024-03-30T13:10:54.946952+0000","last_active":"2024-03-30T13:11:00.980646+0000","last_peered":"2024-03-30T13:11:00.980646+0000","last_clean":"2024-03-30T13:11:00.980646+0000","last_became_active":"2024-03-30T13:10:54.946235+0000","last_became_peered":"2024-03-30T13:10:54.946235+0000","last_unstale":"2024-03-30T13:11:00.980646+0000","last_undegraded":"2024-03-30T13:11:00.980646+0000","last_fullsized":"2024-03-30T13:11:00.980646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857845+0000","last_change":"2024-03-30T13:11:00.017769+0000","last_active":"2024-03-30T13:11:01.857845+0000","last_peered":"2024-03-30T13:11:01.857845+0000","last_clean":"2024-03-30T13:11:01.857845+0000","last_became_active":"2024-03-30T13:11:00.017317+0000","last_became_peered":"2024-03-30T13:11:00.017317+0000","last_unstale":"2024-03-30T13:11:01.857845+0000","last_undegraded":"2024-03-30T13:11:01.857845+0000","last_fullsized":"2024-03-30T13:11:01.857845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:01.211799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.963559+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.963354+0000","last_became_peered":"2024-03-30T13:10:55.963354+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981372+0000","last_change":"2024-03-30T13:10:54.942041+0000","last_active":"2024-03-30T13:11:00.981372+0000","last_peered":"2024-03-30T13:11:00.981372+0000","last_clean":"2024-03-30T13:11:00.981372+0000","last_became_active":"2024-03-30T13:10:54.940995+0000","last_became_peered":"2024-03-30T13:10:54.940995+0000","last_unstale":"2024-03-30T13:11:00.981372+0000","last_undegraded":"2024-03-30T13:11:00.981372+0000","last_fullsized":"2024-03-30T13:11:00.981372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:54:21.415837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982091+0000","last_change":"2024-03-30T13:11:00.010520+0000","last_active":"2024-03-30T13:11:00.982091+0000","last_peered":"2024-03-30T13:11:00.982091+0000","last_clean":"2024-03-30T13:11:00.982091+0000","last_became_active":"2024-03-30T13:11:00.009785+0000","last_became_peered":"2024-03-30T13:11:00.009785+0000","last_unstale":"2024-03-30T13:11:00.982091+0000","last_undegraded":"2024-03-30T13:11:00.982091+0000","last_fullsized":"2024-03-30T13:11:00.982091+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:13:10.451556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856333+0000","last_change":"2024-03-30T13:10:55.979699+0000","last_active":"2024-03-30T13:11:01.856333+0000","last_peered":"2024-03-30T13:11:01.856333+0000","last_clean":"2024-03-30T13:11:01.856333+0000","last_became_active":"2024-03-30T13:10:55.979427+0000","last_became_peered":"2024-03-30T13:10:55.979427+0000","last_unstale":"2024-03-30T13:11:01.856333+0000","last_undegraded":"2024-03-30T13:11:01.856333+0000","last_fullsized":"2024-03-30T13:11:01.856333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856586+0000","last_change":"2024-03-30T13:10:54.945823+0000","last_active":"2024-03-30T13:11:01.856586+0000","last_peered":"2024-03-30T13:11:01.856586+0000","last_clean":"2024-03-30T13:11:01.856586+0000","last_became_active":"2024-03-30T13:10:54.944624+0000","last_became_peered":"2024-03-30T13:10:54.944624+0000","last_unstale":"2024-03-30T13:11:01.856586+0000","last_undegraded":"2024-03-30T13:11:01.856586+0000","last_fullsized":"2024-03-30T13:11:01.856586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853761+0000","last_change":"2024-03-30T13:11:00.002695+0000","last_active":"2024-03-30T13:11:01.853761+0000","last_peered":"2024-03-30T13:11:01.853761+0000","last_clean":"2024-03-30T13:11:01.853761+0000","last_became_active":"2024-03-30T13:11:00.002251+0000","last_became_peered":"2024-03-30T13:11:00.002251+0000","last_unstale":"2024-03-30T13:11:01.853761+0000","last_undegraded":"2024-03-30T13:11:01.853761+0000","last_fullsized":"2024-03-30T13:11:01.853761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:23:48.725109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858389+0000","last_change":"2024-03-30T13:10:55.966115+0000","last_active":"2024-03-30T13:11:01.858389+0000","last_peered":"2024-03-30T13:11:01.858389+0000","last_clean":"2024-03-30T13:11:01.858389+0000","last_became_active":"2024-03-30T13:10:55.965541+0000","last_became_peered":"2024-03-30T13:10:55.965541+0000","last_unstale":"2024-03-30T13:11:01.858389+0000","last_undegraded":"2024-03-30T13:11:01.858389+0000","last_fullsized":"2024-03-30T13:11:01.858389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856413+0000","last_change":"2024-03-30T13:10:54.945660+0000","last_active":"2024-03-30T13:11:01.856413+0000","last_peered":"2024-03-30T13:11:01.856413+0000","last_clean":"2024-03-30T13:11:01.856413+0000","last_became_active":"2024-03-30T13:10:54.944892+0000","last_became_peered":"2024-03-30T13:10:54.944892+0000","last_unstale":"2024-03-30T13:11:01.856413+0000","last_undegraded":"2024-03-30T13:11:01.856413+0000","last_fullsized":"2024-03-30T13:11:01.856413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977650+0000","last_change":"2024-03-30T13:11:00.014434+0000","last_active":"2024-03-30T13:11:00.977650+0000","last_peered":"2024-03-30T13:11:00.977650+0000","last_clean":"2024-03-30T13:11:00.977650+0000","last_became_active":"2024-03-30T13:11:00.013995+0000","last_became_peered":"2024-03-30T13:11:00.013995+0000","last_unstale":"2024-03-30T13:11:00.977650+0000","last_undegraded":"2024-03-30T13:11:00.977650+0000","last_fullsized":"2024-03-30T13:11:00.977650+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:13.896277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853679+0000","last_change":"2024-03-30T13:10:55.960324+0000","last_active":"2024-03-30T13:11:01.853679+0000","last_peered":"2024-03-30T13:11:01.853679+0000","last_clean":"2024-03-30T13:11:01.853679+0000","last_became_active":"2024-03-30T13:10:55.958877+0000","last_became_peered":"2024-03-30T13:10:55.958877+0000","last_unstale":"2024-03-30T13:11:01.853679+0000","last_undegraded":"2024-03-30T13:11:01.853679+0000","last_fullsized":"2024-03-30T13:11:01.853679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855512+0000","last_change":"2024-03-30T13:10:54.943074+0000","last_active":"2024-03-30T13:11:01.855512+0000","last_peered":"2024-03-30T13:11:01.855512+0000","last_clean":"2024-03-30T13:11:01.855512+0000","last_became_active":"2024-03-30T13:10:54.941870+0000","last_became_peered":"2024-03-30T13:10:54.941870+0000","last_unstale":"2024-03-30T13:11:01.855512+0000","last_undegraded":"2024-03-30T13:11:01.855512+0000","last_fullsized":"2024-03-30T13:11:01.855512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983714+0000","last_change":"2024-03-30T13:11:00.023257+0000","last_active":"2024-03-30T13:11:00.983714+0000","last_peered":"2024-03-30T13:11:00.983714+0000","last_clean":"2024-03-30T13:11:00.983714+0000","last_became_active":"2024-03-30T13:11:00.020654+0000","last_became_peered":"2024-03-30T13:11:00.020654+0000","last_unstale":"2024-03-30T13:11:00.983714+0000","last_undegraded":"2024-03-30T13:11:00.983714+0000","last_fullsized":"2024-03-30T13:11:00.983714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:16:15.446866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977511+0000","last_change":"2024-03-30T13:10:55.970389+0000","last_active":"2024-03-30T13:11:00.977511+0000","last_peered":"2024-03-30T13:11:00.977511+0000","last_clean":"2024-03-30T13:11:00.977511+0000","last_became_active":"2024-03-30T13:10:55.969686+0000","last_became_peered":"2024-03-30T13:10:55.969686+0000","last_unstale":"2024-03-30T13:11:00.977511+0000","last_undegraded":"2024-03-30T13:11:00.977511+0000","last_fullsized":"2024-03-30T13:11:00.977511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855488+0000","last_change":"2024-03-30T13:10:54.945121+0000","last_active":"2024-03-30T13:11:01.855488+0000","last_peered":"2024-03-30T13:11:01.855488+0000","last_clean":"2024-03-30T13:11:01.855488+0000","last_became_active":"2024-03-30T13:10:54.944583+0000","last_became_peered":"2024-03-30T13:10:54.944583+0000","last_unstale":"2024-03-30T13:11:01.855488+0000","last_undegraded":"2024-03-30T13:11:01.855488+0000","last_fullsized":"2024-03-30T13:11:01.855488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981873+0000","last_change":"2024-03-30T13:11:00.023347+0000","last_active":"2024-03-30T13:11:00.981873+0000","last_peered":"2024-03-30T13:11:00.981873+0000","last_clean":"2024-03-30T13:11:00.981873+0000","last_became_active":"2024-03-30T13:11:00.021082+0000","last_became_peered":"2024-03-30T13:11:00.021082+0000","last_unstale":"2024-03-30T13:11:00.981873+0000","last_undegraded":"2024-03-30T13:11:00.981873+0000","last_fullsized":"2024-03-30T13:11:00.981873+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:39.699905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854454+0000","last_change":"2024-03-30T13:10:55.964652+0000","last_active":"2024-03-30T13:11:01.854454+0000","last_peered":"2024-03-30T13:11:01.854454+0000","last_clean":"2024-03-30T13:11:01.854454+0000","last_became_active":"2024-03-30T13:10:55.964363+0000","last_became_peered":"2024-03-30T13:10:55.964363+0000","last_unstale":"2024-03-30T13:11:01.854454+0000","last_undegraded":"2024-03-30T13:11:01.854454+0000","last_fullsized":"2024-03-30T13:11:01.854454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:17.023431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854282+0000","last_change":"2024-03-30T13:10:54.936477+0000","last_active":"2024-03-30T13:11:01.854282+0000","last_peered":"2024-03-30T13:11:01.854282+0000","last_clean":"2024-03-30T13:11:01.854282+0000","last_became_active":"2024-03-30T13:10:54.935689+0000","last_became_peered":"2024-03-30T13:10:54.935689+0000","last_unstale":"2024-03-30T13:11:01.854282+0000","last_undegraded":"2024-03-30T13:11:01.854282+0000","last_fullsized":"2024-03-30T13:11:01.854282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:28.044884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980099+0000","last_change":"2024-03-30T13:11:00.044614+0000","last_active":"2024-03-30T13:11:00.980099+0000","last_peered":"2024-03-30T13:11:00.980099+0000","last_clean":"2024-03-30T13:11:00.980099+0000","last_became_active":"2024-03-30T13:11:00.039909+0000","last_became_peered":"2024-03-30T13:11:00.039909+0000","last_unstale":"2024-03-30T13:11:00.980099+0000","last_undegraded":"2024-03-30T13:11:00.980099+0000","last_fullsized":"2024-03-30T13:11:00.980099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979980+0000","last_change":"2024-03-30T13:10:55.985354+0000","last_active":"2024-03-30T13:11:00.979980+0000","last_peered":"2024-03-30T13:11:00.979980+0000","last_clean":"2024-03-30T13:11:00.979980+0000","last_became_active":"2024-03-30T13:10:55.974918+0000","last_became_peered":"2024-03-30T13:10:55.974918+0000","last_unstale":"2024-03-30T13:11:00.979980+0000","last_undegraded":"2024-03-30T13:11:00.979980+0000","last_fullsized":"2024-03-30T13:11:00.979980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:29.402254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859297+0000","last_change":"2024-03-30T13:10:54.955637+0000","last_active":"2024-03-30T13:11:01.859297+0000","last_peered":"2024-03-30T13:11:01.859297+0000","last_clean":"2024-03-30T13:11:01.859297+0000","last_became_active":"2024-03-30T13:10:54.949281+0000","last_became_peered":"2024-03-30T13:10:54.949281+0000","last_unstale":"2024-03-30T13:11:01.859297+0000","last_undegraded":"2024-03-30T13:11:01.859297+0000","last_fullsized":"2024-03-30T13:11:01.859297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855707+0000","last_change":"2024-03-30T13:11:00.015834+0000","last_active":"2024-03-30T13:11:01.855707+0000","last_peered":"2024-03-30T13:11:01.855707+0000","last_clean":"2024-03-30T13:11:01.855707+0000","last_became_active":"2024-03-30T13:11:00.015344+0000","last_became_peered":"2024-03-30T13:11:00.015344+0000","last_unstale":"2024-03-30T13:11:01.855707+0000","last_undegraded":"2024-03-30T13:11:01.855707+0000","last_fullsized":"2024-03-30T13:11:01.855707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:43.288754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855890+0000","last_change":"2024-03-30T13:10:55.993807+0000","last_active":"2024-03-30T13:11:01.855890+0000","last_peered":"2024-03-30T13:11:01.855890+0000","last_clean":"2024-03-30T13:11:01.855890+0000","last_became_active":"2024-03-30T13:10:55.993589+0000","last_became_peered":"2024-03-30T13:10:55.993589+0000","last_unstale":"2024-03-30T13:11:01.855890+0000","last_undegraded":"2024-03-30T13:11:01.855890+0000","last_fullsized":"2024-03-30T13:11:01.855890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:57.150236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:06.622780+0000","last_change":"2024-03-30T13:10:54.939893+0000","last_active":"2024-03-30T13:11:06.622780+0000","last_peered":"2024-03-30T13:11:06.622780+0000","last_clean":"2024-03-30T13:11:06.622780+0000","last_became_active":"2024-03-30T13:10:54.939427+0000","last_became_peered":"2024-03-30T13:10:54.939427+0000","last_unstale":"2024-03-30T13:11:06.622780+0000","last_undegraded":"2024-03-30T13:11:06.622780+0000","last_fullsized":"2024-03-30T13:11:06.622780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:07.195072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855852+0000","last_change":"2024-03-30T13:11:00.013757+0000","last_active":"2024-03-30T13:11:01.855852+0000","last_peered":"2024-03-30T13:11:01.855852+0000","last_clean":"2024-03-30T13:11:01.855852+0000","last_became_active":"2024-03-30T13:11:00.012615+0000","last_became_peered":"2024-03-30T13:11:00.012615+0000","last_unstale":"2024-03-30T13:11:01.855852+0000","last_undegraded":"2024-03-30T13:11:01.855852+0000","last_fullsized":"2024-03-30T13:11:01.855852+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:02:00.559037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853971+0000","last_change":"2024-03-30T13:10:55.957241+0000","last_active":"2024-03-30T13:11:01.853971+0000","last_peered":"2024-03-30T13:11:01.853971+0000","last_clean":"2024-03-30T13:11:01.853971+0000","last_became_active":"2024-03-30T13:10:55.956872+0000","last_became_peered":"2024-03-30T13:10:55.956872+0000","last_unstale":"2024-03-30T13:11:01.853971+0000","last_undegraded":"2024-03-30T13:11:01.853971+0000","last_fullsized":"2024-03-30T13:11:01.853971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:00.663013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859629+0000","last_change":"2024-03-30T13:10:54.947319+0000","last_active":"2024-03-30T13:11:01.859629+0000","last_peered":"2024-03-30T13:11:01.859629+0000","last_clean":"2024-03-30T13:11:01.859629+0000","last_became_active":"2024-03-30T13:10:54.946638+0000","last_became_peered":"2024-03-30T13:10:54.946638+0000","last_unstale":"2024-03-30T13:11:01.859629+0000","last_undegraded":"2024-03-30T13:11:01.859629+0000","last_fullsized":"2024-03-30T13:11:01.859629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858359+0000","last_change":"2024-03-30T13:11:00.032095+0000","last_active":"2024-03-30T13:11:01.858359+0000","last_peered":"2024-03-30T13:11:01.858359+0000","last_clean":"2024-03-30T13:11:01.858359+0000","last_became_active":"2024-03-30T13:11:00.031651+0000","last_became_peered":"2024-03-30T13:11:00.031651+0000","last_unstale":"2024-03-30T13:11:01.858359+0000","last_undegraded":"2024-03-30T13:11:01.858359+0000","last_fullsized":"2024-03-30T13:11:01.858359+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981894+0000","last_change":"2024-03-30T13:10:55.963310+0000","last_active":"2024-03-30T13:11:00.981894+0000","last_peered":"2024-03-30T13:11:00.981894+0000","last_clean":"2024-03-30T13:11:00.981894+0000","last_became_active":"2024-03-30T13:10:55.962608+0000","last_became_peered":"2024-03-30T13:10:55.962608+0000","last_unstale":"2024-03-30T13:11:00.981894+0000","last_undegraded":"2024-03-30T13:11:00.981894+0000","last_fullsized":"2024-03-30T13:11:00.981894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:53.368510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855969+0000","last_change":"2024-03-30T13:10:54.937448+0000","last_active":"2024-03-30T13:11:01.855969+0000","last_peered":"2024-03-30T13:11:01.855969+0000","last_clean":"2024-03-30T13:11:01.855969+0000","last_became_active":"2024-03-30T13:10:54.936836+0000","last_became_peered":"2024-03-30T13:10:54.936836+0000","last_unstale":"2024-03-30T13:11:01.855969+0000","last_undegraded":"2024-03-30T13:11:01.855969+0000","last_fullsized":"2024-03-30T13:11:01.855969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858228+0000","last_change":"2024-03-30T13:11:00.039950+0000","last_active":"2024-03-30T13:11:01.858228+0000","last_peered":"2024-03-30T13:11:01.858228+0000","last_clean":"2024-03-30T13:11:01.858228+0000","last_became_active":"2024-03-30T13:11:00.038871+0000","last_became_peered":"2024-03-30T13:11:00.038871+0000","last_unstale":"2024-03-30T13:11:01.858228+0000","last_undegraded":"2024-03-30T13:11:01.858228+0000","last_fullsized":"2024-03-30T13:11:01.858228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:16.001906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858336+0000","last_change":"2024-03-30T13:10:55.962143+0000","last_active":"2024-03-30T13:11:01.858336+0000","last_peered":"2024-03-30T13:11:01.858336+0000","last_clean":"2024-03-30T13:11:01.858336+0000","last_became_active":"2024-03-30T13:10:55.961186+0000","last_became_peered":"2024-03-30T13:10:55.961186+0000","last_unstale":"2024-03-30T13:11:01.858336+0000","last_undegraded":"2024-03-30T13:11:01.858336+0000","last_fullsized":"2024-03-30T13:11:01.858336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978652+0000","last_change":"2024-03-30T13:10:54.940086+0000","last_active":"2024-03-30T13:11:00.978652+0000","last_peered":"2024-03-30T13:11:00.978652+0000","last_clean":"2024-03-30T13:11:00.978652+0000","last_became_active":"2024-03-30T13:10:54.939206+0000","last_became_peered":"2024-03-30T13:10:54.939206+0000","last_unstale":"2024-03-30T13:11:00.978652+0000","last_undegraded":"2024-03-30T13:11:00.978652+0000","last_fullsized":"2024-03-30T13:11:00.978652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855931+0000","last_change":"2024-03-30T13:11:00.034220+0000","last_active":"2024-03-30T13:11:01.855931+0000","last_peered":"2024-03-30T13:11:01.855931+0000","last_clean":"2024-03-30T13:11:01.855931+0000","last_became_active":"2024-03-30T13:11:00.033211+0000","last_became_peered":"2024-03-30T13:11:00.033211+0000","last_unstale":"2024-03-30T13:11:01.855931+0000","last_undegraded":"2024-03-30T13:11:01.855931+0000","last_fullsized":"2024-03-30T13:11:01.855931+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:22.895535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856526+0000","last_change":"2024-03-30T13:11:00.028139+0000","last_active":"2024-03-30T13:11:01.856526+0000","last_peered":"2024-03-30T13:11:01.856526+0000","last_clean":"2024-03-30T13:11:01.856526+0000","last_became_active":"2024-03-30T13:11:00.027230+0000","last_became_peered":"2024-03-30T13:11:00.027230+0000","last_unstale":"2024-03-30T13:11:01.856526+0000","last_undegraded":"2024-03-30T13:11:01.856526+0000","last_fullsized":"2024-03-30T13:11:01.856526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:04.724576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855891+0000","last_change":"2024-03-30T13:10:55.996825+0000","last_active":"2024-03-30T13:11:01.855891+0000","last_peered":"2024-03-30T13:11:01.855891+0000","last_clean":"2024-03-30T13:11:01.855891+0000","last_became_active":"2024-03-30T13:10:55.977957+0000","last_became_peered":"2024-03-30T13:10:55.977957+0000","last_unstale":"2024-03-30T13:11:01.855891+0000","last_undegraded":"2024-03-30T13:11:01.855891+0000","last_fullsized":"2024-03-30T13:11:01.855891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853813+0000","last_change":"2024-03-30T13:10:54.930843+0000","last_active":"2024-03-30T13:11:01.853813+0000","last_peered":"2024-03-30T13:11:01.853813+0000","last_clean":"2024-03-30T13:11:01.853813+0000","last_became_active":"2024-03-30T13:10:54.930056+0000","last_became_peered":"2024-03-30T13:10:54.930056+0000","last_unstale":"2024-03-30T13:11:01.853813+0000","last_undegraded":"2024-03-30T13:11:01.853813+0000","last_fullsized":"2024-03-30T13:11:01.853813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854932+0000","last_change":"2024-03-30T13:10:59.991481+0000","last_active":"2024-03-30T13:11:01.854932+0000","last_peered":"2024-03-30T13:11:01.854932+0000","last_clean":"2024-03-30T13:11:01.854932+0000","last_became_active":"2024-03-30T13:10:59.991071+0000","last_became_peered":"2024-03-30T13:10:59.991071+0000","last_unstale":"2024-03-30T13:11:01.854932+0000","last_undegraded":"2024-03-30T13:11:01.854932+0000","last_fullsized":"2024-03-30T13:11:01.854932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:29:50.731633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979852+0000","last_change":"2024-03-30T13:10:55.994215+0000","last_active":"2024-03-30T13:11:00.979852+0000","last_peered":"2024-03-30T13:11:00.979852+0000","last_clean":"2024-03-30T13:11:00.979852+0000","last_became_active":"2024-03-30T13:10:55.993553+0000","last_became_peered":"2024-03-30T13:10:55.993553+0000","last_unstale":"2024-03-30T13:11:00.979852+0000","last_undegraded":"2024-03-30T13:11:00.979852+0000","last_fullsized":"2024-03-30T13:11:00.979852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:35.609380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979309+0000","last_change":"2024-03-30T13:10:54.941904+0000","last_active":"2024-03-30T13:11:00.979309+0000","last_peered":"2024-03-30T13:11:00.979309+0000","last_clean":"2024-03-30T13:11:00.979309+0000","last_became_active":"2024-03-30T13:10:54.941223+0000","last_became_peered":"2024-03-30T13:10:54.941223+0000","last_unstale":"2024-03-30T13:11:00.979309+0000","last_undegraded":"2024-03-30T13:11:00.979309+0000","last_fullsized":"2024-03-30T13:11:00.979309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854339+0000","last_change":"2024-03-30T13:10:54.943752+0000","last_active":"2024-03-30T13:11:01.854339+0000","last_peered":"2024-03-30T13:11:01.854339+0000","last_clean":"2024-03-30T13:11:01.854339+0000","last_became_active":"2024-03-30T13:10:54.943525+0000","last_became_peered":"2024-03-30T13:10:54.943525+0000","last_unstale":"2024-03-30T13:11:01.854339+0000","last_undegraded":"2024-03-30T13:11:01.854339+0000","last_fullsized":"2024-03-30T13:11:01.854339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:47.689419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855653+0000","last_change":"2024-03-30T13:10:55.996261+0000","last_active":"2024-03-30T13:11:01.855653+0000","last_peered":"2024-03-30T13:11:01.855653+0000","last_clean":"2024-03-30T13:11:01.855653+0000","last_became_active":"2024-03-30T13:10:55.979906+0000","last_became_peered":"2024-03-30T13:10:55.979906+0000","last_unstale":"2024-03-30T13:11:01.855653+0000","last_undegraded":"2024-03-30T13:11:01.855653+0000","last_fullsized":"2024-03-30T13:11:01.855653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:11.587174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856525+0000","last_change":"2024-03-30T13:11:00.031762+0000","last_active":"2024-03-30T13:11:01.856525+0000","last_peered":"2024-03-30T13:11:01.856525+0000","last_clean":"2024-03-30T13:11:01.856525+0000","last_became_active":"2024-03-30T13:11:00.030859+0000","last_became_peered":"2024-03-30T13:11:00.030859+0000","last_unstale":"2024-03-30T13:11:01.856525+0000","last_undegraded":"2024-03-30T13:11:01.856525+0000","last_fullsized":"2024-03-30T13:11:01.856525+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:21.084995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982601+0000","last_change":"2024-03-30T13:10:54.944088+0000","last_active":"2024-03-30T13:11:00.982601+0000","last_peered":"2024-03-30T13:11:00.982601+0000","last_clean":"2024-03-30T13:11:00.982601+0000","last_became_active":"2024-03-30T13:10:54.943275+0000","last_became_peered":"2024-03-30T13:10:54.943275+0000","last_unstale":"2024-03-30T13:11:00.982601+0000","last_undegraded":"2024-03-30T13:11:00.982601+0000","last_fullsized":"2024-03-30T13:11:00.982601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:35:57.616293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977803+0000","last_change":"2024-03-30T13:10:55.970451+0000","last_active":"2024-03-30T13:11:00.977803+0000","last_peered":"2024-03-30T13:11:00.977803+0000","last_clean":"2024-03-30T13:11:00.977803+0000","last_became_active":"2024-03-30T13:10:55.969774+0000","last_became_peered":"2024-03-30T13:10:55.969774+0000","last_unstale":"2024-03-30T13:11:00.977803+0000","last_undegraded":"2024-03-30T13:11:00.977803+0000","last_fullsized":"2024-03-30T13:11:00.977803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859408+0000","last_change":"2024-03-30T13:11:00.039666+0000","last_active":"2024-03-30T13:11:01.859408+0000","last_peered":"2024-03-30T13:11:01.859408+0000","last_clean":"2024-03-30T13:11:01.859408+0000","last_became_active":"2024-03-30T13:11:00.038738+0000","last_became_peered":"2024-03-30T13:11:00.038738+0000","last_unstale":"2024-03-30T13:11:01.859408+0000","last_undegraded":"2024-03-30T13:11:01.859408+0000","last_fullsized":"2024-03-30T13:11:01.859408+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:53.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857699+0000","last_change":"2024-03-30T13:10:55.985031+0000","last_active":"2024-03-30T13:11:01.857699+0000","last_peered":"2024-03-30T13:11:01.857699+0000","last_clean":"2024-03-30T13:11:01.857699+0000","last_became_active":"2024-03-30T13:10:55.984727+0000","last_became_peered":"2024-03-30T13:10:55.984727+0000","last_unstale":"2024-03-30T13:11:01.857699+0000","last_undegraded":"2024-03-30T13:11:01.857699+0000","last_fullsized":"2024-03-30T13:11:01.857699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858193+0000","last_change":"2024-03-30T13:10:55.977928+0000","last_active":"2024-03-30T13:11:01.858193+0000","last_peered":"2024-03-30T13:11:01.858193+0000","last_clean":"2024-03-30T13:11:01.858193+0000","last_became_active":"2024-03-30T13:10:55.977153+0000","last_became_peered":"2024-03-30T13:10:55.977153+0000","last_unstale":"2024-03-30T13:11:01.858193+0000","last_undegraded":"2024-03-30T13:11:01.858193+0000","last_fullsized":"2024-03-30T13:11:01.858193+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853982+0000","last_change":"2024-03-30T13:10:55.965525+0000","last_active":"2024-03-30T13:11:01.853982+0000","last_peered":"2024-03-30T13:11:01.853982+0000","last_clean":"2024-03-30T13:11:01.853982+0000","last_became_active":"2024-03-30T13:10:55.965066+0000","last_became_peered":"2024-03-30T13:10:55.965066+0000","last_unstale":"2024-03-30T13:11:01.853982+0000","last_undegraded":"2024-03-30T13:11:01.853982+0000","last_fullsized":"2024-03-30T13:11:01.853982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982361+0000","last_change":"2024-03-30T13:10:55.961817+0000","last_active":"2024-03-30T13:11:00.982361+0000","last_peered":"2024-03-30T13:11:00.982361+0000","last_clean":"2024-03-30T13:11:00.982361+0000","last_became_active":"2024-03-30T13:10:55.960939+0000","last_became_peered":"2024-03-30T13:10:55.960939+0000","last_unstale":"2024-03-30T13:11:00.982361+0000","last_undegraded":"2024-03-30T13:11:00.982361+0000","last_fullsized":"2024-03-30T13:11:00.982361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:30:53.806860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980400+0000","last_change":"2024-03-30T13:10:55.991524+0000","last_active":"2024-03-30T13:11:00.980400+0000","last_peered":"2024-03-30T13:11:00.980400+0000","last_clean":"2024-03-30T13:11:00.980400+0000","last_became_active":"2024-03-30T13:10:55.990783+0000","last_became_peered":"2024-03-30T13:10:55.990783+0000","last_unstale":"2024-03-30T13:11:00.980400+0000","last_undegraded":"2024-03-30T13:11:00.980400+0000","last_fullsized":"2024-03-30T13:11:00.980400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:09.834713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981092+0000","last_change":"2024-03-30T13:10:55.985903+0000","last_active":"2024-03-30T13:11:00.981092+0000","last_peered":"2024-03-30T13:11:00.981092+0000","last_clean":"2024-03-30T13:11:00.981092+0000","last_became_active":"2024-03-30T13:10:55.985086+0000","last_became_peered":"2024-03-30T13:10:55.985086+0000","last_unstale":"2024-03-30T13:11:00.981092+0000","last_undegraded":"2024-03-30T13:11:00.981092+0000","last_fullsized":"2024-03-30T13:11:00.981092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858231+0000","last_change":"2024-03-30T13:10:55.961871+0000","last_active":"2024-03-30T13:11:01.858231+0000","last_peered":"2024-03-30T13:11:01.858231+0000","last_clean":"2024-03-30T13:11:01.858231+0000","last_became_active":"2024-03-30T13:10:55.961589+0000","last_became_peered":"2024-03-30T13:10:55.961589+0000","last_unstale":"2024-03-30T13:11:01.858231+0000","last_undegraded":"2024-03-30T13:11:01.858231+0000","last_fullsized":"2024-03-30T13:11:01.858231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981592+0000","last_change":"2024-03-30T13:10:55.979356+0000","last_active":"2024-03-30T13:11:00.981592+0000","last_peered":"2024-03-30T13:11:00.981592+0000","last_clean":"2024-03-30T13:11:00.981592+0000","last_became_active":"2024-03-30T13:10:55.978691+0000","last_became_peered":"2024-03-30T13:10:55.978691+0000","last_unstale":"2024-03-30T13:11:00.981592+0000","last_undegraded":"2024-03-30T13:11:00.981592+0000","last_fullsized":"2024-03-30T13:11:00.981592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:26:40.192186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853812+0000","last_change":"2024-03-30T13:10:55.965264+0000","last_active":"2024-03-30T13:11:01.853812+0000","last_peered":"2024-03-30T13:11:01.853812+0000","last_clean":"2024-03-30T13:11:01.853812+0000","last_became_active":"2024-03-30T13:10:55.964763+0000","last_became_peered":"2024-03-30T13:10:55.964763+0000","last_unstale":"2024-03-30T13:11:01.853812+0000","last_undegraded":"2024-03-30T13:11:01.853812+0000","last_fullsized":"2024-03-30T13:11:01.853812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980777+0000","last_change":"2024-03-30T13:10:55.968931+0000","last_active":"2024-03-30T13:11:00.980777+0000","last_peered":"2024-03-30T13:11:00.980777+0000","last_clean":"2024-03-30T13:11:00.980777+0000","last_became_active":"2024-03-30T13:10:55.968082+0000","last_became_peered":"2024-03-30T13:10:55.968082+0000","last_unstale":"2024-03-30T13:11:00.980777+0000","last_undegraded":"2024-03-30T13:11:00.980777+0000","last_fullsized":"2024-03-30T13:11:00.980777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:06:36.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977480+0000","last_change":"2024-03-30T13:10:55.971076+0000","last_active":"2024-03-30T13:11:00.977480+0000","last_peered":"2024-03-30T13:11:00.977480+0000","last_clean":"2024-03-30T13:11:00.977480+0000","last_became_active":"2024-03-30T13:10:55.970755+0000","last_became_peered":"2024-03-30T13:10:55.970755+0000","last_unstale":"2024-03-30T13:11:00.977480+0000","last_undegraded":"2024-03-30T13:11:00.977480+0000","last_fullsized":"2024-03-30T13:11:00.977480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855370+0000","last_change":"2024-03-30T13:10:55.996858+0000","last_active":"2024-03-30T13:11:01.855370+0000","last_peered":"2024-03-30T13:11:01.855370+0000","last_clean":"2024-03-30T13:11:01.855370+0000","last_became_active":"2024-03-30T13:10:55.980721+0000","last_became_peered":"2024-03-30T13:10:55.980721+0000","last_unstale":"2024-03-30T13:11:01.855370+0000","last_undegraded":"2024-03-30T13:11:01.855370+0000","last_fullsized":"2024-03-30T13:11:01.855370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:49.462756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855814+0000","last_change":"2024-03-30T13:10:55.976030+0000","last_active":"2024-03-30T13:11:01.855814+0000","last_peered":"2024-03-30T13:11:01.855814+0000","last_clean":"2024-03-30T13:11:01.855814+0000","last_became_active":"2024-03-30T13:10:55.975744+0000","last_became_peered":"2024-03-30T13:10:55.975744+0000","last_unstale":"2024-03-30T13:11:01.855814+0000","last_undegraded":"2024-03-30T13:11:01.855814+0000","last_fullsized":"2024-03-30T13:11:01.855814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.998425+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.998137+0000","last_became_peered":"2024-03-30T13:10:55.998137+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:11:09.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859740+0000","last_change":"2024-03-30T13:10:55.961591+0000","last_active":"2024-03-30T13:11:01.859740+0000","last_peered":"2024-03-30T13:11:01.859740+0000","last_clean":"2024-03-30T13:11:01.859740+0000","last_became_active":"2024-03-30T13:10:55.960885+0000","last_became_peered":"2024-03-30T13:10:55.960885+0000","last_unstale":"2024-03-30T13:11:01.859740+0000","last_undegraded":"2024-03-30T13:11:01.859740+0000","last_fullsized":"2024-03-30T13:11:01.859740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855050+0000","last_change":"2024-03-30T13:10:55.964798+0000","last_active":"2024-03-30T13:11:01.855050+0000","last_peered":"2024-03-30T13:11:01.855050+0000","last_clean":"2024-03-30T13:11:01.855050+0000","last_became_active":"2024-03-30T13:10:55.964471+0000","last_became_peered":"2024-03-30T13:10:55.964471+0000","last_unstale":"2024-03-30T13:11:01.855050+0000","last_undegraded":"2024-03-30T13:11:01.855050+0000","last_fullsized":"2024-03-30T13:11:01.855050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:41.557413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853680+0000","last_change":"2024-03-30T13:10:55.959954+0000","last_active":"2024-03-30T13:11:01.853680+0000","last_peered":"2024-03-30T13:11:01.853680+0000","last_clean":"2024-03-30T13:11:01.853680+0000","last_became_active":"2024-03-30T13:10:55.958856+0000","last_became_peered":"2024-03-30T13:10:55.958856+0000","last_unstale":"2024-03-30T13:11:01.853680+0000","last_undegraded":"2024-03-30T13:11:01.853680+0000","last_fullsized":"2024-03-30T13:11:01.853680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980372+0000","last_change":"2024-03-30T13:10:55.957900+0000","last_active":"2024-03-30T13:11:00.980372+0000","last_peered":"2024-03-30T13:11:00.980372+0000","last_clean":"2024-03-30T13:11:00.980372+0000","last_became_active":"2024-03-30T13:10:55.957071+0000","last_became_peered":"2024-03-30T13:10:55.957071+0000","last_unstale":"2024-03-30T13:11:00.980372+0000","last_undegraded":"2024-03-30T13:11:00.980372+0000","last_fullsized":"2024-03-30T13:11:00.980372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:16:27.043781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979715+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.979715+0000","last_peered":"2024-03-30T13:11:00.979715+0000","last_clean":"2024-03-30T13:11:00.979715+0000","last_became_active":"2024-03-30T13:10:55.990646+0000","last_became_peered":"2024-03-30T13:10:55.990646+0000","last_unstale":"2024-03-30T13:11:00.979715+0000","last_undegraded":"2024-03-30T13:11:00.979715+0000","last_fullsized":"2024-03-30T13:11:00.979715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981245+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.981245+0000","last_peered":"2024-03-30T13:11:00.981245+0000","last_clean":"2024-03-30T13:11:00.981245+0000","last_became_active":"2024-03-30T13:10:55.990666+0000","last_became_peered":"2024-03-30T13:10:55.990666+0000","last_unstale":"2024-03-30T13:11:00.981245+0000","last_undegraded":"2024-03-30T13:11:00.981245+0000","last_fullsized":"2024-03-30T13:11:00.981245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855715+0000","last_change":"2024-03-30T13:10:55.975002+0000","last_active":"2024-03-30T13:11:01.855715+0000","last_peered":"2024-03-30T13:11:01.855715+0000","last_clean":"2024-03-30T13:11:01.855715+0000","last_became_active":"2024-03-30T13:10:55.974620+0000","last_became_peered":"2024-03-30T13:10:55.974620+0000","last_unstale":"2024-03-30T13:11:01.855715+0000","last_undegraded":"2024-03-30T13:11:01.855715+0000","last_fullsized":"2024-03-30T13:11:01.855715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:33.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855796+0000","last_change":"2024-03-30T13:10:55.962912+0000","last_active":"2024-03-30T13:11:01.855796+0000","last_peered":"2024-03-30T13:11:01.855796+0000","last_clean":"2024-03-30T13:11:01.855796+0000","last_became_active":"2024-03-30T13:10:55.962404+0000","last_became_peered":"2024-03-30T13:10:55.962404+0000","last_unstale":"2024-03-30T13:11:01.855796+0000","last_undegraded":"2024-03-30T13:11:01.855796+0000","last_fullsized":"2024-03-30T13:11:01.855796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855883+0000","last_change":"2024-03-30T13:10:55.962709+0000","last_active":"2024-03-30T13:11:01.855883+0000","last_peered":"2024-03-30T13:11:01.855883+0000","last_clean":"2024-03-30T13:11:01.855883+0000","last_became_active":"2024-03-30T13:10:55.962426+0000","last_became_peered":"2024-03-30T13:10:55.962426+0000","last_unstale":"2024-03-30T13:11:01.855883+0000","last_undegraded":"2024-03-30T13:11:01.855883+0000","last_fullsized":"2024-03-30T13:11:01.855883+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978582+0000","last_change":"2024-03-30T13:10:55.973390+0000","last_active":"2024-03-30T13:11:00.978582+0000","last_peered":"2024-03-30T13:11:00.978582+0000","last_clean":"2024-03-30T13:11:00.978582+0000","last_became_active":"2024-03-30T13:10:55.969915+0000","last_became_peered":"2024-03-30T13:10:55.969915+0000","last_unstale":"2024-03-30T13:11:00.978582+0000","last_undegraded":"2024-03-30T13:11:00.978582+0000","last_fullsized":"2024-03-30T13:11:00.978582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979479+0000","last_change":"2024-03-30T13:10:55.969396+0000","last_active":"2024-03-30T13:11:00.979479+0000","last_peered":"2024-03-30T13:11:00.979479+0000","last_clean":"2024-03-30T13:11:00.979479+0000","last_became_active":"2024-03-30T13:10:55.965044+0000","last_became_peered":"2024-03-30T13:10:55.965044+0000","last_unstale":"2024-03-30T13:11:00.979479+0000","last_undegraded":"2024-03-30T13:11:00.979479+0000","last_fullsized":"2024-03-30T13:11:00.979479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:34:33.758969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855400+0000","last_change":"2024-03-30T13:10:55.974744+0000","last_active":"2024-03-30T13:11:01.855400+0000","last_peered":"2024-03-30T13:11:01.855400+0000","last_clean":"2024-03-30T13:11:01.855400+0000","last_became_active":"2024-03-30T13:10:55.974421+0000","last_became_peered":"2024-03-30T13:10:55.974421+0000","last_unstale":"2024-03-30T13:11:01.855400+0000","last_undegraded":"2024-03-30T13:11:01.855400+0000","last_fullsized":"2024-03-30T13:11:01.855400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:40.791064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857905+0000","last_change":"2024-03-30T13:10:55.998371+0000","last_active":"2024-03-30T13:11:01.857905+0000","last_peered":"2024-03-30T13:11:01.857905+0000","last_clean":"2024-03-30T13:11:01.857905+0000","last_became_active":"2024-03-30T13:10:55.997851+0000","last_became_peered":"2024-03-30T13:10:55.997851+0000","last_unstale":"2024-03-30T13:11:01.857905+0000","last_undegraded":"2024-03-30T13:11:01.857905+0000","last_fullsized":"2024-03-30T13:11:01.857905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:46.705515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855480+0000","last_change":"2024-03-30T13:10:55.967579+0000","last_active":"2024-03-30T13:11:01.855480+0000","last_peered":"2024-03-30T13:11:01.855480+0000","last_clean":"2024-03-30T13:11:01.855480+0000","last_became_active":"2024-03-30T13:10:55.967284+0000","last_became_peered":"2024-03-30T13:10:55.967284+0000","last_unstale":"2024-03-30T13:11:01.855480+0000","last_undegraded":"2024-03-30T13:11:01.855480+0000","last_fullsized":"2024-03-30T13:11:01.855480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:39.989019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858659+0000","last_change":"2024-03-30T13:10:55.998028+0000","last_active":"2024-03-30T13:11:01.858659+0000","last_peered":"2024-03-30T13:11:01.858659+0000","last_clean":"2024-03-30T13:11:01.858659+0000","last_became_active":"2024-03-30T13:10:55.997628+0000","last_became_peered":"2024-03-30T13:10:55.997628+0000","last_unstale":"2024-03-30T13:11:01.858659+0000","last_undegraded":"2024-03-30T13:11:01.858659+0000","last_fullsized":"2024-03-30T13:11:01.858659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982130+0000","last_change":"2024-03-30T13:10:55.983619+0000","last_active":"2024-03-30T13:11:00.982130+0000","last_peered":"2024-03-30T13:11:00.982130+0000","last_clean":"2024-03-30T13:11:00.982130+0000","last_became_active":"2024-03-30T13:10:55.982947+0000","last_became_peered":"2024-03-30T13:10:55.982947+0000","last_unstale":"2024-03-30T13:11:00.982130+0000","last_undegraded":"2024-03-30T13:11:00.982130+0000","last_fullsized":"2024-03-30T13:11:00.982130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:00.291625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983477+0000","last_change":"2024-03-30T13:10:55.992614+0000","last_active":"2024-03-30T13:11:00.983477+0000","last_peered":"2024-03-30T13:11:00.983477+0000","last_clean":"2024-03-30T13:11:00.983477+0000","last_became_active":"2024-03-30T13:10:55.991559+0000","last_became_peered":"2024-03-30T13:10:55.991559+0000","last_unstale":"2024-03-30T13:11:00.983477+0000","last_undegraded":"2024-03-30T13:11:00.983477+0000","last_fullsized":"2024-03-30T13:11:00.983477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:28:13.766039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856251+0000","last_change":"2024-03-30T13:10:55.960973+0000","last_active":"2024-03-30T13:11:01.856251+0000","last_peered":"2024-03-30T13:11:01.856251+0000","last_clean":"2024-03-30T13:11:01.856251+0000","last_became_active":"2024-03-30T13:10:55.960665+0000","last_became_peered":"2024-03-30T13:10:55.960665+0000","last_unstale":"2024-03-30T13:11:01.856251+0000","last_undegraded":"2024-03-30T13:11:01.856251+0000","last_fullsized":"2024-03-30T13:11:01.856251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509449,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:20.551 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T13:11:20.552 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-30T13:11:20.552 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-30T13:11:20.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-30T13:11:20.552 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-30T13:11:20.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 Processor -- start 2024-03-30T13:11:20.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 -- start start 2024-03-30T13:11:20.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006f9e0 0x7f267006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f267006c770 0x7f267006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f26700b4530 0x7f26700b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2670152550 con 0x7f26700b4530 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2670070300 con 0x7f267006c770 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f26749b9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f267006d430 con 0x7f267006f9e0 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f26700b4530 0x7f26700b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f26700b4530 0x7f26700b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41116/0 (socket says 172.21.15.43:41116) 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 -- 172.21.15.43:0/1679215715 learned_addr learned my addr 172.21.15.43:0/1679215715 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266e59c640 1 --2- 172.21.15.43:0/1679215715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006f9e0 0x7f267006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f2667fff640 1 --2- 172.21.15.43:0/1679215715 >> v2:172.21.15.113:3300/0 conn(0x7f267006c770 0x7f267006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 -- 172.21.15.43:0/1679215715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006f9e0 msgr2=0x7f267006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 --2- 172.21.15.43:0/1679215715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006f9e0 0x7f267006fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 -- 172.21.15.43:0/1679215715 >> v2:172.21.15.113:3300/0 conn(0x7f267006c770 msgr2=0x7f267006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:20.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 --2- 172.21.15.43:0/1679215715 >> v2:172.21.15.113:3300/0 conn(0x7f267006c770 0x7f267006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.711+0000 7f266ed9d640 1 -- 172.21.15.43:0/1679215715 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267006d710 con 0x7f26700b4530 2024-03-30T13:11:20.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f266ed9d640 1 --2- 172.21.15.43:0/1679215715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26700b4530 0x7f26700b4980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f26700d1f70 tx=0x7f266801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8ca4b218f213d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:20.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f266dd9b640 1 -- 172.21.15.43:0/1679215715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26680a5840 con 0x7f26700b4530 2024-03-30T13:11:20.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f266dd9b640 1 -- 172.21.15.43:0/1679215715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26680a59e0 con 0x7f26700b4530 2024-03-30T13:11:20.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f266dd9b640 1 -- 172.21.15.43:0/1679215715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26680a5db0 con 0x7f26700b4530 2024-03-30T13:11:20.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- 172.21.15.43:0/1679215715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26700b4530 msgr2=0x7f26700b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 --2- 172.21.15.43:0/1679215715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26700b4530 0x7f26700b4980 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f26700d1f70 tx=0x7f266801c9c0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- 172.21.15.43:0/1679215715 shutdown_connections 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 --2- 172.21.15.43:0/1679215715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26700b4530 0x7f26700b4980 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 --2- 172.21.15.43:0/1679215715 >> v2:172.21.15.113:3300/0 conn(0x7f267006c770 0x7f267006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 --2- 172.21.15.43:0/1679215715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006f9e0 0x7f267006fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- 172.21.15.43:0/1679215715 >> 172.21.15.43:0/1679215715 conn(0x7f267006e410 msgr2=0x7f267006e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- 172.21.15.43:0/1679215715 shutdown_connections 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- 172.21.15.43:0/1679215715 wait complete. 2024-03-30T13:11:20.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 Processor -- start 2024-03-30T13:11:20.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- start start 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006c770 0x7f267013bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006f9e0 0x7f26701506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26700b4530 0x7f2670150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f267015c460 con 0x7f267006f9e0 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f267006d130 con 0x7f26700b4530 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.715+0000 7f26749b9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f26700cec30 con 0x7f267006c770 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006f9e0 0x7f26701506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006f9e0 0x7f26701506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41120/0 (socket says 172.21.15.43:41120) 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 -- 172.21.15.43:0/3841984795 learned_addr learned my addr 172.21.15.43:0/3841984795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f266e59c640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006c770 0x7f267013bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f266ed9d640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26700b4530 0x7f2670150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 -- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006c770 msgr2=0x7f267013bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006c770 0x7f267013bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 -- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26700b4530 msgr2=0x7f2670150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26700b4530 0x7f2670150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 -- 172.21.15.43:0/3841984795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267013c070 con 0x7f267006f9e0 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f266ed9d640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26700b4530 0x7f2670150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2667fff640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006f9e0 0x7f26701506d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f265800d800 tx=0x7f265800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2666ffd640 1 -- 172.21.15.43:0/3841984795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265800b6c0 con 0x7f267006f9e0 2024-03-30T13:11:20.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2666ffd640 1 -- 172.21.15.43:0/3841984795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2658019070 con 0x7f267006f9e0 2024-03-30T13:11:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267014abf0 con 0x7f267006f9e0 2024-03-30T13:11:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2666ffd640 1 -- 172.21.15.43:0/3841984795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265800ba60 con 0x7f267006f9e0 2024-03-30T13:11:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26700ce8c0 con 0x7f267006f9e0 2024-03-30T13:11:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2670068890 con 0x7f267006f9e0 2024-03-30T13:11:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2666ffd640 1 -- 172.21.15.43:0/3841984795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f265800bc00 con 0x7f267006f9e0 2024-03-30T13:11:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2666ffd640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2650067fc0 0x7f265006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f2666ffd640 1 -- 172.21.15.43:0/3841984795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2658086d70 con 0x7f267006f9e0 2024-03-30T13:11:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.719+0000 7f266e59c640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2650067fc0 0x7f265006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.723+0000 7f266e59c640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2650067fc0 0x7f265006a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f266000b950 tx=0x7f2660008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.723+0000 7f2666ffd640 1 -- 172.21.15.43:0/3841984795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2658057150 con 0x7f267006f9e0 2024-03-30T13:11:20.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.927+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f26700cd770 con 0x7f267006f9e0 2024-03-30T13:11:20.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.935+0000 7f2666ffd640 1 -- 172.21.15.43:0/3841984795 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+167686 (secure 0 0 0) 0x7f2658056a50 con 0x7f267006f9e0 2024-03-30T13:11:20.945 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2650067fc0 msgr2=0x7f265006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2650067fc0 0x7f265006a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f266000b950 tx=0x7f2660008040 comp rx=0 tx=0).stop 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006f9e0 msgr2=0x7f26701506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006f9e0 0x7f26701506d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f265800d800 tx=0x7f265800dcc0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 shutdown_connections 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2650067fc0 0x7f265006a480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26700b4530 0x7f2670150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006f9e0 0x7f26701506d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 --2- 172.21.15.43:0/3841984795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267006c770 0x7f267013bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:20.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 >> 172.21.15.43:0/3841984795 conn(0x7f267006e410 msgr2=0x7f267014f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:20.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 shutdown_connections 2024-03-30T13:11:20.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:20.939+0000 7f26749b9640 1 -- 172.21.15.43:0/3841984795 wait complete. 2024-03-30T13:11:20.964 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"flags":0,"active_gid":4106,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6832","nonce":153501405},{"type":"v1","addr":"172.21.15.113:6833","nonce":153501405}]},"active_addr":"172.21.15.113:6833/153501405","active_change":"2024-03-30T13:10:35.816675+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1791291491}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1934990242}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":1817455576}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.113:0","nonce":736850648}]}]} 2024-03-30T13:11:20.967 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-30T13:11:20.967 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-30T13:11:20.967 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:21.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af9001640 1 Processor -- start 2024-03-30T13:11:21.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af9001640 1 -- start start 2024-03-30T13:11:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af9001640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4af4142440 0x7f4af40cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af9001640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4af40cf5e0 0x7f4af40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af9001640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4af40cfb60 con 0x7f4af40cd680 2024-03-30T13:11:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af9001640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4af40cdad0 con 0x7f4af4142440 2024-03-30T13:11:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af9001640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4af40cfe40 con 0x7f4af40cf5e0 2024-03-30T13:11:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af2d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4af4142440 0x7f4af40cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af2d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4af4142440 0x7f4af40cf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41128/0 (socket says 172.21.15.43:41128) 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af2d9d640 1 -- 172.21.15.43:0/3904302490 learned_addr learned my addr 172.21.15.43:0/3904302490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af359e640 1 --1- 172.21.15.43:0/3904302490 >> v1:172.21.15.43:6789/0 conn(0x7f4af40cd680 0x7f4af415fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50488/0 (socket says 172.21.15.43:50488) 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af259c640 1 --2- 172.21.15.43:0/3904302490 >> v2:172.21.15.113:3300/0 conn(0x7f4af40cf5e0 0x7f4af40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af2d9d640 1 -- 172.21.15.43:0/3904302490 >> v2:172.21.15.113:3300/0 conn(0x7f4af40cf5e0 msgr2=0x7f4af40cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af2d9d640 1 --2- 172.21.15.43:0/3904302490 >> v2:172.21.15.113:3300/0 conn(0x7f4af40cf5e0 0x7f4af40cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af2d9d640 1 -- 172.21.15.43:0/3904302490 >> v1:172.21.15.43:6789/0 conn(0x7f4af40cd680 legacy=0x7f4af415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af2d9d640 1 -- 172.21.15.43:0/3904302490 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af40d0120 con 0x7f4af4142440 2024-03-30T13:11:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.095+0000 7f4af259c640 1 --2- 172.21.15.43:0/3904302490 >> v2:172.21.15.113:3300/0 conn(0x7f4af40cf5e0 0x7f4af40cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af2d9d640 1 --2- 172.21.15.43:0/3904302490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af40cf0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4af40b3080 tx=0x7f4ae401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99ee7ca588e1991 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af1d9b640 1 -- 172.21.15.43:0/3904302490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40aa650 con 0x7f4af4142440 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af1d9b640 1 -- 172.21.15.43:0/3904302490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ae40af070 con 0x7f4af4142440 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af1d9b640 1 -- 172.21.15.43:0/3904302490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40aaa10 con 0x7f4af4142440 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- 172.21.15.43:0/3904302490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 msgr2=0x7f4af40cf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 --2- 172.21.15.43:0/3904302490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af40cf0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4af40b3080 tx=0x7f4ae401c4f0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- 172.21.15.43:0/3904302490 shutdown_connections 2024-03-30T13:11:21.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 --2- 172.21.15.43:0/3904302490 >> v2:172.21.15.113:3300/0 conn(0x7f4af40cf5e0 0x7f4af40cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 --2- 172.21.15.43:0/3904302490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af40cf0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- 172.21.15.43:0/3904302490 >> 172.21.15.43:0/3904302490 conn(0x7f4af40b2b30 msgr2=0x7f4af40d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- 172.21.15.43:0/3904302490 shutdown_connections 2024-03-30T13:11:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- 172.21.15.43:0/3904302490 wait complete. 2024-03-30T13:11:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 Processor -- start 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- start start 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4af40cd680 0x7f4af4141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4af40cf5e0 0x7f4af4133c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af4158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40ccbe0 con 0x7f4af4142440 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4af40cca60 con 0x7f4af40cf5e0 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af9001640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4af4169140 con 0x7f4af40cd680 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af4158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af4158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41142/0 (socket says 172.21.15.43:41142) 2024-03-30T13:11:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 -- 172.21.15.43:0/1843567303 learned_addr learned my addr 172.21.15.43:0/1843567303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af2d9d640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4af40cd680 0x7f4af4141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af259c640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4af40cf5e0 0x7f4af4133c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 -- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4af40cd680 msgr2=0x7f4af4141370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4af40cd680 0x7f4af4141370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 -- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4af40cf5e0 msgr2=0x7f4af4133c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4af40cf5e0 0x7f4af4133c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af359e640 1 -- 172.21.15.43:0/1843567303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af415c2b0 con 0x7f4af4142440 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.099+0000 7f4af259c640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4af40cf5e0 0x7f4af4133c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4af359e640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af4158aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00c710 tx=0x7f4aec00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4adb7fe640 1 -- 172.21.15.43:0/1843567303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec017070 con 0x7f4af4142440 2024-03-30T13:11:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4adb7fe640 1 -- 172.21.15.43:0/1843567303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aec00ce20 con 0x7f4af4142440 2024-03-30T13:11:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af406f940 con 0x7f4af4142440 2024-03-30T13:11:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4adb7fe640 1 -- 172.21.15.43:0/1843567303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00a860 con 0x7f4af4142440 2024-03-30T13:11:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af406ea10 con 0x7f4af4142440 2024-03-30T13:11:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af4068890 con 0x7f4af4142440 2024-03-30T13:11:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4adb7fe640 1 -- 172.21.15.43:0/1843567303 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4aec01a070 con 0x7f4af4142440 2024-03-30T13:11:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4adb7fe640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4ae0067fc0 0x7f4ae006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4adb7fe640 1 -- 172.21.15.43:0/1843567303 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4aec086170 con 0x7f4af4142440 2024-03-30T13:11:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4af2d9d640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4ae0067fc0 0x7f4ae006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.103+0000 7f4af2d9d640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4ae0067fc0 0x7f4ae006a480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40ae000 tx=0x7f4ae40a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.107+0000 7f4adb7fe640 1 -- 172.21.15.43:0/1843567303 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4aec021020 con 0x7f4af4142440 2024-03-30T13:11:21.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.287+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4af40d1470 con 0x7f4af4142440 2024-03-30T13:11:21.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4adb7fe640 1 -- 172.21.15.43:0/1843567303 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7f4aec056750 con 0x7f4af4142440 2024-03-30T13:11:21.298 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:21.298 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4ae0067fc0 msgr2=0x7f4ae006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4ae0067fc0 0x7f4ae006a480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ae40ae000 tx=0x7f4ae40a7000 comp rx=0 tx=0).stop 2024-03-30T13:11:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 msgr2=0x7f4af4158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af4158aa0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4aec00c710 tx=0x7f4aec00cbd0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 shutdown_connections 2024-03-30T13:11:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4ae0067fc0 0x7f4ae006a480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4af4142440 0x7f4af4158aa0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4af40cf5e0 0x7f4af4133c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 --2- 172.21.15.43:0/1843567303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4af40cd680 0x7f4af4141370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 >> 172.21.15.43:0/1843567303 conn(0x7f4af40b2b30 msgr2=0x7f4af415ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 shutdown_connections 2024-03-30T13:11:21.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.291+0000 7f4af9001640 1 -- 172.21.15.43:0/1843567303 wait complete. 2024-03-30T13:11:21.322 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-30T13:11:21.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f5961175640 1 Processor -- start 2024-03-30T13:11:21.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f5961175640 1 -- start start 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f5961175640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f595c0b5540 0x7f595c06c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f5961175640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06ccb0 0x7f595c164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f5961175640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f595c06d430 con 0x7f595c06f9e0 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f5961175640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f595c152550 con 0x7f595c0b5540 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f5961175640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f595c06d710 con 0x7f595c06ccb0 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f595c0b5540 0x7f595c06c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f595c0b5540 0x7f595c06c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34728/0 (socket says 172.21.15.43:34728) 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595a59c640 1 -- 172.21.15.43:0/2996248121 learned_addr learned my addr 172.21.15.43:0/2996248121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595b59e640 1 --2- 172.21.15.43:0/2996248121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06ccb0 0x7f595c164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595a59c640 1 -- 172.21.15.43:0/2996248121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06ccb0 msgr2=0x7f595c164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595a59c640 1 --2- 172.21.15.43:0/2996248121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06ccb0 0x7f595c164bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595a59c640 1 -- 172.21.15.43:0/2996248121 >> v1:172.21.15.113:6789/0 conn(0x7f595c06f9e0 legacy=0x7f595c06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.447+0000 7f595a59c640 1 -- 172.21.15.43:0/2996248121 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c06d9f0 con 0x7f595c0b5540 2024-03-30T13:11:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f595a59c640 1 --2- 172.21.15.43:0/2996248121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c06c770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5950009870 tx=0x7f595001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fa8b86bbd52fd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5959d9b640 1 -- 172.21.15.43:0/2996248121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500aa610 con 0x7f595c0b5540 2024-03-30T13:11:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5959d9b640 1 -- 172.21.15.43:0/2996248121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59500af070 con 0x7f595c0b5540 2024-03-30T13:11:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5959d9b640 1 -- 172.21.15.43:0/2996248121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500aa9d0 con 0x7f595c0b5540 2024-03-30T13:11:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- 172.21.15.43:0/2996248121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 msgr2=0x7f595c06c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 --2- 172.21.15.43:0/2996248121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c06c770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5950009870 tx=0x7f595001c4f0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- 172.21.15.43:0/2996248121 shutdown_connections 2024-03-30T13:11:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 --2- 172.21.15.43:0/2996248121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06ccb0 0x7f595c164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 --2- 172.21.15.43:0/2996248121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c06c770 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- 172.21.15.43:0/2996248121 >> 172.21.15.43:0/2996248121 conn(0x7f595c06e410 msgr2=0x7f595c06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- 172.21.15.43:0/2996248121 shutdown_connections 2024-03-30T13:11:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- 172.21.15.43:0/2996248121 wait complete. 2024-03-30T13:11:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 Processor -- start 2024-03-30T13:11:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- start start 2024-03-30T13:11:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f595c06ccb0 0x7f595c1f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06f9e0 0x7f595c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c0b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f595c15c460 con 0x7f595c0b5540 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f595c167950 con 0x7f595c06ccb0 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f5961175640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f595c1f8310 con 0x7f595c06f9e0 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f595b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c0b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f595b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c0b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34740/0 (socket says 172.21.15.43:34740) 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f595b59e640 1 -- 172.21.15.43:0/3387981741 learned_addr learned my addr 172.21.15.43:0/3387981741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f595a59c640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06f9e0 0x7f595c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.451+0000 7f595ad9d640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f595c06ccb0 0x7f595c1f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595b59e640 1 -- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06f9e0 msgr2=0x7f595c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595b59e640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06f9e0 0x7f595c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595b59e640 1 -- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f595c06ccb0 msgr2=0x7f595c1f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595b59e640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f595c06ccb0 0x7f595c1f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595b59e640 1 -- 172.21.15.43:0/3387981741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c1505d0 con 0x7f595c0b5540 2024-03-30T13:11:21.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595ad9d640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f595c06ccb0 0x7f595c1f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595b59e640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c0b0be0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f594c00cd70 tx=0x7f594c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f59437fe640 1 -- 172.21.15.43:0/3387981741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c003ba0 con 0x7f595c0b5540 2024-03-30T13:11:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f59437fe640 1 -- 172.21.15.43:0/3387981741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c003d40 con 0x7f595c0b5540 2024-03-30T13:11:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c1508b0 con 0x7f595c0b5540 2024-03-30T13:11:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f59437fe640 1 -- 172.21.15.43:0/3387981741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c004240 con 0x7f595c0b5540 2024-03-30T13:11:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c150e70 con 0x7f595c0b5540 2024-03-30T13:11:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c068890 con 0x7f595c0b5540 2024-03-30T13:11:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f59437fe640 1 -- 172.21.15.43:0/3387981741 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f594c0043e0 con 0x7f595c0b5540 2024-03-30T13:11:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f59437fe640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5948067fc0 0x7f594806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f59437fe640 1 -- 172.21.15.43:0/3387981741 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f594c087410 con 0x7f595c0b5540 2024-03-30T13:11:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595ad9d640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5948067fc0 0x7f594806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.455+0000 7f595ad9d640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5948067fc0 0x7f594806a480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5954003e40 tx=0x7f5954009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.459+0000 7f59437fe640 1 -- 172.21.15.43:0/3387981741 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f594c0577f0 con 0x7f595c0b5540 2024-03-30T13:11:21.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.691+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f595c0cd770 con 0x7f595c0b5540 2024-03-30T13:11:21.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.695+0000 7f59437fe640 1 -- 172.21.15.43:0/3387981741 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7f594c0570f0 con 0x7f595c0b5540 2024-03-30T13:11:21.702 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:21.702 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:21.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5948067fc0 msgr2=0x7f594806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5948067fc0 0x7f594806a480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5954003e40 tx=0x7f5954009040 comp rx=0 tx=0).stop 2024-03-30T13:11:21.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 msgr2=0x7f595c0b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c0b0be0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f594c00cd70 tx=0x7f594c00a660 comp rx=0 tx=0).stop 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 shutdown_connections 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5948067fc0 0x7f594806a480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f595c0b5540 0x7f595c0b0be0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f595c06f9e0 0x7f595c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 --2- 172.21.15.43:0/3387981741 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f595c06ccb0 0x7f595c1f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 >> 172.21.15.43:0/3387981741 conn(0x7f595c06e410 msgr2=0x7f595c1599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 shutdown_connections 2024-03-30T13:11:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.699+0000 7f5961175640 1 -- 172.21.15.43:0/3387981741 wait complete. 2024-03-30T13:11:21.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-30T13:11:21.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-30T13:11:21.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-30T13:11:21.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-30T13:11:21.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-30T13:11:21.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-30T13:11:21.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-30T13:11:21.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-30T13:11:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.899+0000 7fb63ca15640 1 Processor -- start 2024-03-30T13:11:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.899+0000 7fb63ca15640 1 -- start start 2024-03-30T13:11:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb63ca15640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6380b50c0 0x7fb638072c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb63ca15640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 0x7fb63806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb636d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb63816b460 0x7fb63816b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59610/0 (socket says 172.21.15.43:59610) 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb63659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6380b50c0 0x7fb638072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb636d9d640 1 -- 172.21.15.43:0/3382154650 learned_addr learned my addr 172.21.15.43:0/3382154650 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb63806db70 con 0x7fb63816b460 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb62ffff640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 0x7fb63806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb638071eb0 con 0x7fb6380b50c0 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb638072190 con 0x7fb63806d670 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3113144817 0 0) 0x7fb63806db70 con 0x7fb63816b460 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb624003530 con 0x7fb63816b460 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3440790324 0 0) 0x7fb624003530 con 0x7fb63816b460 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 msgr2=0x7fb63806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 0x7fb63806dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 >> v2:172.21.15.43:3300/0 conn(0x7fb6380b50c0 msgr2=0x7fb638072c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 --2- 172.21.15.43:0/3382154650 >> v2:172.21.15.43:3300/0 conn(0x7fb6380b50c0 0x7fb638072c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb638072470 con 0x7fb63816b460 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fb630001a00 con 0x7fb63816b460 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 >> v1:172.21.15.43:6789/0 conn(0x7fb63816b460 legacy=0x7fb63816b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6240039c0 0x7fb624003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb624004810 0x7fb624004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 0x7fb6240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb638071eb0 con 0x7fb624004810 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb638072190 con 0x7fb6240039c0 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb62401c9e0 con 0x7fb63806d670 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb63659c640 1 --2- 172.21.15.43:0/3382154650 >> v2:172.21.15.43:3300/0 conn(0x7fb6380b50c0 0x7fb638072c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb62ffff640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb624004810 0x7fb624004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb636d9d640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 0x7fb6240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb62ffff640 1 -- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 msgr2=0x7fb6240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb62ffff640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 0x7fb6240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb62ffff640 1 -- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6240039c0 msgr2=0x7fb624003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb62ffff640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6240039c0 0x7fb624003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb62ffff640 1 -- 172.21.15.43:0/3382154650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb624003530 con 0x7fb624004810 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.903+0000 7fb636d9d640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63806d670 0x7fb6240051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 Processor -- start 2024-03-30T13:11:21.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 -- start start 2024-03-30T13:11:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fb62ffff640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb624004810 0x7fb624004c90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb62000dd10 tx=0x7fb62001c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1faf57e1e3796f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa86815e400 0x7fa8680b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8680ad8e0 0x7fa86806d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa86806dcc0 0x7fa86806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fa86806f190 con 0x7fa8680ad8e0 2024-03-30T13:11:21.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa8680b08a0 con 0x7fa86806dcc0 2024-03-30T13:11:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa87057c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa86806f3e0 con 0x7fa86815e400 2024-03-30T13:11:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62000d960 con 0x7fb624004810 2024-03-30T13:11:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6200af070 con 0x7fb624004810 2024-03-30T13:11:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fb635d9b640 1 -- 172.21.15.43:0/3382154650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200aa8c0 con 0x7fb624004810 2024-03-30T13:11:21.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8680ad8e0 0x7fa86806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8680ad8e0 0x7fa86806d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34774/0 (socket says 172.21.15.43:34774) 2024-03-30T13:11:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 -- 172.21.15.43:0/895088353 learned_addr learned my addr 172.21.15.43:0/895088353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86e318640 1 --2- 172.21.15.43:0/895088353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa86815e400 0x7fa8680b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86eb19640 1 --2- 172.21.15.43:0/895088353 >> v2:172.21.15.113:3300/0 conn(0x7fa86806dcc0 0x7fa86806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb624004810 msgr2=0x7fb624004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fb63ca15640 1 --2- 172.21.15.43:0/3382154650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb624004810 0x7fb624004c90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb62000dd10 tx=0x7fb62001c660 comp rx=0 tx=0).stop 2024-03-30T13:11:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fb636d9d640 1 -- 172.21.15.43:0/3382154650 reap_dead start 2024-03-30T13:11:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 shutdown_connections 2024-03-30T13:11:21.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 >> 172.21.15.43:0/3382154650 conn(0x7fb63806b660 msgr2=0x7fb63806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 -- 172.21.15.43:0/895088353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa86815e400 msgr2=0x7fa8680b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 --2- 172.21.15.43:0/895088353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa86815e400 0x7fa8680b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 -- 172.21.15.43:0/895088353 >> v2:172.21.15.113:3300/0 conn(0x7fa86806dcc0 msgr2=0x7fa86806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 --2- 172.21.15.43:0/895088353 >> v2:172.21.15.113:3300/0 conn(0x7fa86806dcc0 0x7fa86806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.907+0000 7fa86db17640 1 -- 172.21.15.43:0/895088353 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86806f6c0 con 0x7fa8680ad8e0 2024-03-30T13:11:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa86e318640 1 --2- 172.21.15.43:0/895088353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa86815e400 0x7fa8680b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 shutdown_connections 2024-03-30T13:11:21.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fb63ca15640 1 -- 172.21.15.43:0/3382154650 wait complete. 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa86db17640 1 --2- 172.21.15.43:0/895088353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8680ad8e0 0x7fa86806d780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa8680cd070 tx=0x7fa85801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc123a56f15724b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa86d316640 1 -- 172.21.15.43:0/895088353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8580b0070 con 0x7fa8680ad8e0 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa86d316640 1 -- 172.21.15.43:0/895088353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8580ab600 con 0x7fa8680ad8e0 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa86d316640 1 -- 172.21.15.43:0/895088353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8580aba10 con 0x7fa8680ad8e0 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 -- 172.21.15.43:0/895088353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8680ad8e0 msgr2=0x7fa86806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 --2- 172.21.15.43:0/895088353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8680ad8e0 0x7fa86806d780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa8680cd070 tx=0x7fa85801c280 comp rx=0 tx=0).stop 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 -- 172.21.15.43:0/895088353 shutdown_connections 2024-03-30T13:11:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 --2- 172.21.15.43:0/895088353 >> v2:172.21.15.113:3300/0 conn(0x7fa86806dcc0 0x7fa86806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 --2- 172.21.15.43:0/895088353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8680ad8e0 0x7fa86806d780 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 --2- 172.21.15.43:0/895088353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa86815e400 0x7fa8680b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 -- 172.21.15.43:0/895088353 >> 172.21.15.43:0/895088353 conn(0x7fa868072500 msgr2=0x7fa868072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 -- 172.21.15.43:0/895088353 shutdown_connections 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.911+0000 7fa87057c640 1 -- 172.21.15.43:0/895088353 wait complete. 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 Processor -- start 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 -- start start 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fb63ca15640 1 Processor -- start 2024-03-30T13:11:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fb63ca15640 1 -- start start 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa86806dcc0 0x7fa86814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8680ad8e0 0x7fa8681582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86815e400 0x7fa868158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680affe0 con 0x7fa86815e400 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8680afe60 con 0x7fa86806dcc0 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa8680b54e0 con 0x7fa8680ad8e0 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86815e400 0x7fa868158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86815e400 0x7fa868158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34788/0 (socket says 172.21.15.43:34788) 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 -- 172.21.15.43:0/3448818611 learned_addr learned my addr 172.21.15.43:0/3448818611 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86db17640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8680ad8e0 0x7fa8681582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86e318640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa86806dcc0 0x7fa86814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 -- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8680ad8e0 msgr2=0x7fa8681582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8680ad8e0 0x7fa8681582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 -- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa86806dcc0 msgr2=0x7fa86814eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa86806dcc0 0x7fa86814eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 -- 172.21.15.43:0/3448818611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8681614e0 con 0x7fa86815e400 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa86eb19640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86815e400 0x7fa868158830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa860007e40 tx=0x7fa86000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860014620 con 0x7fa86815e400 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa860019070 con 0x7fa86815e400 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa860014a20 con 0x7fa86815e400 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 -- 172.21.15.43:0/3448818611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8681617c0 con 0x7fa86815e400 2024-03-30T13:11:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 -- 172.21.15.43:0/3448818611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa868159770 con 0x7fa86815e400 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.915+0000 7fa87057c640 1 -- 172.21.15.43:0/3448818611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa868068890 con 0x7fa86815e400 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa860014bc0 con 0x7fa86815e400 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa85effd640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa84c067fc0 0x7fa84c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa860087f90 con 0x7fa86815e400 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa85effd640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7fa84c0716c0 0x7fa84c073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 --> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa860016e70 con 0x7fa84c0716c0 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa860088420 con 0x7fa86815e400 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63ca15640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63815bb20 0x7fb6380ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63ca15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6380affb0 0x7fb6380ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa86db17640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7fa84c0716c0 0x7fa84c073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa86e318640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa84c067fc0 0x7fa84c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63815bb20 0x7fb6380ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6380affb0 0x7fb6380ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6380affb0 0x7fb6380ace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34802/0 (socket says 172.21.15.43:34802) 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63815bb20 0x7fb6380ab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37254/0 (socket says 172.21.15.43:37254) 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 -- 172.21.15.43:0/1631795249 learned_addr learned my addr 172.21.15.43:0/1631795249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63ca15640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb638167ce0 0x7fb638168160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63ca15640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb63806e170 con 0x7fb6380affb0 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63ca15640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb63806dff0 con 0x7fb638167ce0 2024-03-30T13:11:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb63ca15640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb63806e2f0 con 0x7fb63815bb20 2024-03-30T13:11:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa86db17640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7fa84c0716c0 0x7fa84c073b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb636d9d640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb638167ce0 0x7fb638168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa86e318640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa84c067fc0 0x7fa84c06a480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa86400d8d0 tx=0x7fa864008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 -- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63815bb20 msgr2=0x7fb6380ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb63815bb20 0x7fb6380ab460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 -- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb638167ce0 msgr2=0x7fb638168160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb638167ce0 0x7fb638168160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6380abdd0 con 0x7fb6380affb0 2024-03-30T13:11:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62ffff640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6380affb0 0x7fb6380ace70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb62001c490 tx=0x7fb62000d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== osd.1 v2:172.21.15.43:6816/2193216203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa860016e70 con 0x7fa84c0716c0 2024-03-30T13:11:21.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200061c0 con 0x7fb6380affb0 2024-03-30T13:11:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb620006380 con 0x7fb6380affb0 2024-03-30T13:11:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620006770 con 0x7fb6380affb0 2024-03-30T13:11:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7ffafb0d5640 1 Processor -- start 2024-03-30T13:11:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.919+0000 7ffafb0d5640 1 -- start start 2024-03-30T13:11:21.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb26a6e9640 1 Processor -- start 2024-03-30T13:11:21.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb26a6e9640 1 -- start start 2024-03-30T13:11:21.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb63ca15640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6380ac0b0 con 0x7fb6380affb0 2024-03-30T13:11:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffafb0d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffaec0e7820 0x7ffaec038290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffafb0d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaec0e7c00 con 0x7ffaec0eb310 2024-03-30T13:11:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffafb0d5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaec057f30 con 0x7ffaec0061b0 2024-03-30T13:11:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffaec0eb310 0x7ffaec0eb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59616/0 (socket says 172.21.15.43:59616) 2024-03-30T13:11:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 -- 172.21.15.43:0/3550953034 learned_addr learned my addr 172.21.15.43:0/3550953034 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffafb0d5640 1 -- 172.21.15.43:0/3550953034 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ffaec058210 con 0x7ffaec0e7820 2024-03-30T13:11:21.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb26a6e9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb264147e80 0x7fb26406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb26a6e9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c1c0 0x7fb264161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb26a6e9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb26406bc40 con 0x7fb26406c640 2024-03-30T13:11:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb26a6e9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb264070c50 con 0x7fb264147e80 2024-03-30T13:11:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb26a6e9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb264070600 con 0x7fb26406c1c0 2024-03-30T13:11:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5895613 0 0) 0x7ffaec057f30 con 0x7ffaec0061b0 2024-03-30T13:11:21.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffae0003530 con 0x7ffaec0061b0 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb263fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb264147e80 0x7fb26406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb263fff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb264147e80 0x7fb26406b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:46730/0 (socket says 172.21.15.43:46730) 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb263fff640 1 -- 172.21.15.43:0/1880831778 learned_addr learned my addr 172.21.15.43:0/1880831778 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb268c86640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb26406c640 0x7fb26415e9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:44460/0 (socket says 172.21.15.43:44460) 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb2637fe640 1 --2- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c1c0 0x7fb264161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb263fff640 1 -- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c1c0 msgr2=0x7fb264161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb263fff640 1 --2- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c1c0 0x7fb264161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb263fff640 1 -- 172.21.15.43:0/1880831778 >> v1:172.21.15.113:6789/0 conn(0x7fb26406c640 legacy=0x7fb26415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb263fff640 1 -- 172.21.15.43:0/1880831778 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2640708e0 con 0x7fb264147e80 2024-03-30T13:11:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279925095 0 0) 0x7ffaec0e7c00 con 0x7ffaec0eb310 2024-03-30T13:11:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaec057f30 con 0x7ffaec0eb310 2024-03-30T13:11:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3957410240 0 0) 0x7ffae0003530 con 0x7ffaec0061b0 2024-03-30T13:11:21.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 >> v2:172.21.15.43:3300/0 conn(0x7ffaec0e7820 msgr2=0x7ffaec038290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 --2- 172.21.15.43:0/3550953034 >> v2:172.21.15.43:3300/0 conn(0x7ffaec0e7820 0x7ffaec038290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 >> v1:172.21.15.43:6789/0 conn(0x7ffaec0eb310 legacy=0x7ffaec0eb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec0584f0 con 0x7ffaec0061b0 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ffaf40696a0 con 0x7ffaec0061b0 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 >> v1:172.21.15.113:6789/0 conn(0x7ffaec0061b0 legacy=0x7ffaec006640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0003b50 0x7ffae0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae0004810 0x7ffae0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffae00051d0 0x7ffae000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec058210 con 0x7ffae0003b50 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffae001d0b0 con 0x7ffae0004810 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ffae001d2b0 con 0x7ffae00051d0 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae0004810 0x7ffae0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffafa8d4640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffae00051d0 0x7ffae000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 -- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffae00051d0 msgr2=0x7ffae000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffae00051d0 0x7ffae000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 -- 172.21.15.43:0/3550953034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0003b50 msgr2=0x7ffae0003f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0003b50 0x7ffae0003f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffaf98d2640 1 -- 172.21.15.43:0/3550953034 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec0584f0 con 0x7ffae0004810 2024-03-30T13:11:21.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7ffafa8d4640 1 -- 172.21.15.43:0/3550953034 reap_dead start 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.923+0000 7fb63ca15640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6380ac670 con 0x7fb6380affb0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb6200a8050 con 0x7fb6380affb0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62effd640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb618067ff0 0x7fb61806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb610001040 con 0x7fb6380affb0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb620125060 con 0x7fb6380affb0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62effd640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7fb6180716f0 0x7fb618073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb620004480 con 0x7fb6180716f0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb636d9d640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7fb6180716f0 0x7fb618073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb63659c640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb618067ff0 0x7fb61806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6200ec5d0 con 0x7fb6380affb0 2024-03-30T13:11:21.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb636d9d640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7fb6180716f0 0x7fb618073b30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb63659c640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb618067ff0 0x7fb61806a4b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb628014940 tx=0x7fb62800d040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb263fff640 1 --2- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb264147e80 0x7fb26406b660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009e50 tx=0x7fb25c01c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6caf0e2afedbd128 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffaf98d2640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae0004810 0x7ffae0004c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffae40015c0 tx=0x7ffae403e660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95048aa6667b02f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb262ffd640 1 -- 172.21.15.43:0/1880831778 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0a5780 con 0x7fb264147e80 2024-03-30T13:11:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb262ffd640 1 -- 172.21.15.43:0/1880831778 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb25c0a5920 con 0x7fb264147e80 2024-03-30T13:11:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb262ffd640 1 -- 172.21.15.43:0/1880831778 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0a5cf0 con 0x7fb264147e80 2024-03-30T13:11:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4001600 con 0x7ffae0004810 2024-03-30T13:11:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffae4004ab0 con 0x7ffae0004810 2024-03-30T13:11:21.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffaf90d1640 1 -- 172.21.15.43:0/3550953034 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4007070 con 0x7ffae0004810 2024-03-30T13:11:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb26a6e9640 1 -- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb264147e80 msgr2=0x7fb26406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb26a6e9640 1 --2- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb264147e80 0x7fb26406b660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009e50 tx=0x7fb25c01c9f0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffafb0d5640 1 -- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae0004810 msgr2=0x7ffae0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffafb0d5640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae0004810 0x7ffae0004c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffae40015c0 tx=0x7ffae403e660 comp rx=0 tx=0).stop 2024-03-30T13:11:21.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== osd.3 v2:172.21.15.43:6801/367000764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb620004480 con 0x7fb6180716f0 2024-03-30T13:11:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffafb0d5640 1 -- 172.21.15.43:0/3550953034 shutdown_connections 2024-03-30T13:11:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffafb0d5640 1 --2- 172.21.15.43:0/3550953034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffae0004810 0x7ffae0004c90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffafb0d5640 1 -- 172.21.15.43:0/3550953034 >> 172.21.15.43:0/3550953034 conn(0x7ffaec0559f0 msgr2=0x7ffaec055e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.927+0000 7ffafb0d5640 1 -- 172.21.15.43:0/3550953034 shutdown_connections 2024-03-30T13:11:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fb26a6e9640 1 -- 172.21.15.43:0/1880831778 shutdown_connections 2024-03-30T13:11:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fb26a6e9640 1 --2- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c1c0 0x7fb264161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fb26a6e9640 1 --2- 172.21.15.43:0/1880831778 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb264147e80 0x7fb26406b660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fb26a6e9640 1 -- 172.21.15.43:0/1880831778 >> 172.21.15.43:0/1880831778 conn(0x7fb26406e410 msgr2=0x7fb26406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe78cdfa640 1 Processor -- start 2024-03-30T13:11:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7ffafb0d5640 1 -- 172.21.15.43:0/3550953034 wait complete. 2024-03-30T13:11:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe78cdfa640 1 -- start start 2024-03-30T13:11:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe78cdfa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe7881459e0 0x7fe7880cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe78cdfa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7880cdbe0 0x7fe7880b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe78cdfa640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7880ce160 con 0x7fe7880b4540 2024-03-30T13:11:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe78cdfa640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe78815df80 con 0x7fe7881459e0 2024-03-30T13:11:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe78cdfa640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe78815d930 con 0x7fe7880cdbe0 2024-03-30T13:11:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7ffafb0d5640 1 Processor -- start 2024-03-30T13:11:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe787fff640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fe7880b4540 0x7fe78815e410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:44472/0 (socket says 172.21.15.43:44472) 2024-03-30T13:11:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe787fff640 1 -- 172.21.15.43:0/639359986 learned_addr learned my addr 172.21.15.43:0/639359986 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7fe786ffd640 1 --2- 172.21.15.43:0/639359986 >> v2:172.21.15.113:3300/0 conn(0x7fe7880cdbe0 0x7fe7880b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.931+0000 7ffafb0d5640 1 -- start start 2024-03-30T13:11:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fe7867fc640 1 -- 172.21.15.43:0/639359986 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1794272281 0 0) 0x7fe7880ce160 con 0x7fe7880b4540 2024-03-30T13:11:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fe7867fc640 1 -- 172.21.15.43:0/639359986 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe770003530 con 0x7fe7880b4540 2024-03-30T13:11:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fe786ffd640 1 -- 172.21.15.43:0/639359986 >> v2:172.21.15.43:3300/0 conn(0x7fe7881459e0 msgr2=0x7fe7880cd6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fe786ffd640 1 --2- 172.21.15.43:0/639359986 >> v2:172.21.15.43:3300/0 conn(0x7fe7881459e0 0x7fe7880cd6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fe786ffd640 1 -- 172.21.15.43:0/639359986 >> v1:172.21.15.113:6789/0 conn(0x7fe7880b4540 legacy=0x7fe78815e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fe786ffd640 1 -- 172.21.15.43:0/639359986 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78815dc10 con 0x7fe7880cdbe0 2024-03-30T13:11:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0004810 0x7ffaec0de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffaec0dde00 0x7ffaec0dc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffaec0ea250 0x7ffaec0ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec03ed50 con 0x7ffae0004810 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec054700 con 0x7ffaec0ea250 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ffaec03eed0 con 0x7ffaec0dde00 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffaec0ea250 0x7ffaec0ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffaec0ea250 0x7ffaec0ea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:46748/0 (socket says 172.21.15.43:46748) 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 -- 172.21.15.43:0/2953154967 learned_addr learned my addr 172.21.15.43:0/2953154967 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa0d3640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0004810 0x7ffaec0de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 -- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffaec0dde00 msgr2=0x7ffaec0dc190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffaec0dde00 0x7ffaec0dc190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 -- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0004810 msgr2=0x7ffaec0de6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0004810 0x7ffaec0de6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 -- 172.21.15.43:0/2953154967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec0ddbe0 con 0x7ffaec0ea250 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa0d3640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffae0004810 0x7ffaec0de6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafa8d4640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffaec0ea250 0x7ffaec0ea6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaf404fe20 tx=0x7ffaf40759d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf406c7e0 con 0x7ffaec0ea250 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 -- 172.21.15.43:0/2953154967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec0dfe60 con 0x7ffaec0ea250 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffaf406c980 con 0x7ffaec0ea250 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf406cd30 con 0x7ffaec0ea250 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 -- 172.21.15.43:0/2953154967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec0df0a0 con 0x7ffaec0ea250 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fb26a6e9640 1 -- 172.21.15.43:0/1880831778 shutdown_connections 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fb26a6e9640 1 -- 172.21.15.43:0/1880831778 wait complete. 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fb26a6e9640 1 Processor -- start 2024-03-30T13:11:21.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7fb26a6e9640 1 -- start start 2024-03-30T13:11:21.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.935+0000 7ffafb0d5640 1 -- 172.21.15.43:0/2953154967 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaec0dc6d0 con 0x7ffaec0ea250 2024-03-30T13:11:21.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb26406c1c0 0x7fb2641434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c640 0x7fb26415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb264147e80 0x7fb26415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb26416b4c0 con 0x7fb26406c1c0 2024-03-30T13:11:21.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb26416b340 con 0x7fb264147e80 2024-03-30T13:11:21.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb26416b640 con 0x7fb26406c640 2024-03-30T13:11:21.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe786ffd640 1 --2- 172.21.15.43:0/639359986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 0x7fe7880b4000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7880cea60 tx=0x7fe78001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff17131f0b787d67 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb26406c1c0 0x7fb2641434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb26406c1c0 0x7fb2641434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34824/0 (socket says 172.21.15.43:34824) 2024-03-30T13:11:21.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 -- 172.21.15.43:0/2885211377 learned_addr learned my addr 172.21.15.43:0/2885211377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb2637fe640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c640 0x7fb26415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe7867fc640 1 -- 172.21.15.43:0/639359986 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800b0070 con 0x7fe7880cdbe0 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 -- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c640 msgr2=0x7fb26415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb26406c640 0x7fb26415c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 -- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb264147e80 msgr2=0x7fb26415cd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb264147e80 0x7fb26415cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 -- 172.21.15.43:0/2885211377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2641674d0 con 0x7fb26406c1c0 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb263fff640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb26406c1c0 0x7fb2641434f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb25c01b0a0 tx=0x7fb25c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c0bc020 con 0x7fb26406c1c0 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb25c0b8880 con 0x7fb26406c1c0 2024-03-30T13:11:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c009510 con 0x7fb26406c1c0 2024-03-30T13:11:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ffaf4074ad0 con 0x7ffaec0ea250 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2641677b0 con 0x7fb26406c1c0 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2640cd4d0 con 0x7fb26406c1c0 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe7867fc640 1 -- 172.21.15.43:0/639359986 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7800ab600 con 0x7fe7880cdbe0 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe7867fc640 1 -- 172.21.15.43:0/639359986 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7800ab9b0 con 0x7fe7880cdbe0 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe78cdfa640 1 -- 172.21.15.43:0/639359986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 msgr2=0x7fe7880b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe78cdfa640 1 --2- 172.21.15.43:0/639359986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 0x7fe7880b4000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe7880cea60 tx=0x7fe78001c410 comp rx=0 tx=0).stop 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe78cdfa640 1 -- 172.21.15.43:0/639359986 shutdown_connections 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe78cdfa640 1 --2- 172.21.15.43:0/639359986 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 0x7fe7880b4000 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe78cdfa640 1 --2- 172.21.15.43:0/639359986 >> v2:172.21.15.43:3300/0 conn(0x7fe7881459e0 0x7fe7880cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fe78cdfa640 1 -- 172.21.15.43:0/639359986 >> 172.21.15.43:0/639359986 conn(0x7fe78806c5a0 msgr2=0x7fe78806c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaeaffd640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffad8067fc0 0x7ffad806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ffaf4101830 con 0x7ffaec0ea250 2024-03-30T13:11:21.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaeaffd640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7ffad8071640 0x7ffad8073a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 --> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffad8074150 con 0x7ffad8071640 2024-03-30T13:11:21.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffafa0d3640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffad8067fc0 0x7ffad806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffaf4101cc0 con 0x7ffaec0ea250 2024-03-30T13:11:21.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb264068890 con 0x7fb26406c1c0 2024-03-30T13:11:21.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaf98d2640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7ffad8071640 0x7ffad8073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffafa0d3640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffad8067fc0 0x7ffad806a480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0006fd0 tx=0x7ffaf00090e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.939+0000 7ffaf98d2640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7ffad8071640 0x7ffad8073a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb25c0bf020 con 0x7fb26406c1c0 2024-03-30T13:11:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== osd.2 v2:172.21.15.43:6800/3862342839 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffad8074150 con 0x7ffad8071640 2024-03-30T13:11:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb260ff9640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb250067ff0 0x7fb25006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb25c121360 con 0x7fb26406c1c0 2024-03-30T13:11:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb260ff9640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7fb250071670 0x7fb250073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 --> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb25c0b9e80 con 0x7fb250071670 2024-03-30T13:11:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb25c128050 con 0x7fb26406c1c0 2024-03-30T13:11:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb268c86640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7fb250071670 0x7fb250073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb268c86640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7fb250071670 0x7fb250073ab0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.943+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== osd.5 v2:172.21.15.113:6816/2302616840 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb25c0b9e80 con 0x7fb250071670 2024-03-30T13:11:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 -- 172.21.15.43:0/639359986 shutdown_connections 2024-03-30T13:11:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 -- 172.21.15.43:0/639359986 wait complete. 2024-03-30T13:11:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 Processor -- start 2024-03-30T13:11:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fb2637fe640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb250067ff0 0x7fb25006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 -- start start 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7880b4540 0x7fe7881f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 0x7fe7880722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7881459e0 0x7fe7880727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe78815f6c0 con 0x7fe7881459e0 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe78815f540 con 0x7fe7880cdbe0 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.951+0000 7fe78cdfa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe788168c10 con 0x7fe7880b4540 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7881459e0 0x7fe7880727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7881459e0 0x7fe7880727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34826/0 (socket says 172.21.15.43:34826) 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 -- 172.21.15.43:0/3279484790 learned_addr learned my addr 172.21.15.43:0/3279484790 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe786ffd640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 0x7fe7880722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 -- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7880b4540 msgr2=0x7fe7881f7380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7880b4540 0x7fe7881f7380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 -- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 msgr2=0x7fe7880722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7880cdbe0 0x7fe7880722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 -- 172.21.15.43:0/3279484790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7881440b0 con 0x7fe7881459e0 2024-03-30T13:11:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe787fff640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7881459e0 0x7fe7880727f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe77800caa0 tx=0x7fe77800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fb2637fe640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb250067ff0 0x7fb25006a4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb2640b07d0 tx=0x7fb258008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778017070 con 0x7fe7881459e0 2024-03-30T13:11:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe788144390 con 0x7fe7881459e0 2024-03-30T13:11:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe788203950 con 0x7fe7881459e0 2024-03-30T13:11:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 --> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb610002de0 con 0x7fb6180716f0 2024-03-30T13:11:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77800aba0 con 0x7fe7881459e0 2024-03-30T13:11:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.955+0000 7fb62effd640 1 -- 172.21.15.43:0/1631795249 <== osd.3 v2:172.21.15.43:6801/367000764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb610002de0 con 0x7fb6180716f0 2024-03-30T13:11:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7ffbbd593640 1 Processor -- start 2024-03-30T13:11:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7ffbbd593640 1 -- start start 2024-03-30T13:11:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778012400 con 0x7fe7881459e0 2024-03-30T13:11:21.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe768001040 con 0x7fe7881459e0 2024-03-30T13:11:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe77801a070 con 0x7fe7881459e0 2024-03-30T13:11:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7fb6180716f0 msgr2=0x7fb618073b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7fb6180716f0 0x7fb618073b30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe76ffff640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7700684b0 0x7fe77006a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe778086470 con 0x7fe7881459e0 2024-03-30T13:11:21.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe76ffff640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fe770071c30 0x7fe770074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 --> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe77800aba0 con 0x7fe770071c30 2024-03-30T13:11:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe77808d050 con 0x7fe7881459e0 2024-03-30T13:11:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb618067ff0 msgr2=0x7fb61806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb618067ff0 0x7fb61806a4b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb628014940 tx=0x7fb62800d040 comp rx=0 tx=0).stop 2024-03-30T13:11:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6380affb0 msgr2=0x7fb6380ace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 --2- 172.21.15.43:0/1631795249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6380affb0 0x7fb6380ace70 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb62001c490 tx=0x7fb62000d5a0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb636d9d640 1 -- 172.21.15.43:0/1631795249 reap_dead start 2024-03-30T13:11:21.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 shutdown_connections 2024-03-30T13:11:21.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 >> 172.21.15.43:0/1631795249 conn(0x7fb63806b660 msgr2=0x7fb6380b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 shutdown_connections 2024-03-30T13:11:21.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fb62cff9640 1 -- 172.21.15.43:0/1631795249 wait complete. 2024-03-30T13:11:21.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe7877fe640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7700684b0 0x7fe77006a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe786ffd640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fe770071c30 0x7fe770074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe7877fe640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7700684b0 0x7fe77006a970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0095f0 tx=0x7fe77c006b50 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7fe786ffd640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fe770071c30 0x7fe770074070 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.959+0000 7ffbbd593640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffbb8072c70 0x7ffbb815e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffbb8072c70 0x7ffbb815e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb67fc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ffbb8072c70 0x7ffbb815e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:46788/0 (socket says 172.21.15.43:46788) 2024-03-30T13:11:21.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb6ffd640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ffbb8142f80 0x7ffbb815df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:44478/0 (socket says 172.21.15.43:44478) 2024-03-30T13:11:21.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb67fc640 1 -- 172.21.15.43:0/1506569936 learned_addr learned my addr 172.21.15.43:0/1506569936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb80b48d0 con 0x7ffbb815ebe0 2024-03-30T13:11:21.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb80b5410 con 0x7ffbb8142f80 2024-03-30T13:11:21.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== osd.0 v2:172.21.15.43:6824/3443768212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe77800aba0 con 0x7fe770071c30 2024-03-30T13:11:21.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb80b4b70 con 0x7ffbb8072c70 2024-03-30T13:11:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffafb0d5640 1 -- 172.21.15.43:0/2953154967 --> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffaec0057f0 con 0x7ffad8071640 2024-03-30T13:11:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb5ffb640 1 -- 172.21.15.43:0/1506569936 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152348150 0 0) 0x7ffbb80b5410 con 0x7ffbb8142f80 2024-03-30T13:11:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb5ffb640 1 -- 172.21.15.43:0/1506569936 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffba0003530 con 0x7ffbb8142f80 2024-03-30T13:11:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb5ffb640 1 -- 172.21.15.43:0/1506569936 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2256551376 0 0) 0x7ffbb80b48d0 con 0x7ffbb815ebe0 2024-03-30T13:11:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb5ffb640 1 -- 172.21.15.43:0/1506569936 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffbb80b5410 con 0x7ffbb815ebe0 2024-03-30T13:11:21.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffaeaffd640 1 -- 172.21.15.43:0/2953154967 <== osd.2 v2:172.21.15.43:6800/3862342839 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffaec0057f0 con 0x7ffad8071640 2024-03-30T13:11:21.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb67fc640 1 -- 172.21.15.43:0/1506569936 >> v1:172.21.15.113:6789/0 conn(0x7ffbb8142f80 legacy=0x7ffbb815df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb67fc640 1 -- 172.21.15.43:0/1506569936 >> v1:172.21.15.43:6789/0 conn(0x7ffbb815ebe0 legacy=0x7ffbb806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffbb67fc640 1 -- 172.21.15.43:0/1506569936 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb80b4e50 con 0x7ffbb8072c70 2024-03-30T13:11:21.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7fa87057c640 1 -- 172.21.15.43:0/3448818611 --> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa86814efe0 con 0x7fa84c0716c0 2024-03-30T13:11:21.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffae8ff9640 1 -- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7ffad8071640 msgr2=0x7ffad8073a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7ffae8ff9640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7ffad8071640 0x7ffad8073a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7fa85effd640 1 -- 172.21.15.43:0/3448818611 <== osd.1 v2:172.21.15.43:6816/2193216203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa86814efe0 con 0x7fa84c0716c0 2024-03-30T13:11:21.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7fa85cff9640 1 -- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7fa84c0716c0 msgr2=0x7fa84c073b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7fa85cff9640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7fa84c0716c0 0x7fa84c073b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7fa85cff9640 1 -- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa84c067fc0 msgr2=0x7fa84c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.963+0000 7fa85cff9640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa84c067fc0 0x7fa84c06a480 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa86400d8d0 tx=0x7fa864008040 comp rx=0 tx=0).stop 2024-03-30T13:11:21.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7fa85cff9640 1 -- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86815e400 msgr2=0x7fa868158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7fa85cff9640 1 --2- 172.21.15.43:0/3448818611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa86815e400 0x7fa868158830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa860007e40 tx=0x7fa86000e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7fa86eb19640 1 -- 172.21.15.43:0/3448818611 reap_dead start 2024-03-30T13:11:21.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbb67fc640 1 --2- 172.21.15.43:0/1506569936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb8072c70 0x7ffbb815e6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80b55e0 tx=0x7ffbb001eb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=595a841ea5c77037 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbb5ffb640 1 -- 172.21.15.43:0/1506569936 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00a7950 con 0x7ffbb8072c70 2024-03-30T13:11:21.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbb5ffb640 1 -- 172.21.15.43:0/1506569936 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbb00a7af0 con 0x7ffbb8072c70 2024-03-30T13:11:21.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbb5ffb640 1 -- 172.21.15.43:0/1506569936 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb00af3f0 con 0x7ffbb8072c70 2024-03-30T13:11:21.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb8072c70 msgr2=0x7ffbb815e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbbd593640 1 --2- 172.21.15.43:0/1506569936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb8072c70 0x7ffbb815e6a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80b55e0 tx=0x7ffbb001eb70 comp rx=0 tx=0).stop 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 shutdown_connections 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbbd593640 1 --2- 172.21.15.43:0/1506569936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb8072c70 0x7ffbb815e6a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 >> 172.21.15.43:0/1506569936 conn(0x7ffbb806b660 msgr2=0x7ffbb806ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 shutdown_connections 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.967+0000 7ffbbd593640 1 -- 172.21.15.43:0/1506569936 wait complete. 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7fa85cff9640 1 -- 172.21.15.43:0/3448818611 shutdown_connections 2024-03-30T13:11:21.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 Processor -- start 2024-03-30T13:11:21.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7fa85cff9640 1 -- 172.21.15.43:0/3448818611 >> 172.21.15.43:0/3448818611 conn(0x7fa868072500 msgr2=0x7fa8680b5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 --> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb26406f9e0 con 0x7fb250071670 2024-03-30T13:11:21.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 -- start start 2024-03-30T13:11:21.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7fa85cff9640 1 -- 172.21.15.43:0/3448818611 shutdown_connections 2024-03-30T13:11:21.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7fa85cff9640 1 -- 172.21.15.43:0/3448818611 wait complete. 2024-03-30T13:11:21.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7fb260ff9640 1 -- 172.21.15.43:0/2885211377 <== osd.5 v2:172.21.15.113:6816/2302616840 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb26406f9e0 con 0x7fb250071670 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb8072c70 0x7ffbb813f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffbb8142f80 0x7ffbb8134430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb815ebe0 0x7ffbb8134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb8072c70 0x7ffbb813f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb8072c70 0x7ffbb813f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34828/0 (socket says 172.21.15.43:34828) 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 -- 172.21.15.43:0/89287919 learned_addr learned my addr 172.21.15.43:0/89287919 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb806e920 con 0x7ffbb8072c70 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb806e7a0 con 0x7ffbb815ebe0 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbbd593640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb806eaa0 con 0x7ffbb8142f80 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb77fe640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb815ebe0 0x7ffbb8134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 -- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffbb8142f80 msgr2=0x7ffbb8134430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffbb8142f80 0x7ffbb8134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 -- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb815ebe0 msgr2=0x7ffbb8134970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffbb815ebe0 0x7ffbb8134970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.971+0000 7ffbb6ffd640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb813f830 con 0x7ffbb8072c70 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.975+0000 7ffbb6ffd640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb8072c70 0x7ffbb813f2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c520 tx=0x7ffbac00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.975+0000 7ffb9f7fe640 1 -- 172.21.15.43:0/89287919 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac017070 con 0x7ffbb8072c70 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.975+0000 7ffbbd593640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb8140070 con 0x7ffbb8072c70 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.975+0000 7ffbbd593640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb813fd90 con 0x7ffbb8072c70 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.975+0000 7ffb9f7fe640 1 -- 172.21.15.43:0/89287919 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbac005d10 con 0x7ffbb8072c70 2024-03-30T13:11:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 -- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffad8067fc0 msgr2=0x7ffad806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffad8067fc0 0x7ffad806a480 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0006fd0 tx=0x7ffaf00090e0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 -- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffaec0ea250 msgr2=0x7ffaec0ea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 --2- 172.21.15.43:0/2953154967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffaec0ea250 0x7ffaec0ea6d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffaf404fe20 tx=0x7ffaf40759d0 comp rx=0 tx=0).stop 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffafa8d4640 1 -- 172.21.15.43:0/2953154967 reap_dead start 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 -- 172.21.15.43:0/2953154967 shutdown_connections 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 -- 172.21.15.43:0/2953154967 >> 172.21.15.43:0/2953154967 conn(0x7ffaec0559f0 msgr2=0x7ffaec03d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffb9f7fe640 1 -- 172.21.15.43:0/89287919 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0072b0 con 0x7ffbb8072c70 2024-03-30T13:11:21.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffb9f7fe640 1 -- 172.21.15.43:0/89287919 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ffbac01a070 con 0x7ffbb8072c70 2024-03-30T13:11:21.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 -- 172.21.15.43:0/2953154967 shutdown_connections 2024-03-30T13:11:21.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffae8ff9640 1 -- 172.21.15.43:0/2953154967 wait complete. 2024-03-30T13:11:21.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffb9f7fe640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffba40680c0 0x7ffba406a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:21.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffb9f7fe640 1 -- 172.21.15.43:0/89287919 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ffbac08a020 con 0x7ffbb8072c70 2024-03-30T13:11:21.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffbb67fc640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffba40680c0 0x7ffba406a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7fb250071670 msgr2=0x7fb250073ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7fb250071670 0x7fb250073ab0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb250067ff0 msgr2=0x7fb25006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb250067ff0 0x7fb25006a4b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb2640b07d0 tx=0x7fb258008040 comp rx=0 tx=0).stop 2024-03-30T13:11:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb26406c1c0 msgr2=0x7fb2641434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 --2- 172.21.15.43:0/2885211377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb26406c1c0 0x7fb2641434f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb25c01b0a0 tx=0x7fb25c019790 comp rx=0 tx=0).stop 2024-03-30T13:11:22.000 INFO:teuthology.orchestra.run.smithi043.stdout:64424509451 2024-03-30T13:11:22.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb268c86640 1 -- 172.21.15.43:0/2885211377 reap_dead start 2024-03-30T13:11:22.001 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T13:11:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7ffbb67fc640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffba40680c0 0x7ffba406a580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00b0000 tx=0x7ffbb0004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 shutdown_connections 2024-03-30T13:11:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 >> 172.21.15.43:0/2885211377 conn(0x7fb26406e410 msgr2=0x7fb2640b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 shutdown_connections 2024-03-30T13:11:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.979+0000 7fb26a6e9640 1 -- 172.21.15.43:0/2885211377 wait complete. 2024-03-30T13:11:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.983+0000 7ffbbd593640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7ffb98001660 0x7ffb98003b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.983+0000 7ffbbd593640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb98006bc0 con 0x7ffb98001660 2024-03-30T13:11:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7ffbb77fe640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7ffb98001660 0x7ffb98003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7ffbb77fe640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7ffb98001660 0x7ffb98003b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 --> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe768002de0 con 0x7fe770071c30 2024-03-30T13:11:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7fe76ffff640 1 -- 172.21.15.43:0/3279484790 <== osd.0 v2:172.21.15.43:6824/3443768212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe768002de0 con 0x7fe770071c30 2024-03-30T13:11:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fe770071c30 msgr2=0x7fe770074070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7fe78cdfa640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fe770071c30 0x7fe770074070 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7ffb9f7fe640 1 -- 172.21.15.43:0/89287919 <== osd.7 v2:172.21.15.113:6808/3435220755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb98006bc0 con 0x7ffb98001660 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.987+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7700684b0 msgr2=0x7fe77006a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe78cdfa640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7700684b0 0x7fe77006a970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe77c0095f0 tx=0x7fe77c006b50 comp rx=0 tx=0).stop 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7881459e0 msgr2=0x7fe7880727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe78cdfa640 1 --2- 172.21.15.43:0/3279484790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7881459e0 0x7fe7880727f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe77800caa0 tx=0x7fe77800cf60 comp rx=0 tx=0).stop 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe787fff640 1 -- 172.21.15.43:0/3279484790 reap_dead start 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 shutdown_connections 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 >> 172.21.15.43:0/3279484790 conn(0x7fe78806c5a0 msgr2=0x7fe7880afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 shutdown_connections 2024-03-30T13:11:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.991+0000 7fe78cdfa640 1 -- 172.21.15.43:0/3279484790 wait complete. 2024-03-30T13:11:22.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfed69e640 1 Processor -- start 2024-03-30T13:11:22.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfed69e640 1 -- start start 2024-03-30T13:11:22.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfed69e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdfe0052ca0 0x7fdfe00355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfed69e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfe0055640 con 0x7fdfe0055260 2024-03-30T13:11:22.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfed69e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfe0037450 con 0x7fdfe0036fa0 2024-03-30T13:11:22.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfed69e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe003b670 con 0x7fdfe0052ca0 2024-03-30T13:11:22.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfe7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdfe0055260 0x7fdfe0052590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59632/0 (socket says 172.21.15.43:59632) 2024-03-30T13:11:22.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.995+0000 7fdfe7fff640 1 -- 172.21.15.43:0/3089169678 learned_addr learned my addr 172.21.15.43:0/3089169678 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:22.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385087536 0 0) 0x7fdfe0037450 con 0x7fdfe0036fa0 2024-03-30T13:11:22.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdfd8005740 con 0x7fdfe0036fa0 2024-03-30T13:11:22.009 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:22.009 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 714699621 0 0) 0x7fdfd8005740 con 0x7fdfe0036fa0 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 >> v2:172.21.15.43:3300/0 conn(0x7fdfe0052ca0 msgr2=0x7fdfe00355b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 --2- 172.21.15.43:0/3089169678 >> v2:172.21.15.43:3300/0 conn(0x7fdfe0052ca0 0x7fdfe00355b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 >> v1:172.21.15.43:6789/0 conn(0x7fdfe0055260 legacy=0x7fdfe0052590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe003aa60 con 0x7fdfe0036fa0 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fdfe8050150 con 0x7fdfe0036fa0 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 >> v1:172.21.15.113:6789/0 conn(0x7fdfe0036fa0 legacy=0x7fdfe00df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8005bd0 0x7fdfd8005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd8006a20 0x7fdfd8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80073e0 0x7fdfd800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfece9d640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80073e0 0x7fdfd800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd800e350 con 0x7fdfd80073e0 2024-03-30T13:11:22.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd800e500 con 0x7fdfd8006a20 2024-03-30T13:11:22.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd800e700 con 0x7fdfd8005bd0 2024-03-30T13:11:22.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfece9d640 1 -- 172.21.15.43:0/3089169678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8005bd0 msgr2=0x7fdfd8005ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:22.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfece9d640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfd8005bd0 0x7fdfd8005ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfece9d640 1 -- 172.21.15.43:0/3089169678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd8006a20 msgr2=0x7fdfd8006ea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:22.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfece9d640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfd8006a20 0x7fdfd8006ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:21.999+0000 7fdfece9d640 1 -- 172.21.15.43:0/3089169678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd8005740 con 0x7fdfd80073e0 2024-03-30T13:11:22.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.003+0000 7fdfece9d640 1 -- 172.21.15.43:0/3089169678 reap_dead start 2024-03-30T13:11:22.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.003+0000 7fdfece9d640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80073e0 0x7fdfd800dbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8069d80 tx=0x7fdfe809c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffebf639b95ee91 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.003+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8068880 con 0x7fdfd80073e0 2024-03-30T13:11:22.013 INFO:teuthology.orchestra.run.smithi043.stdout:64424509451 2024-03-30T13:11:22.014 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T13:11:22.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.003+0000 7fdfe6ffd640 1 -- 172.21.15.43:0/3089169678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfe8126540 con 0x7fdfd80073e0 2024-03-30T13:11:22.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 -- 172.21.15.43:0/3089169678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80073e0 msgr2=0x7fdfd800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80073e0 0x7fdfd800dbe0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8069d80 tx=0x7fdfe809c920 comp rx=0 tx=0).stop 2024-03-30T13:11:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 -- 172.21.15.43:0/3089169678 shutdown_connections 2024-03-30T13:11:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 --2- 172.21.15.43:0/3089169678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfd80073e0 0x7fdfd800dbe0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 -- 172.21.15.43:0/3089169678 >> 172.21.15.43:0/3089169678 conn(0x7fdfe001ef50 msgr2=0x7fdfe0055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 -- 172.21.15.43:0/3089169678 shutdown_connections 2024-03-30T13:11:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 -- 172.21.15.43:0/3089169678 wait complete. 2024-03-30T13:11:22.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 Processor -- start 2024-03-30T13:11:22.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.007+0000 7fdfed69e640 1 -- start start 2024-03-30T13:11:22.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0008510 0x7fdfe00dab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe00088f0 0x7fdfe004bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe004c1e0 0x7fdfe00d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe003b300 con 0x7fdfe0008510 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe003b020 con 0x7fdfe004c1e0 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe0052590 con 0x7fdfe00088f0 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0008510 0x7fdfe00dab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfece9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe004c1e0 0x7fdfe00d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0008510 0x7fdfe00dab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34846/0 (socket says 172.21.15.43:34846) 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 -- 172.21.15.43:0/3373868173 learned_addr learned my addr 172.21.15.43:0/3373868173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe77fe640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe00088f0 0x7fdfe004bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 -- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe00088f0 msgr2=0x7fdfe004bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe00088f0 0x7fdfe004bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 -- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe004c1e0 msgr2=0x7fdfe00d8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe004c1e0 0x7fdfe00d8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 -- 172.21.15.43:0/3373868173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe00490a0 con 0x7fdfe0008510 2024-03-30T13:11:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe7fff640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0008510 0x7fdfe00dab60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c4c0 tx=0x7fdfd400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cdec6640 1 Processor -- start 2024-03-30T13:11:22.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4017070 con 0x7fdfe0008510 2024-03-30T13:11:22.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd4005cf0 con 0x7fdfe0008510 2024-03-30T13:11:22.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4007270 con 0x7fdfe0008510 2024-03-30T13:11:22.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe00d9ae0 con 0x7fdfe0008510 2024-03-30T13:11:22.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe00da0a0 con 0x7fdfe0008510 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cdec6640 1 -- start start 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cdec6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140690 0x7f58c80cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cdec6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58c80cf520 0x7f58c815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cdec6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58c80cdf40 con 0x7f58c8140a70 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cdec6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f58c80ce220 con 0x7f58c80cf520 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cdec6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f58c80ce500 con 0x7f58c8140690 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.011+0000 7f58cd6c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58c80cf520 0x7f58c815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7f58cd6c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58c80cf520 0x7f58c815dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34858/0 (socket says 172.21.15.43:34858) 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7f58cd6c5640 1 -- 172.21.15.43:0/3383170170 learned_addr learned my addr 172.21.15.43:0/3383170170 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7f58ccec4640 1 --2- 172.21.15.43:0/3383170170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140690 0x7f58c80cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7f58cd6c5640 1 -- 172.21.15.43:0/3383170170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140690 msgr2=0x7f58c80cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:22.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7f58cd6c5640 1 --2- 172.21.15.43:0/3383170170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140690 0x7f58c80cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7f58cd6c5640 1 -- 172.21.15.43:0/3383170170 >> v1:172.21.15.113:6789/0 conn(0x7f58c8140a70 legacy=0x7f58c8141060 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7f58cd6c5640 1 -- 172.21.15.43:0/3383170170 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c80ce7e0 con 0x7f58c80cf520 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdfe00057f0 con 0x7fdfe0008510 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdfd401a070 con 0x7fdfe0008510 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfe4ff9640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfcc067fc0 0x7fdfcc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdfd4085b10 con 0x7fdfe0008510 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfe4ff9640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7fdfcc0716c0 0x7fdfcc073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 --> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdfcc0741d0 con 0x7fdfcc0716c0 2024-03-30T13:11:22.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfece9d640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7fdfcc0716c0 0x7fdfcc073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.029 INFO:teuthology.orchestra.run.smithi043.stdout:55834574859 2024-03-30T13:11:22.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T13:11:22.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfece9d640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7fdfcc0716c0 0x7fdfcc073b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfd4051b30 con 0x7fdfe0008510 2024-03-30T13:11:22.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.015+0000 7fdfe77fe640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfcc067fc0 0x7fdfcc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7f58cd6c5640 1 --2- 172.21.15.43:0/3383170170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c80cf520 0x7f58c815dee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d370 tx=0x7f58c001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7383f5d1eb1a099 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7fdfe77fe640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfcc067fc0 0x7fdfcc06a480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfe004b790 tx=0x7fdfdc006f10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.031 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:22.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T13:11:22.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== osd.4 v2:172.21.15.113:6824/840052469 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdfcc0741d0 con 0x7fdfcc0716c0 2024-03-30T13:11:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7f58c77fe640 1 -- 172.21.15.43:0/3383170170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c00ad070 con 0x7f58c80cf520 2024-03-30T13:11:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7f58c77fe640 1 -- 172.21.15.43:0/3383170170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c8143e90 con 0x7f58c80cf520 2024-03-30T13:11:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7f58c77fe640 1 -- 172.21.15.43:0/3383170170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c000be60 con 0x7f58c80cf520 2024-03-30T13:11:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7f58cdec6640 1 -- 172.21.15.43:0/3383170170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c80cf520 msgr2=0x7f58c815dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.019+0000 7f58cdec6640 1 --2- 172.21.15.43:0/3383170170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c80cf520 0x7f58c815dee0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f58c000d370 tx=0x7f58c001ecd0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 -- 172.21.15.43:0/3383170170 shutdown_connections 2024-03-30T13:11:22.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 --2- 172.21.15.43:0/3383170170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c80cf520 0x7f58c815dee0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 --2- 172.21.15.43:0/3383170170 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140690 0x7f58c80cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 -- 172.21.15.43:0/3383170170 >> 172.21.15.43:0/3383170170 conn(0x7f58c80afb60 msgr2=0x7f58c80aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 -- 172.21.15.43:0/3383170170 shutdown_connections 2024-03-30T13:11:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 -- 172.21.15.43:0/3383170170 wait complete. 2024-03-30T13:11:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 Processor -- start 2024-03-30T13:11:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7f58cdec6640 1 -- start start 2024-03-30T13:11:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7ffbbd593640 1 -- 172.21.15.43:0/89287919 --> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb98005ca0 con 0x7ffb98001660 2024-03-30T13:11:22.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.023+0000 7ffb9f7fe640 1 -- 172.21.15.43:0/89287919 <== osd.7 v2:172.21.15.113:6808/3435220755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffb98005ca0 con 0x7ffb98001660 2024-03-30T13:11:22.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c8140690 0x7f58c8204180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140a70 0x7f58c8207e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c82083c0 0x7f58c80b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f58c8141770 con 0x7f58c8140690 2024-03-30T13:11:22.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f58c81418f0 con 0x7f58c82083c0 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f58c80b5bf0 con 0x7f58c8140a70 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c8140690 0x7f58c8204180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c8140690 0x7f58c8204180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34864/0 (socket says 172.21.15.43:34864) 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 -- 172.21.15.43:0/2788792615 learned_addr learned my addr 172.21.15.43:0/2788792615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58c7fff640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140a70 0x7f58c8207e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cd6c5640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c82083c0 0x7f58c80b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 -- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140a70 msgr2=0x7f58c8207e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f58c8140a70 0x7f58c8207e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 -- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c82083c0 msgr2=0x7f58c80b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f58c82083c0 0x7f58c80b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 -- 172.21.15.43:0/2788792615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c80b4f70 con 0x7f58c8140690 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58ccec4640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c8140690 0x7f58c8204180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00d830 tx=0x7f58bc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00b660 con 0x7f58c8140690 2024-03-30T13:11:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58bc019070 con 0x7f58c8140690 2024-03-30T13:11:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc00ba00 con 0x7f58c8140690 2024-03-30T13:11:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c80b5250 con 0x7f58c8140690 2024-03-30T13:11:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c80b5810 con 0x7f58c8140690 2024-03-30T13:11:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.027+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f58c8143e90 con 0x7f58c8140690 2024-03-30T13:11:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f58bc00bba0 con 0x7f58c8140690 2024-03-30T13:11:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c57fa640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f58b4067ff0 0x7f58b406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f58bc086570 con 0x7f58c8140690 2024-03-30T13:11:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c7fff640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f58b4067ff0 0x7f58b406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c57fa640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f58b40716f0 0x7f58b4073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 --> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58b4074200 con 0x7f58b40716f0 2024-03-30T13:11:22.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58bc08b060 con 0x7f58c8140690 2024-03-30T13:11:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58cd6c5640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f58b40716f0 0x7f58b4073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58c7fff640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f58b4067ff0 0x7f58b406a4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58c81ee1f0 tx=0x7f58b80090e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.031+0000 7f58cd6c5640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f58b40716f0 0x7f58b4073b30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.035+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== osd.6 v2:172.21.15.113:6800/594900508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58b4074200 con 0x7f58b40716f0 2024-03-30T13:11:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.035+0000 7ffb9d7fa640 1 -- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7ffb98001660 msgr2=0x7ffb98003b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.035+0000 7ffb9d7fa640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7ffb98001660 0x7ffb98003b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.039+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 --> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdfe003bb10 con 0x7fdfcc0716c0 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 -- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffba40680c0 msgr2=0x7ffba406a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffba40680c0 0x7ffba406a580 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00b0000 tx=0x7ffbb0004ac0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 -- 172.21.15.43:0/89287919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb8072c70 msgr2=0x7ffbb813f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 --2- 172.21.15.43:0/89287919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb8072c70 0x7ffbb813f2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00c520 tx=0x7ffbac00c9e0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffbb77fe640 1 -- 172.21.15.43:0/89287919 reap_dead start 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 -- 172.21.15.43:0/89287919 shutdown_connections 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 -- 172.21.15.43:0/89287919 >> 172.21.15.43:0/89287919 conn(0x7ffbb806b660 msgr2=0x7ffbb80b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 -- 172.21.15.43:0/89287919 shutdown_connections 2024-03-30T13:11:22.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7ffb9d7fa640 1 -- 172.21.15.43:0/89287919 wait complete. 2024-03-30T13:11:22.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfe4ff9640 1 -- 172.21.15.43:0/3373868173 <== osd.4 v2:172.21.15.113:6824/840052469 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdfe003bb10 con 0x7fdfcc0716c0 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7fdfcc0716c0 msgr2=0x7fdfcc073b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7fdfcc0716c0 0x7fdfcc073b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfcc067fc0 msgr2=0x7fdfcc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfcc067fc0 0x7fdfcc06a480 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfe004b790 tx=0x7fdfdc006f10 comp rx=0 tx=0).stop 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0008510 msgr2=0x7fdfe00dab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 --2- 172.21.15.43:0/3373868173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe0008510 0x7fdfe00dab60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c4c0 tx=0x7fdfd400c980 comp rx=0 tx=0).stop 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfece9d640 1 -- 172.21.15.43:0/3373868173 reap_dead start 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 shutdown_connections 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 >> 172.21.15.43:0/3373868173 conn(0x7fdfe001ef50 msgr2=0x7fdfe0053830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 shutdown_connections 2024-03-30T13:11:22.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.043+0000 7fdfed69e640 1 -- 172.21.15.43:0/3373868173 wait complete. 2024-03-30T13:11:22.082 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:22.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T13:11:22.088 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:22.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T13:11:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 --> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f58c8068670 con 0x7f58b40716f0 2024-03-30T13:11:22.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58c57fa640 1 -- 172.21.15.43:0/2788792615 <== osd.6 v2:172.21.15.113:6800/594900508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f58c8068670 con 0x7f58b40716f0 2024-03-30T13:11:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f58b40716f0 msgr2=0x7f58b4073b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f58b40716f0 0x7f58b4073b30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f58b4067ff0 msgr2=0x7f58b406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f58b4067ff0 0x7f58b406a4b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58c81ee1f0 tx=0x7f58b80090e0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c8140690 msgr2=0x7f58c8204180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 --2- 172.21.15.43:0/2788792615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58c8140690 0x7f58c8204180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f58bc00d830 tx=0x7f58bc00dcf0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cd6c5640 1 -- 172.21.15.43:0/2788792615 reap_dead start 2024-03-30T13:11:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 shutdown_connections 2024-03-30T13:11:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 >> 172.21.15.43:0/2788792615 conn(0x7f58c80afb60 msgr2=0x7f58c8142490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 shutdown_connections 2024-03-30T13:11:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.087+0000 7f58cdec6640 1 -- 172.21.15.43:0/2788792615 wait complete. 2024-03-30T13:11:22.127 INFO:teuthology.orchestra.run.smithi043.stdout:64424509449 2024-03-30T13:11:22.127 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T13:11:22.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.183+0000 7fc47ee73640 1 Processor -- start 2024-03-30T13:11:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.183+0000 7fc47ee73640 1 -- start start 2024-03-30T13:11:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47ee73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780cef80 0x7fc4780b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47ee73640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4780cf360 0x7fc4780cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47ee73640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc4780b3840 0x7fc4780b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47ee73640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc4780b4230 con 0x7fc4780cf360 2024-03-30T13:11:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47ee73640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc4780b4500 con 0x7fc4780b3840 2024-03-30T13:11:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47ee73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc4780b47e0 con 0x7fc4780cef80 2024-03-30T13:11:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4780cf360 0x7fc4780cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487b2d3640 1 Processor -- start 2024-03-30T13:11:22.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487b2d3640 1 -- start start 2024-03-30T13:11:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4780cf360 0x7fc4780cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34868/0 (socket says 172.21.15.43:34868) 2024-03-30T13:11:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 -- 172.21.15.43:0/2664514551 learned_addr learned my addr 172.21.15.43:0/2664514551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47cc0f640 1 --2- 172.21.15.43:0/2664514551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780cef80 0x7fc4780b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47d410640 1 --2- 172.21.15.43:0/2664514551 >> v2:172.21.15.113:3300/0 conn(0x7fc4780b3840 0x7fc4780b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487b2d3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4874072bc0 0x7f487415d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487b2d3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48740ce750 con 0x7f4874156830 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487b2d3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4874156c10 con 0x7f4874142000 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487b2d3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4874072fd0 con 0x7f4874072bc0 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487906f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4874156830 0x7f48741418f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59644/0 (socket says 172.21.15.43:59644) 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f487906f640 1 -- 172.21.15.43:0/1566228025 learned_addr learned my addr 172.21.15.43:0/1566228025 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f4879870640 1 --2- 172.21.15.43:0/1566228025 >> v2:172.21.15.43:3300/0 conn(0x7f4874072bc0 0x7f487415d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f4879870640 1 -- 172.21.15.43:0/1566228025 >> v1:172.21.15.113:6789/0 conn(0x7f4874142000 legacy=0x7f48740714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f4879870640 1 -- 172.21.15.43:0/1566228025 >> v1:172.21.15.43:6789/0 conn(0x7f4874156830 legacy=0x7f48741418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7f4879870640 1 -- 172.21.15.43:0/1566228025 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48740ce9f0 con 0x7f4874072bc0 2024-03-30T13:11:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 -- 172.21.15.43:0/2664514551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780cef80 msgr2=0x7fc4780b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 --2- 172.21.15.43:0/2664514551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780cef80 0x7fc4780b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 -- 172.21.15.43:0/2664514551 >> v2:172.21.15.113:3300/0 conn(0x7fc4780b3840 msgr2=0x7fc4780b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 --2- 172.21.15.43:0/2664514551 >> v2:172.21.15.113:3300/0 conn(0x7fc4780b3840 0x7fc4780b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc477fff640 1 -- 172.21.15.43:0/2664514551 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4780b4ac0 con 0x7fc4780cf360 2024-03-30T13:11:22.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.187+0000 7fc47d410640 1 --2- 172.21.15.43:0/2664514551 >> v2:172.21.15.113:3300/0 conn(0x7fc4780b3840 0x7fc4780b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc477fff640 1 --2- 172.21.15.43:0/2664514551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780cf360 0x7fc4780cf7c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4780ce8e0 tx=0x7fc46801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d4a71e74ffb412 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7f4879870640 1 --2- 172.21.15.43:0/1566228025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 0x7f487415d280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48640097c0 tx=0x7f486401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8352f594e9a84cac server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc4777fe640 1 -- 172.21.15.43:0/2664514551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4680a55d0 con 0x7fc4780cf360 2024-03-30T13:11:22.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc4777fe640 1 -- 172.21.15.43:0/2664514551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4680a5770 con 0x7fc4780cf360 2024-03-30T13:11:22.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc4777fe640 1 -- 172.21.15.43:0/2664514551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4680a5b40 con 0x7fc4780cf360 2024-03-30T13:11:22.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 -- 172.21.15.43:0/2664514551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780cf360 msgr2=0x7fc4780cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 --2- 172.21.15.43:0/2664514551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780cf360 0x7fc4780cf7c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4780ce8e0 tx=0x7fc46801c9f0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 -- 172.21.15.43:0/2664514551 shutdown_connections 2024-03-30T13:11:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 --2- 172.21.15.43:0/2664514551 >> v2:172.21.15.113:3300/0 conn(0x7fc4780b3840 0x7fc4780b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 --2- 172.21.15.43:0/2664514551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780cf360 0x7fc4780cf7c0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc4780ce8e0 tx=0x7fc46801c9f0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 --2- 172.21.15.43:0/2664514551 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780cef80 0x7fc4780b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 -- 172.21.15.43:0/2664514551 >> 172.21.15.43:0/2664514551 conn(0x7fc4780d4310 msgr2=0x7fc4780d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7f486bfff640 1 -- 172.21.15.43:0/1566228025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640ab650 con 0x7f4874072bc0 2024-03-30T13:11:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7f486bfff640 1 -- 172.21.15.43:0/1566228025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48640b0070 con 0x7f4874072bc0 2024-03-30T13:11:22.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7f486bfff640 1 -- 172.21.15.43:0/1566228025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48640aba10 con 0x7f4874072bc0 2024-03-30T13:11:22.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 -- 172.21.15.43:0/2664514551 shutdown_connections 2024-03-30T13:11:22.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7f487b2d3640 1 -- 172.21.15.43:0/1566228025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 msgr2=0x7f487415d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7f487b2d3640 1 --2- 172.21.15.43:0/1566228025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 0x7f487415d280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48640097c0 tx=0x7f486401c430 comp rx=0 tx=0).stop 2024-03-30T13:11:22.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7f487b2d3640 1 -- 172.21.15.43:0/1566228025 shutdown_connections 2024-03-30T13:11:22.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7f487b2d3640 1 --2- 172.21.15.43:0/1566228025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 0x7f487415d280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7f487b2d3640 1 -- 172.21.15.43:0/1566228025 >> 172.21.15.43:0/1566228025 conn(0x7f487406b660 msgr2=0x7f487406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.191+0000 7fc47ee73640 1 -- 172.21.15.43:0/2664514551 wait complete. 2024-03-30T13:11:22.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 Processor -- start 2024-03-30T13:11:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 -- start start 2024-03-30T13:11:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780b3840 0x7fc4781616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4780cef80 0x7fc478152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780c6f90 0x7fc478152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4780b82c0 con 0x7fc4780b3840 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4780b8140 con 0x7fc4780cef80 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc4780b8440 con 0x7fc4780c6f90 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780b3840 0x7fc4781616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780b3840 0x7fc4781616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34882/0 (socket says 172.21.15.43:34882) 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 -- 172.21.15.43:0/575691749 learned_addr learned my addr 172.21.15.43:0/575691749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47d410640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780c6f90 0x7fc478152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc477fff640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4780cef80 0x7fc478152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 -- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780c6f90 msgr2=0x7fc478152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780c6f90 0x7fc478152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 -- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4780cef80 msgr2=0x7fc478152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4780cef80 0x7fc478152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 -- 172.21.15.43:0/575691749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4781542b0 con 0x7fc4780b3840 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47cc0f640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780b3840 0x7fc4781616b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d830 tx=0x7fc47000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc4757fa640 1 -- 172.21.15.43:0/575691749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47000b6c0 con 0x7fc4780b3840 2024-03-30T13:11:22.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc4757fa640 1 -- 172.21.15.43:0/575691749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc470019070 con 0x7fc4780b3840 2024-03-30T13:11:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc4757fa640 1 -- 172.21.15.43:0/575691749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47000ba60 con 0x7fc4780b3840 2024-03-30T13:11:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.195+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc478161bf0 con 0x7fc4780b3840 2024-03-30T13:11:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47815ca50 con 0x7fc4780b3840 2024-03-30T13:11:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 -- 172.21.15.43:0/1566228025 shutdown_connections 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 -- 172.21.15.43:0/1566228025 wait complete. 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 Processor -- start 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 -- start start 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 0x7f487413f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4874142000 0x7f4874143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4874156830 0x7f487406eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f48740cd010 con 0x7f4874072bc0 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f48740ccd20 con 0x7f4874142000 2024-03-30T13:11:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487b2d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f48740cd190 con 0x7f4874156830 2024-03-30T13:11:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368f59e640 1 Processor -- start 2024-03-30T13:11:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368f59e640 1 -- start start 2024-03-30T13:11:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4874142000 0x7f4874143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4874142000 0x7f4874143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:46844/0 (socket says 172.21.15.43:46844) 2024-03-30T13:11:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 -- 172.21.15.43:0/4090875476 learned_addr learned my addr 172.21.15.43:0/4090875476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7fc4757fa640 1 -- 172.21.15.43:0/575691749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc47000bc00 con 0x7fc4780b3840 2024-03-30T13:11:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368f59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3688035470 0x7f36880ebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368f59e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36880347b0 con 0x7f36880e1220 2024-03-30T13:11:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368f59e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3688034a90 con 0x7f36880e1600 2024-03-30T13:11:22.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368f59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3688032060 con 0x7f3688035470 2024-03-30T13:11:22.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 -- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4874156830 msgr2=0x7f487406eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4874156830 0x7f487406eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 -- 172.21.15.43:0/4090875476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 msgr2=0x7f487413f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 0x7f487413f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f487886e640 1 -- 172.21.15.43:0/4090875476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4874204860 con 0x7f4874142000 2024-03-30T13:11:22.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc478068890 con 0x7fc4780b3840 2024-03-30T13:11:22.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7fc4757fa640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc464067fc0 0x7fc46406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f487886e640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4874142000 0x7f4874143760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c780 tx=0x7f486c00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7fc4757fa640 1 -- 172.21.15.43:0/575691749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc470086bd0 con 0x7fc4780b3840 2024-03-30T13:11:22.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7fc477fff640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc464067fc0 0x7fc46406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f36880e1220 0x7f3688036430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59654/0 (socket says 172.21.15.43:59654) 2024-03-30T13:11:22.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.199+0000 7f368e59c640 1 -- 172.21.15.43:0/2031128796 learned_addr learned my addr 172.21.15.43:0/2031128796 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:22.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f368ed9d640 1 --2- 172.21.15.43:0/2031128796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3688035470 0x7f36880ebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f4869ffb640 1 -- 172.21.15.43:0/4090875476 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c016030 con 0x7f4874142000 2024-03-30T13:11:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f368ed9d640 1 -- 172.21.15.43:0/2031128796 >> v1:172.21.15.113:6789/0 conn(0x7f36880e1600 legacy=0x7f3688034d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f368ed9d640 1 -- 172.21.15.43:0/2031128796 >> v1:172.21.15.43:6789/0 conn(0x7f36880e1220 legacy=0x7f3688036430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f368ed9d640 1 -- 172.21.15.43:0/2031128796 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3688032340 con 0x7f3688035470 2024-03-30T13:11:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7fc477fff640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc464067fc0 0x7fc46406a480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc4680ae000 tx=0x7fc4680a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4874204b40 con 0x7f4874142000 2024-03-30T13:11:22.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4874205100 con 0x7f4874142000 2024-03-30T13:11:22.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f4869ffb640 1 -- 172.21.15.43:0/4090875476 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c006b60 con 0x7f4874142000 2024-03-30T13:11:22.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f4869ffb640 1 -- 172.21.15.43:0/4090875476 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c00a850 con 0x7f4874142000 2024-03-30T13:11:22.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48540052a0 con 0x7f4874142000 2024-03-30T13:11:22.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f368ed9d640 1 --2- 172.21.15.43:0/2031128796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3688035470 0x7f36880ebf50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f368000f960 tx=0x7f3680020680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de2e5214ed67ed76 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f4869ffb640 1 -- 172.21.15.43:0/4090875476 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f486c019070 con 0x7f4874142000 2024-03-30T13:11:22.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.203+0000 7f4869ffb640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4858067fc0 0x7f485806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f4869ffb640 1 -- 172.21.15.43:0/4090875476 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f486c086ed0 con 0x7f4874142000 2024-03-30T13:11:22.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f368d59a640 1 -- 172.21.15.43:0/2031128796 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36800af410 con 0x7f3688035470 2024-03-30T13:11:22.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f368d59a640 1 -- 172.21.15.43:0/2031128796 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36800b4070 con 0x7f3688035470 2024-03-30T13:11:22.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f368d59a640 1 -- 172.21.15.43:0/2031128796 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36800af7d0 con 0x7f3688035470 2024-03-30T13:11:22.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f487906f640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4858067fc0 0x7f485806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f487906f640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4858067fc0 0x7f485806a480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4870000f70 tx=0x7f4870007a90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f368f59e640 1 -- 172.21.15.43:0/2031128796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3688035470 msgr2=0x7f36880ebf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.207+0000 7f368f59e640 1 --2- 172.21.15.43:0/2031128796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3688035470 0x7f36880ebf50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f368000f960 tx=0x7f3680020680 comp rx=0 tx=0).stop 2024-03-30T13:11:22.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.211+0000 7fc4757fa640 1 -- 172.21.15.43:0/575691749 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc470057040 con 0x7fc4780b3840 2024-03-30T13:11:22.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- 172.21.15.43:0/2031128796 shutdown_connections 2024-03-30T13:11:22.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 --2- 172.21.15.43:0/2031128796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3688035470 0x7f36880ebf50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- 172.21.15.43:0/2031128796 >> 172.21.15.43:0/2031128796 conn(0x7f368801e6d0 msgr2=0x7f3688055400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- 172.21.15.43:0/2031128796 shutdown_connections 2024-03-30T13:11:22.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- 172.21.15.43:0/2031128796 wait complete. 2024-03-30T13:11:22.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 Processor -- start 2024-03-30T13:11:22.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- start start 2024-03-30T13:11:22.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3688035470 0x7f36880e2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36880e1220 0x7f3688037150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36880e1600 0x7f368804c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36880ed030 con 0x7f3688035470 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36880eceb0 con 0x7f36880e1600 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.215+0000 7f368f59e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f36880ed1b0 con 0x7f36880e1220 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3688035470 0x7f36880e2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3688035470 0x7f36880e2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34888/0 (socket says 172.21.15.43:34888) 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 -- 172.21.15.43:0/1739417254 learned_addr learned my addr 172.21.15.43:0/1739417254 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368ed9d640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36880e1600 0x7f368804c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368dd9b640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36880e1220 0x7f3688037150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 -- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36880e1220 msgr2=0x7f3688037150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36880e1220 0x7f3688037150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 -- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36880e1600 msgr2=0x7f368804c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f4869ffb640 1 -- 172.21.15.43:0/4090875476 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f486c057420 con 0x7f4874142000 2024-03-30T13:11:22.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36880e1600 0x7f368804c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 -- 172.21.15.43:0/1739417254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36880e1ea0 con 0x7f3688035470 2024-03-30T13:11:22.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.219+0000 7f368e59c640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3688035470 0x7f36880e2f40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f368400c480 tx=0x7f368400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.223+0000 7f367effd640 1 -- 172.21.15.43:0/1739417254 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684017070 con 0x7f3688035470 2024-03-30T13:11:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.223+0000 7f367effd640 1 -- 172.21.15.43:0/1739417254 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3684005cf0 con 0x7f3688035470 2024-03-30T13:11:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.223+0000 7f367effd640 1 -- 172.21.15.43:0/1739417254 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684007290 con 0x7f3688035470 2024-03-30T13:11:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.223+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36880e2180 con 0x7f3688035470 2024-03-30T13:11:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.223+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36880e2740 con 0x7f3688035470 2024-03-30T13:11:22.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36880057f0 con 0x7f3688035470 2024-03-30T13:11:22.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f4552400640 1 Processor -- start 2024-03-30T13:11:22.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f4552400640 1 -- start start 2024-03-30T13:11:22.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f4552400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a2c0 0x7f454c06f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f4552400640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f454c072f20 0x7f454c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f4552400640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f454c0b32f0 con 0x7f454c15a6a0 2024-03-30T13:11:22.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f4552400640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f454c06f9a0 con 0x7f454c072f20 2024-03-30T13:11:22.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f4552400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f454c0b3e70 con 0x7f454c15a2c0 2024-03-30T13:11:22.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f367effd640 1 -- 172.21.15.43:0/1739417254 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f368401a070 con 0x7f3688035470 2024-03-30T13:11:22.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.227+0000 7f454bfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a2c0 0x7f454c06f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f454bfff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a2c0 0x7f454c06f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37370/0 (socket says 172.21.15.43:37370) 2024-03-30T13:11:22.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f454bfff640 1 -- 172.21.15.43:0/1263076946 learned_addr learned my addr 172.21.15.43:0/1263076946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f455099d640 1 --2- 172.21.15.43:0/1263076946 >> v2:172.21.15.113:3300/0 conn(0x7f454c072f20 0x7f454c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f367effd640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3670068090 0x7f367006a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f455099d640 1 -- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a2c0 msgr2=0x7f454c06f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:22.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f455099d640 1 --2- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a2c0 0x7f454c06f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f455099d640 1 -- 172.21.15.43:0/1263076946 >> v1:172.21.15.113:6789/0 conn(0x7f454c15a6a0 legacy=0x7f454c072810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:11:22.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f455099d640 1 -- 172.21.15.43:0/1263076946 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c0b35d0 con 0x7f454c072f20 2024-03-30T13:11:22.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f454bfff640 1 --2- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a2c0 0x7f454c06f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:22.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f367effd640 1 -- 172.21.15.43:0/1739417254 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3684085b10 con 0x7f3688035470 2024-03-30T13:11:22.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.231+0000 7f368dd9b640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3670068090 0x7f367006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.235+0000 7f455099d640 1 --2- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c072f20 0x7f454c06d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f453c0097c0 tx=0x7f453c01c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8e1da0d23e10d4d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.235+0000 7f454affd640 1 -- 172.21.15.43:0/1263076946 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c0b2070 con 0x7f454c072f20 2024-03-30T13:11:22.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.235+0000 7f368dd9b640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3670068090 0x7f367006a550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3690051240 tx=0x7f36900693f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.235+0000 7f454affd640 1 -- 172.21.15.43:0/1263076946 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f453c0a9070 con 0x7f454c072f20 2024-03-30T13:11:22.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.235+0000 7f4552400640 1 -- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c072f20 msgr2=0x7f454c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.235+0000 7f4552400640 1 --2- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c072f20 0x7f454c06d9e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f453c0097c0 tx=0x7f453c01c3a0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.235+0000 7f367effd640 1 -- 172.21.15.43:0/1739417254 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f36840559a0 con 0x7f3688035470 2024-03-30T13:11:22.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- 172.21.15.43:0/1263076946 shutdown_connections 2024-03-30T13:11:22.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 --2- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c072f20 0x7f454c06d9e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 --2- 172.21.15.43:0/1263076946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a2c0 0x7f454c06f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- 172.21.15.43:0/1263076946 >> 172.21.15.43:0/1263076946 conn(0x7f454c06c570 msgr2=0x7f454c06c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- 172.21.15.43:0/1263076946 shutdown_connections 2024-03-30T13:11:22.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- 172.21.15.43:0/1263076946 wait complete. 2024-03-30T13:11:22.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 Processor -- start 2024-03-30T13:11:22.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- start start 2024-03-30T13:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f454c072f20 0x7f454c137610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c15a2c0 0x7f454c137e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a6a0 0x7f454c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f454c0b2e80 con 0x7f454c072f20 2024-03-30T13:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f454c0b3000 con 0x7f454c15a2c0 2024-03-30T13:11:22.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f4552400640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f454c167250 con 0x7f454c15a6a0 2024-03-30T13:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f454bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f454c072f20 0x7f454c137610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f454b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c15a2c0 0x7f454c137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f454bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f454c072f20 0x7f454c137610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34892/0 (socket says 172.21.15.43:34892) 2024-03-30T13:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.239+0000 7f454bfff640 1 -- 172.21.15.43:0/2493172854 learned_addr learned my addr 172.21.15.43:0/2493172854 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.243+0000 7f455099d640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a6a0 0x7f454c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.243+0000 7f454b7fe640 1 -- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a6a0 msgr2=0x7f454c155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.243+0000 7f454b7fe640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a6a0 0x7f454c155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.243+0000 7f454b7fe640 1 -- 172.21.15.43:0/2493172854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f454c072f20 msgr2=0x7f454c137610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.243+0000 7f454b7fe640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f454c072f20 0x7f454c137610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.243+0000 7f454b7fe640 1 -- 172.21.15.43:0/2493172854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c137b50 con 0x7f454c15a2c0 2024-03-30T13:11:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.243+0000 7f454b7fe640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c15a2c0 0x7f454c137e70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f454000b3f0 tx=0x7f454000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4548ff9640 1 -- 172.21.15.43:0/2493172854 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454001d070 con 0x7f454c15a2c0 2024-03-30T13:11:22.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4552400640 1 -- 172.21.15.43:0/2493172854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454c153680 con 0x7f454c15a2c0 2024-03-30T13:11:22.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4552400640 1 -- 172.21.15.43:0/2493172854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454c153c40 con 0x7f454c15a2c0 2024-03-30T13:11:22.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4548ff9640 1 -- 172.21.15.43:0/2493172854 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4540010ab0 con 0x7f454c15a2c0 2024-03-30T13:11:22.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4548ff9640 1 -- 172.21.15.43:0/2493172854 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540018440 con 0x7f454c15a2c0 2024-03-30T13:11:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4548ff9640 1 -- 172.21.15.43:0/2493172854 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4540020070 con 0x7f454c15a2c0 2024-03-30T13:11:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4548ff9640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4530068010 0x7f453006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.247+0000 7f4548ff9640 1 -- 172.21.15.43:0/2493172854 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4540089ff0 con 0x7f454c15a2c0 2024-03-30T13:11:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.251+0000 7f454bfff640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4530068010 0x7f453006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.251+0000 7f454bfff640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4530068010 0x7f453006a4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4544008150 tx=0x7f454400f040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7f4552400640 1 -- 172.21.15.43:0/2493172854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45280052a0 con 0x7f454c15a2c0 2024-03-30T13:11:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa788cb640 1 Processor -- start 2024-03-30T13:11:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa788cb640 1 -- start start 2024-03-30T13:11:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa788cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa6c04f850 0x7faa6c006990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa788cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6c050140 con 0x7faa6c04f3e0 2024-03-30T13:11:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa788cb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa6c053800 con 0x7faa6c100e30 2024-03-30T13:11:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa788cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7faa6c037b30 con 0x7faa6c04f850 2024-03-30T13:11:22.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa7359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa6c04f850 0x7faa6c006990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa7359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faa6c04f850 0x7faa6c006990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34896/0 (socket says 172.21.15.43:34896) 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa7359e640 1 -- 172.21.15.43:0/2802202221 learned_addr learned my addr 172.21.15.43:0/2802202221 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.255+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261318019 0 0) 0x7faa6c050140 con 0x7faa6c04f3e0 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa5c003510 con 0x7faa6c04f3e0 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1238422922 0 0) 0x7faa5c003510 con 0x7faa6c04f3e0 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 >> v2:172.21.15.43:3300/0 conn(0x7faa6c04f850 msgr2=0x7faa6c006990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 --2- 172.21.15.43:0/2802202221 >> v2:172.21.15.43:3300/0 conn(0x7faa6c04f850 0x7faa6c006990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 >> v1:172.21.15.113:6789/0 conn(0x7faa6c100e30 legacy=0x7faa6c054230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c03a550 con 0x7faa6c04f3e0 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7faa64001680 con 0x7faa6c04f3e0 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 >> v1:172.21.15.43:6789/0 conn(0x7faa6c04f3e0 legacy=0x7faa6c006280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c003ce0 0x7faa5c004120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa5c004660 0x7faa5c011620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c004e40 0x7faa5c0072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c007b80 con 0x7faa5c004e40 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c007d30 con 0x7faa5c003ce0 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7faa5c007f30 con 0x7faa5c004660 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.259+0000 7faa7359e640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c004e40 0x7faa5c0072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa7359e640 1 -- 172.21.15.43:0/2802202221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa5c004660 msgr2=0x7faa5c011620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa7359e640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa5c004660 0x7faa5c011620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa7359e640 1 -- 172.21.15.43:0/2802202221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c003ce0 msgr2=0x7faa5c004120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa7359e640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c003ce0 0x7faa5c004120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa7359e640 1 -- 172.21.15.43:0/2802202221 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c03a550 con 0x7faa5c004e40 2024-03-30T13:11:22.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa7359e640 1 -- 172.21.15.43:0/2802202221 reap_dead start 2024-03-30T13:11:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa7359e640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c004e40 0x7faa5c0072d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faa7407aa30 tx=0x7faa74086e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43eab3483b96f713 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7404fe70 con 0x7faa5c004e40 2024-03-30T13:11:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa7410fba0 con 0x7faa5c004e40 2024-03-30T13:11:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.263+0000 7faa71d9b640 1 -- 172.21.15.43:0/2802202221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74117440 con 0x7faa5c004e40 2024-03-30T13:11:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.267+0000 7faa788cb640 1 -- 172.21.15.43:0/2802202221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c004e40 msgr2=0x7faa5c0072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.267+0000 7faa788cb640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c004e40 0x7faa5c0072d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faa7407aa30 tx=0x7faa74086e90 comp rx=0 tx=0).stop 2024-03-30T13:11:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.267+0000 7f4548ff9640 1 -- 172.21.15.43:0/2493172854 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4540059ec0 con 0x7f454c15a2c0 2024-03-30T13:11:22.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.275+0000 7fdb57fb5640 1 Processor -- start 2024-03-30T13:11:22.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.275+0000 7fdb57fb5640 1 -- start start 2024-03-30T13:11:22.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb55d51640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdb5006f9e0 0x7fdb5006fdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59674/0 (socket says 172.21.15.43:59674) 2024-03-30T13:11:22.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb57fb5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdb5006ce80 0x7fdb50165c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb55d51640 1 -- 172.21.15.43:0/1252936102 learned_addr learned my addr 172.21.15.43:0/1252936102 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:22.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5006d430 con 0x7fdb5006f9e0 2024-03-30T13:11:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5006d710 con 0x7fdb500b5540 2024-03-30T13:11:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb56552640 1 --2- 172.21.15.43:0/1252936102 >> v2:172.21.15.113:3300/0 conn(0x7fdb5006ce80 0x7fdb50165c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdb50152550 con 0x7fdb5006ce80 2024-03-30T13:11:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb56552640 1 -- 172.21.15.43:0/1252936102 >> v1:172.21.15.113:6789/0 conn(0x7fdb500b5540 legacy=0x7fdb5006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb56552640 1 -- 172.21.15.43:0/1252936102 >> v1:172.21.15.43:6789/0 conn(0x7fdb5006f9e0 legacy=0x7fdb5006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb56552640 1 -- 172.21.15.43:0/1252936102 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5006d9f0 con 0x7fdb5006ce80 2024-03-30T13:11:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7faa788cb640 1 -- 172.21.15.43:0/2802202221 shutdown_connections 2024-03-30T13:11:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7faa788cb640 1 --2- 172.21.15.43:0/2802202221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa5c004e40 0x7faa5c0072d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7faa788cb640 1 -- 172.21.15.43:0/2802202221 >> 172.21.15.43:0/2802202221 conn(0x7faa6c009fc0 msgr2=0x7faa6c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7faa788cb640 1 -- 172.21.15.43:0/2802202221 shutdown_connections 2024-03-30T13:11:22.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7faa788cb640 1 -- 172.21.15.43:0/2802202221 wait complete. 2024-03-30T13:11:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.279+0000 7fdb56552640 1 --2- 172.21.15.43:0/1252936102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5006ce80 0x7fdb50165c00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009d90 tx=0x7fdb4801c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3220da61cd3ffd0c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 Processor -- start 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 -- start start 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c004e40 0x7faa6c04ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c196420 0x7faa6c04c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa6c0f5e20 0x7faa6c0f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c04fca0 con 0x7faa6c196420 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faa6c0533d0 con 0x7faa5c004e40 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa788cb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7faa6c053550 con 0x7faa6c0f5e20 2024-03-30T13:11:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c196420 0x7faa6c04c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c196420 0x7faa6c04c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34920/0 (socket says 172.21.15.43:34920) 2024-03-30T13:11:22.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 -- 172.21.15.43:0/3313850878 learned_addr learned my addr 172.21.15.43:0/3313850878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7359e640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa6c0f5e20 0x7faa6c0f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa72d9d640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c004e40 0x7faa6c04ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 -- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa6c0f5e20 msgr2=0x7faa6c0f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa6c0f5e20 0x7faa6c0f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 -- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c004e40 msgr2=0x7faa6c04ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c004e40 0x7faa6c04ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 -- 172.21.15.43:0/3313850878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa6c1968a0 con 0x7faa6c196420 2024-03-30T13:11:22.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.283+0000 7faa7259c640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c196420 0x7faa6c04c5b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c820 tx=0x7faa6400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.287+0000 7faa637fe640 1 -- 172.21.15.43:0/3313850878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640052d0 con 0x7faa6c196420 2024-03-30T13:11:22.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.287+0000 7faa637fe640 1 -- 172.21.15.43:0/3313850878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa64005470 con 0x7faa6c196420 2024-03-30T13:11:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.287+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa6c04caf0 con 0x7faa6c196420 2024-03-30T13:11:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.287+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa6c04d070 con 0x7faa6c196420 2024-03-30T13:11:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.287+0000 7faa637fe640 1 -- 172.21.15.43:0/3313850878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6400aa30 con 0x7faa6c196420 2024-03-30T13:11:22.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.287+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa500052a0 con 0x7faa6c196420 2024-03-30T13:11:22.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.295+0000 7fcba0977640 1 Processor -- start 2024-03-30T13:11:22.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb54d4f640 1 -- 172.21.15.43:0/1252936102 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480a6710 con 0x7fdb5006ce80 2024-03-30T13:11:22.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcba0977640 1 -- start start 2024-03-30T13:11:22.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7faa637fe640 1 -- 172.21.15.43:0/3313850878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7faa6400abd0 con 0x7faa6c196420 2024-03-30T13:11:22.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb54d4f640 1 -- 172.21.15.43:0/1252936102 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb480a68b0 con 0x7fdb5006ce80 2024-03-30T13:11:22.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7faa637fe640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa580680c0 0x7faa5806a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7faa637fe640 1 -- 172.21.15.43:0/3313850878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7faa64086ce0 con 0x7faa6c196420 2024-03-30T13:11:22.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5006ce80 msgr2=0x7fdb50165c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1252936102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5006ce80 0x7fdb50165c00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009d90 tx=0x7fdb4801c680 comp rx=0 tx=0).stop 2024-03-30T13:11:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 shutdown_connections 2024-03-30T13:11:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1252936102 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb5006ce80 0x7fdb50165c00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 >> 172.21.15.43:0/1252936102 conn(0x7fdb5006e410 msgr2=0x7fdb5006e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7faa72d9d640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa580680c0 0x7faa5806a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7faa637fe640 1 -- 172.21.15.43:0/3313850878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7faa64086600 con 0x7faa6c196420 2024-03-30T13:11:22.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcba0977640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb9c0cff60 0x7fcb9c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcba0977640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb9c06e460 0x7fcb9c0b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcba0977640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9c06ea80 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcba0977640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9c06ed60 con 0x7fcb9c06e460 2024-03-30T13:11:22.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcba0977640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9c06f040 con 0x7fcb9c0cff60 2024-03-30T13:11:22.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb9c06e460 0x7fcb9c0b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb9c06e460 0x7fcb9c0b3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34926/0 (socket says 172.21.15.43:34926) 2024-03-30T13:11:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9b7fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcb9c0cff60 0x7fcb9c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9bfff640 1 -- 172.21.15.43:0/2635053502 learned_addr learned my addr 172.21.15.43:0/2635053502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9affd640 1 --1- 172.21.15.43:0/2635053502 >> v1:172.21.15.113:6789/0 conn(0x7fcb9c06cbb0 0x7fcb9c06dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:44528/0 (socket says 172.21.15.43:44528) 2024-03-30T13:11:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9bfff640 1 -- 172.21.15.43:0/2635053502 >> v2:172.21.15.113:3300/0 conn(0x7fcb9c0cff60 msgr2=0x7fcb9c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9bfff640 1 --2- 172.21.15.43:0/2635053502 >> v2:172.21.15.113:3300/0 conn(0x7fcb9c0cff60 0x7fcb9c06c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9bfff640 1 -- 172.21.15.43:0/2635053502 >> v1:172.21.15.113:6789/0 conn(0x7fcb9c06cbb0 legacy=0x7fcb9c06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fcb9bfff640 1 -- 172.21.15.43:0/2635053502 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c06f320 con 0x7fcb9c06e460 2024-03-30T13:11:22.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 shutdown_connections 2024-03-30T13:11:22.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.299+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1252936102 wait complete. 2024-03-30T13:11:22.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 Processor -- start 2024-03-30T13:11:22.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fcb9bfff640 1 --2- 172.21.15.43:0/2635053502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 0x7fcb9c0b3af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c072990 tx=0x7fcb9401d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afafed3fcf4e514 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7faa72d9d640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa580680c0 0x7faa5806a580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa68009900 tx=0x7faa68009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 -- start start 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb5006ce80 0x7fdb50147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb5006f9e0 0x7fdb50159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb500b5540 0x7fdb5015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50168ae0 con 0x7fdb5006f9e0 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdb50168960 con 0x7fdb500b5540 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb57fb5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdb50168c60 con 0x7fdb5006ce80 2024-03-30T13:11:22.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb55d51640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb5006ce80 0x7fdb50147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb55550640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb5006f9e0 0x7fdb50159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb55550640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb5006f9e0 0x7fdb50159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34942/0 (socket says 172.21.15.43:34942) 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb55550640 1 -- 172.21.15.43:0/1042486918 learned_addr learned my addr 172.21.15.43:0/1042486918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.303+0000 7fdb56552640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb500b5540 0x7fdb5015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.307+0000 7fdb55550640 1 -- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb5006ce80 msgr2=0x7fdb50147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.307+0000 7fdb55550640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb5006ce80 0x7fdb50147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.307+0000 7fdb55550640 1 -- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb500b5540 msgr2=0x7fdb5015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.307+0000 7fdb55550640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb500b5540 0x7fdb5015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.307+0000 7fdb55550640 1 -- 172.21.15.43:0/1042486918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb500cf730 con 0x7fdb5006f9e0 2024-03-30T13:11:22.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.307+0000 7fdb55550640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb5006f9e0 0x7fdb50159e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c550 tx=0x7fdb4000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcb9a7fc640 1 -- 172.21.15.43:0/2635053502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400a2d0 con 0x7fcb9c06e460 2024-03-30T13:11:22.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcb9a7fc640 1 -- 172.21.15.43:0/2635053502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb940a6800 con 0x7fcb9c06e460 2024-03-30T13:11:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 -- 172.21.15.43:0/2635053502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 msgr2=0x7fcb9c0b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 --2- 172.21.15.43:0/2635053502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 0x7fcb9c0b3af0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c072990 tx=0x7fcb9401d980 comp rx=0 tx=0).stop 2024-03-30T13:11:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fdb467fc640 1 -- 172.21.15.43:0/1042486918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40017070 con 0x7fdb5006f9e0 2024-03-30T13:11:22.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fdb467fc640 1 -- 172.21.15.43:0/1042486918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb40003600 con 0x7fdb5006f9e0 2024-03-30T13:11:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb500cfa10 con 0x7fdb5006f9e0 2024-03-30T13:11:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb500cffd0 con 0x7fdb5006f9e0 2024-03-30T13:11:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fdb467fc640 1 -- 172.21.15.43:0/1042486918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb400039f0 con 0x7fdb5006f9e0 2024-03-30T13:11:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 -- 172.21.15.43:0/2635053502 shutdown_connections 2024-03-30T13:11:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 --2- 172.21.15.43:0/2635053502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 0x7fcb9c0b3af0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 --2- 172.21.15.43:0/2635053502 >> v2:172.21.15.113:3300/0 conn(0x7fcb9c0cff60 0x7fcb9c06c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 -- 172.21.15.43:0/2635053502 >> 172.21.15.43:0/2635053502 conn(0x7fcb9c071ca0 msgr2=0x7fcb9c0720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 -- 172.21.15.43:0/2635053502 shutdown_connections 2024-03-30T13:11:22.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.323+0000 7fcba0977640 1 -- 172.21.15.43:0/2635053502 wait complete. 2024-03-30T13:11:22.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 Processor -- start 2024-03-30T13:11:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 -- start start 2024-03-30T13:11:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c06cbb0 0x7fcb9c1e93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 0x7fcb9c204c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c0cff60 0x7fcb9c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c067fd0 con 0x7fcb9c06e460 2024-03-30T13:11:22.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c067e50 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcba0977640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcb9c068150 con 0x7fcb9c0cff60 2024-03-30T13:11:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c06cbb0 0x7fcb9c1e93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcb9b7fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c06cbb0 0x7fcb9c1e93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:46926/0 (socket says 172.21.15.43:46926) 2024-03-30T13:11:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcb9b7fe640 1 -- 172.21.15.43:0/3964745944 learned_addr learned my addr 172.21.15.43:0/3964745944 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcb9bfff640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c0cff60 0x7fcb9c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.327+0000 7fcb9affd640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 0x7fcb9c204c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb9b7fe640 1 -- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c0cff60 msgr2=0x7fcb9c151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb9b7fe640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c0cff60 0x7fcb9c151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb9b7fe640 1 -- 172.21.15.43:0/3964745944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 msgr2=0x7fcb9c204c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb9b7fe640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 0x7fcb9c204c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb9b7fe640 1 -- 172.21.15.43:0/3964745944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c205180 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb9b7fe640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c06cbb0 0x7fcb9c1e93b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000e8d0 tx=0x7fcb9000ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fdb467fc640 1 -- 172.21.15.43:0/1042486918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdb4001a070 con 0x7fdb5006f9e0 2024-03-30T13:11:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fdb467fc640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdb34068090 0x7fdb3406a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fdb467fc640 1 -- 172.21.15.43:0/1042486918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdb40085b90 con 0x7fdb5006f9e0 2024-03-30T13:11:22.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fdb55d51640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdb34068090 0x7fdb3406a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb83fff640 1 -- 172.21.15.43:0/3964745944 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90017070 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c1584b0 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c158cf0 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb83fff640 1 -- 172.21.15.43:0/3964745944 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb9000c940 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fcb83fff640 1 -- 172.21.15.43:0/3964745944 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000cd10 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.331+0000 7fdb55d51640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdb34068090 0x7fdb3406a550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000f70 tx=0x7fdb4c007df0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.335+0000 7fcb83fff640 1 -- 172.21.15.43:0/3964745944 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcb9001a070 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.335+0000 7fcb83fff640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcb88068010 0x7fcb8806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.335+0000 7fcb83fff640 1 -- 172.21.15.43:0/3964745944 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fcb90086700 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.335+0000 7fcb9affd640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcb88068010 0x7fcb8806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.335+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c0052a0 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.335+0000 7fcb9affd640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcb88068010 0x7fcb8806a4d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c1e6650 tx=0x7fcb8c009e60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.343+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb380052a0 con 0x7fdb5006f9e0 2024-03-30T13:11:22.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.347+0000 7fcb83fff640 1 -- 172.21.15.43:0/3964745944 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcb9c1584b0 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.351+0000 7fdb467fc640 1 -- 172.21.15.43:0/1042486918 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdb40055a20 con 0x7fdb5006f9e0 2024-03-30T13:11:22.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f47714c4640 1 Processor -- start 2024-03-30T13:11:22.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f47714c4640 1 -- start start 2024-03-30T13:11:22.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f47714c4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f476c073300 0x7f476c15e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f47714c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476c15b120 con 0x7f476c15eb30 2024-03-30T13:11:22.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f47714c4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f476c15b400 con 0x7f476c157e50 2024-03-30T13:11:22.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f47714c4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f476c15e160 con 0x7f476c073300 2024-03-30T13:11:22.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f476b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f476c073300 0x7f476c15e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f4770cc3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f476c15eb30 0x7f476c0b4450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59682/0 (socket says 172.21.15.43:59682) 2024-03-30T13:11:22.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f4770cc3640 1 -- 172.21.15.43:0/3871885400 learned_addr learned my addr 172.21.15.43:0/3871885400 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:22.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f476b7fe640 1 -- 172.21.15.43:0/3871885400 >> v1:172.21.15.113:6789/0 conn(0x7f476c157e50 legacy=0x7f476c15a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f476b7fe640 1 -- 172.21.15.43:0/3871885400 >> v1:172.21.15.43:6789/0 conn(0x7f476c15eb30 legacy=0x7f476c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.395+0000 7f476b7fe640 1 -- 172.21.15.43:0/3871885400 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c15b6e0 con 0x7f476c073300 2024-03-30T13:11:22.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.399+0000 7f476b7fe640 1 --2- 172.21.15.43:0/3871885400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c073300 0x7f476c15e5f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f476c15e2e0 tx=0x7f476001d7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa648375d0a987 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f476affd640 1 -- 172.21.15.43:0/3871885400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476000b040 con 0x7f476c073300 2024-03-30T13:11:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f476affd640 1 -- 172.21.15.43:0/3871885400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47600a76c0 con 0x7f476c073300 2024-03-30T13:11:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f476affd640 1 -- 172.21.15.43:0/3871885400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47600a7a90 con 0x7f476c073300 2024-03-30T13:11:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f47714c4640 1 -- 172.21.15.43:0/3871885400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c073300 msgr2=0x7f476c15e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f47714c4640 1 --2- 172.21.15.43:0/3871885400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c073300 0x7f476c15e5f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f476c15e2e0 tx=0x7f476001d7e0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f47714c4640 1 -- 172.21.15.43:0/3871885400 shutdown_connections 2024-03-30T13:11:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f47714c4640 1 --2- 172.21.15.43:0/3871885400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c073300 0x7f476c15e5f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.403+0000 7f47714c4640 1 -- 172.21.15.43:0/3871885400 >> 172.21.15.43:0/3871885400 conn(0x7f476c06c5a0 msgr2=0x7f476c06c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 -- 172.21.15.43:0/3871885400 shutdown_connections 2024-03-30T13:11:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 -- 172.21.15.43:0/3871885400 wait complete. 2024-03-30T13:11:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 Processor -- start 2024-03-30T13:11:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 -- start start 2024-03-30T13:11:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c073300 0x7f476c14a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c157e50 0x7f476c1419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f476c15eb30 0x7f476c141f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f476c0b4b60 con 0x7f476c157e50 2024-03-30T13:11:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f476c0ce470 con 0x7f476c073300 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f47714c4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f476c0b4ce0 con 0x7f476c15eb30 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f4770cc3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f476c15eb30 0x7f476c141f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f476bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c073300 0x7f476c14a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f476bfff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c073300 0x7f476c14a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:46938/0 (socket says 172.21.15.43:46938) 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.415+0000 7f476bfff640 1 -- 172.21.15.43:0/4085435472 learned_addr learned my addr 172.21.15.43:0/4085435472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.419+0000 7f476b7fe640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c157e50 0x7f476c1419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.419+0000 7f476bfff640 1 -- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f476c15eb30 msgr2=0x7f476c141f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.419+0000 7f476bfff640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f476c15eb30 0x7f476c141f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.419+0000 7f476bfff640 1 -- 172.21.15.43:0/4085435472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c157e50 msgr2=0x7f476c1419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.419+0000 7f476bfff640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c157e50 0x7f476c1419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.419+0000 7f476bfff640 1 -- 172.21.15.43:0/4085435472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476c159280 con 0x7f476c073300 2024-03-30T13:11:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.419+0000 7f476bfff640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c073300 0x7f476c14a540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f476400c730 tx=0x7f476400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.427+0000 7f4768ff9640 1 -- 172.21.15.43:0/4085435472 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4764003b80 con 0x7f476c073300 2024-03-30T13:11:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.427+0000 7f4768ff9640 1 -- 172.21.15.43:0/4085435472 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4764003d40 con 0x7f476c073300 2024-03-30T13:11:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.427+0000 7f4768ff9640 1 -- 172.21.15.43:0/4085435472 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476400aa90 con 0x7f476c073300 2024-03-30T13:11:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.427+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476c159560 con 0x7f476c073300 2024-03-30T13:11:22.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.427+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f476c159b20 con 0x7f476c073300 2024-03-30T13:11:22.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.435+0000 7f4768ff9640 1 -- 172.21.15.43:0/4085435472 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f476400ac30 con 0x7f476c073300 2024-03-30T13:11:22.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.435+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f476c068670 con 0x7f476c073300 2024-03-30T13:11:22.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.435+0000 7f4768ff9640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f47580680c0 0x7f475806a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:22.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.435+0000 7f4768ff9640 1 -- 172.21.15.43:0/4085435472 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4764086610 con 0x7f476c073300 2024-03-30T13:11:22.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.435+0000 7f476b7fe640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f47580680c0 0x7f475806a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:22.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.435+0000 7f476b7fe640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f47580680c0 0x7f475806a580 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47600b0000 tx=0x7f476000c040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:22.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.447+0000 7f4768ff9640 1 -- 172.21.15.43:0/4085435472 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4764056410 con 0x7f476c073300 2024-03-30T13:11:22.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.495+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4854005860 con 0x7f4874142000 2024-03-30T13:11:22.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.499+0000 7f4869ffb640 1 -- 172.21.15.43:0/4090875476 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f486c056d20 con 0x7f4874142000 2024-03-30T13:11:22.505 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4858067fc0 msgr2=0x7f485806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4858067fc0 0x7f485806a480 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4870000f70 tx=0x7f4870007a90 comp rx=0 tx=0).stop 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4874142000 msgr2=0x7f4874143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4874142000 0x7f4874143760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f486c00c780 tx=0x7f486c00cc40 comp rx=0 tx=0).stop 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 shutdown_connections 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4858067fc0 0x7f485806a480 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4874156830 0x7f487406eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4874142000 0x7f4874143760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 --2- 172.21.15.43:0/4090875476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4874072bc0 0x7f487413f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 >> 172.21.15.43:0/4090875476 conn(0x7f487406b660 msgr2=0x7f487415a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 shutdown_connections 2024-03-30T13:11:22.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.523+0000 7f487b2d3640 1 -- 172.21.15.43:0/4090875476 wait complete. 2024-03-30T13:11:22.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.2 2024-03-30T13:11:22.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.571+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc478153fd0 con 0x7fc4780b3840 2024-03-30T13:11:22.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.575+0000 7fc4757fa640 1 -- 172.21.15.43:0/575691749 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc470056940 con 0x7fc4780b3840 2024-03-30T13:11:22.583 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:22.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.587+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f36880dfc00 con 0x7f3688035470 2024-03-30T13:11:22.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.587+0000 7f367effd640 1 -- 172.21.15.43:0/1739417254 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3684059ca0 con 0x7f3688035470 2024-03-30T13:11:22.594 INFO:teuthology.orchestra.run.smithi043.stdout:64424509449 2024-03-30T13:11:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3670068090 msgr2=0x7f367006a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3670068090 0x7f367006a550 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3690051240 tx=0x7f36900693f0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3688035470 msgr2=0x7f36880e2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3688035470 0x7f36880e2f40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f368400c480 tx=0x7f368400c940 comp rx=0 tx=0).stop 2024-03-30T13:11:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 shutdown_connections 2024-03-30T13:11:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3670068090 0x7f367006a550 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f36880e1600 0x7f368804c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36880e1220 0x7f3688037150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 --2- 172.21.15.43:0/1739417254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3688035470 0x7f36880e2f40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 >> 172.21.15.43:0/1739417254 conn(0x7f368801e6d0 msgr2=0x7f368803a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 shutdown_connections 2024-03-30T13:11:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7f368f59e640 1 -- 172.21.15.43:0/1739417254 wait complete. 2024-03-30T13:11:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc464067fc0 msgr2=0x7fc46406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc464067fc0 0x7fc46406a480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc4680ae000 tx=0x7fc4680a7070 comp rx=0 tx=0).stop 2024-03-30T13:11:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780b3840 msgr2=0x7fc4781616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780b3840 0x7fc4781616b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d830 tx=0x7fc47000dc30 comp rx=0 tx=0).stop 2024-03-30T13:11:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 shutdown_connections 2024-03-30T13:11:22.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc464067fc0 0x7fc46406a480 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4780c6f90 0x7fc478152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4780cef80 0x7fc478152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 --2- 172.21.15.43:0/575691749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4780b3840 0x7fc4781616b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 >> 172.21.15.43:0/575691749 conn(0x7fc4780d4310 msgr2=0x7fc47806c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 shutdown_connections 2024-03-30T13:11:22.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.591+0000 7fc47ee73640 1 -- 172.21.15.43:0/575691749 wait complete. 2024-03-30T13:11:22.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.611+0000 7f4552400640 1 -- 172.21.15.43:0/2493172854 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f4528005860 con 0x7f454c15a2c0 2024-03-30T13:11:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.611+0000 7f4548ff9640 1 -- 172.21.15.43:0/2493172854 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f454005e1c0 con 0x7f454c15a2c0 2024-03-30T13:11:22.618 INFO:teuthology.orchestra.run.smithi043.stdout:64424509449 2024-03-30T13:11:22.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.1 2024-03-30T13:11:22.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.3 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 -- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4530068010 msgr2=0x7f453006a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4530068010 0x7f453006a4d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4544008150 tx=0x7f454400f040 comp rx=0 tx=0).stop 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 -- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c15a2c0 msgr2=0x7f454c137e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c15a2c0 0x7f454c137e70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f454000b3f0 tx=0x7f454000b8b0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 -- 172.21.15.43:0/2493172854 shutdown_connections 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4530068010 0x7f453006a4d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f454c15a6a0 0x7f454c155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f454c15a2c0 0x7f454c137e70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 --2- 172.21.15.43:0/2493172854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f454c072f20 0x7f454c137610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 -- 172.21.15.43:0/2493172854 >> 172.21.15.43:0/2493172854 conn(0x7f454c06c570 msgr2=0x7f454c0cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 -- 172.21.15.43:0/2493172854 shutdown_connections 2024-03-30T13:11:22.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.623+0000 7f453a7fc640 1 -- 172.21.15.43:0/2493172854 wait complete. 2024-03-30T13:11:22.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.0 2024-03-30T13:11:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.647+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7faa50005590 con 0x7faa6c196420 2024-03-30T13:11:22.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.647+0000 7faa637fe640 1 -- 172.21.15.43:0/3313850878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faa64056990 con 0x7faa6c196420 2024-03-30T13:11:22.656 INFO:teuthology.orchestra.run.smithi043.stdout:55834574858 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa580680c0 msgr2=0x7faa5806a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa580680c0 0x7faa5806a580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa68009900 tx=0x7faa68009060 comp rx=0 tx=0).stop 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c196420 msgr2=0x7faa6c04c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c196420 0x7faa6c04c5b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa6400c820 tx=0x7faa6400cce0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 shutdown_connections 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa580680c0 0x7faa5806a580 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa6c0f5e20 0x7faa6c0f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa6c196420 0x7faa6c04c5b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 --2- 172.21.15.43:0/3313850878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa5c004e40 0x7faa6c04ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 >> 172.21.15.43:0/3313850878 conn(0x7faa6c009fc0 msgr2=0x7faa6c0fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 shutdown_connections 2024-03-30T13:11:22.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.651+0000 7faa788cb640 1 -- 172.21.15.43:0/3313850878 wait complete. 2024-03-30T13:11:22.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-03-30T13:11:22.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcb7c005860 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcb83fff640 1 -- 172.21.15.43:0/3964745944 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb9c205180 con 0x7fcb9c06cbb0 2024-03-30T13:11:22.682 INFO:teuthology.orchestra.run.smithi043.stdout:64424509449 2024-03-30T13:11:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcb88068010 msgr2=0x7fcb8806a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcb88068010 0x7fcb8806a4d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c1e6650 tx=0x7fcb8c009e60 comp rx=0 tx=0).stop 2024-03-30T13:11:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c06cbb0 msgr2=0x7fcb9c1e93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c06cbb0 0x7fcb9c1e93b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000e8d0 tx=0x7fcb9000ed90 comp rx=0 tx=0).stop 2024-03-30T13:11:22.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 shutdown_connections 2024-03-30T13:11:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcb88068010 0x7fcb8806a4d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcb9c0cff60 0x7fcb9c151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c06e460 0x7fcb9c204c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 --2- 172.21.15.43:0/3964745944 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcb9c06cbb0 0x7fcb9c1e93b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 >> 172.21.15.43:0/3964745944 conn(0x7fcb9c071ca0 msgr2=0x7fcb9c0b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 shutdown_connections 2024-03-30T13:11:22.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.675+0000 7fcba0977640 1 -- 172.21.15.43:0/3964745944 wait complete. 2024-03-30T13:11:22.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.683+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdb38005590 con 0x7fdb5006f9e0 2024-03-30T13:11:22.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.683+0000 7fdb467fc640 1 -- 172.21.15.43:0/1042486918 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb40059d20 con 0x7fdb5006f9e0 2024-03-30T13:11:22.691 INFO:teuthology.orchestra.run.smithi043.stdout:64424509449 2024-03-30T13:11:22.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdb34068090 msgr2=0x7fdb3406a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdb34068090 0x7fdb3406a550 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000f70 tx=0x7fdb4c007df0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb5006f9e0 msgr2=0x7fdb50159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb5006f9e0 0x7fdb50159e10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000c550 tx=0x7fdb4000ca10 comp rx=0 tx=0).stop 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 shutdown_connections 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdb34068090 0x7fdb3406a550 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdb500b5540 0x7fdb5015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb5006f9e0 0x7fdb50159e10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 --2- 172.21.15.43:0/1042486918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdb5006ce80 0x7fdb50147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 >> 172.21.15.43:0/1042486918 conn(0x7fdb5006e410 msgr2=0x7fdb500b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 shutdown_connections 2024-03-30T13:11:22.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.687+0000 7fdb57fb5640 1 -- 172.21.15.43:0/1042486918 wait complete. 2024-03-30T13:11:22.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.4 2024-03-30T13:11:22.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-03-30T13:11:22.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.751+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f476c0729f0 con 0x7f476c073300 2024-03-30T13:11:22.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.751+0000 7f4768ff9640 1 -- 172.21.15.43:0/4085435472 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f476405a710 con 0x7f476c073300 2024-03-30T13:11:22.760 INFO:teuthology.orchestra.run.smithi043.stdout:64424509448 2024-03-30T13:11:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f47580680c0 msgr2=0x7f475806a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f47580680c0 0x7f475806a580 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f47600b0000 tx=0x7f476000c040 comp rx=0 tx=0).stop 2024-03-30T13:11:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c073300 msgr2=0x7f476c14a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c073300 0x7f476c14a540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f476400c730 tx=0x7f476400cbf0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 shutdown_connections 2024-03-30T13:11:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f47580680c0 0x7f475806a580 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f476c15eb30 0x7f476c141f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f476c157e50 0x7f476c1419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 --2- 172.21.15.43:0/4085435472 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f476c073300 0x7f476c14a540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 >> 172.21.15.43:0/4085435472 conn(0x7f476c06c5a0 msgr2=0x7f476c154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 shutdown_connections 2024-03-30T13:11:22.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:22.755+0000 7f47714c4640 1 -- 172.21.15.43:0/4085435472 wait complete. 2024-03-30T13:11:22.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-03-30T13:11:23.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T13:11:23.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-30T13:11:23.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-30T13:11:23.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-30T13:11:23.680 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-30T13:11:23.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510e3bf640 1 Processor -- start 2024-03-30T13:11:23.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510e3bf640 1 -- start start 2024-03-30T13:11:23.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510e3bf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f510806f9e0 0x7f510806fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510e3bf640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f510806c770 0x7f510806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510e3bf640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5108070300 con 0x7f51080b4530 2024-03-30T13:11:23.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510e3bf640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5108152550 con 0x7f510806c770 2024-03-30T13:11:23.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510e3bf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f510806d430 con 0x7f510806f9e0 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510c95c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f51080b4530 0x7f51080b4980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59694/0 (socket says 172.21.15.43:59694) 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f510c95c640 1 -- 172.21.15.43:0/849057203 learned_addr learned my addr 172.21.15.43:0/849057203 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f51077fe640 1 --2- 172.21.15.43:0/849057203 >> v2:172.21.15.113:3300/0 conn(0x7f510806c770 0x7f510806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f5107fff640 1 --2- 172.21.15.43:0/849057203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f510806f9e0 0x7f510806fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f5106ffd640 1 -- 172.21.15.43:0/849057203 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252541031 0 0) 0x7f5108070300 con 0x7f51080b4530 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f5106ffd640 1 -- 172.21.15.43:0/849057203 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f0003530 con 0x7f51080b4530 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f5107fff640 1 -- 172.21.15.43:0/849057203 >> v2:172.21.15.113:3300/0 conn(0x7f510806c770 msgr2=0x7f510806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.691+0000 7f5107fff640 1 --2- 172.21.15.43:0/849057203 >> v2:172.21.15.113:3300/0 conn(0x7f510806c770 0x7f510806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.695+0000 7f5107fff640 1 -- 172.21.15.43:0/849057203 >> v1:172.21.15.43:6789/0 conn(0x7f51080b4530 legacy=0x7f51080b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.695+0000 7f5107fff640 1 -- 172.21.15.43:0/849057203 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510806d710 con 0x7f510806f9e0 2024-03-30T13:11:23.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-30T13:11:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.695+0000 7f5107fff640 1 --2- 172.21.15.43:0/849057203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f510806f9e0 0x7f510806fdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50f80097c0 tx=0x7f50f801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=480972f3288ee3bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.695+0000 7f5106ffd640 1 -- 172.21.15.43:0/849057203 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80a7070 con 0x7f510806f9e0 2024-03-30T13:11:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.695+0000 7f5106ffd640 1 -- 172.21.15.43:0/849057203 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f80b0070 con 0x7f510806f9e0 2024-03-30T13:11:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.695+0000 7f5106ffd640 1 -- 172.21.15.43:0/849057203 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80ab890 con 0x7f510806f9e0 2024-03-30T13:11:23.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- 172.21.15.43:0/849057203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f510806f9e0 msgr2=0x7f510806fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 --2- 172.21.15.43:0/849057203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f510806f9e0 0x7f510806fdc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f50f80097c0 tx=0x7f50f801c470 comp rx=0 tx=0).stop 2024-03-30T13:11:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- 172.21.15.43:0/849057203 shutdown_connections 2024-03-30T13:11:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 --2- 172.21.15.43:0/849057203 >> v2:172.21.15.113:3300/0 conn(0x7f510806c770 0x7f510806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 --2- 172.21.15.43:0/849057203 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f510806f9e0 0x7f510806fdc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- 172.21.15.43:0/849057203 >> 172.21.15.43:0/849057203 conn(0x7f510806e410 msgr2=0x7f510806e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- 172.21.15.43:0/849057203 shutdown_connections 2024-03-30T13:11:23.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- 172.21.15.43:0/849057203 wait complete. 2024-03-30T13:11:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 Processor -- start 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- start start 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f510806c770 0x7f510814ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510806f9e0 0x7f510814f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51080b4530 0x7f51081493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f510815c460 con 0x7f510806c770 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f510806d130 con 0x7f510806f9e0 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510e3bf640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5108151830 con 0x7f51080b4530 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f5107fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f510806c770 0x7f510814ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f5107fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f510806c770 0x7f510814ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34970/0 (socket says 172.21.15.43:34970) 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f5107fff640 1 -- 172.21.15.43:0/3829773323 learned_addr learned my addr 172.21.15.43:0/3829773323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f510c95c640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51080b4530 0x7f51081493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.699+0000 7f51077fe640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510806f9e0 0x7f510814f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f5107fff640 1 -- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51080b4530 msgr2=0x7f51081493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f5107fff640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51080b4530 0x7f51081493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f5107fff640 1 -- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510806f9e0 msgr2=0x7f510814f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f5107fff640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510806f9e0 0x7f510814f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f5107fff640 1 -- 172.21.15.43:0/3829773323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5108150fb0 con 0x7f510806c770 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f51077fe640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510806f9e0 0x7f510814f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f510c95c640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51080b4530 0x7f51081493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:23.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.703+0000 7f5107fff640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f510806c770 0x7f510814ef40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f50f80ad040 tx=0x7f50f801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.711 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-30T13:11:23.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.707+0000 7f5104ff9640 1 -- 172.21.15.43:0/3829773323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80bf610 con 0x7f510806c770 2024-03-30T13:11:23.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.707+0000 7f5104ff9640 1 -- 172.21.15.43:0/3829773323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f80abe50 con 0x7f510806c770 2024-03-30T13:11:23.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.707+0000 7f5104ff9640 1 -- 172.21.15.43:0/3829773323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f80bfa10 con 0x7f510806c770 2024-03-30T13:11:23.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.707+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510814d530 con 0x7f510806c770 2024-03-30T13:11:23.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.707+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5108151550 con 0x7f510806c770 2024-03-30T13:11:23.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.707+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5108068890 con 0x7f510806c770 2024-03-30T13:11:23.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.707+0000 7f5104ff9640 1 -- 172.21.15.43:0/3829773323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f50f80bd020 con 0x7f510806c770 2024-03-30T13:11:23.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.711+0000 7f5104ff9640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50f4068090 0x7f50f406a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.711+0000 7f5104ff9640 1 -- 172.21.15.43:0/3829773323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f50f8121010 con 0x7f510806c770 2024-03-30T13:11:23.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.715+0000 7f51077fe640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50f4068090 0x7f50f406a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.715+0000 7f51077fe640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50f4068090 0x7f50f406a550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5108150da0 tx=0x7f50fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.719+0000 7f5104ff9640 1 -- 172.21.15.43:0/3829773323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f50f8016090 con 0x7f510806c770 2024-03-30T13:11:23.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-30T13:11:23.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 Processor -- start 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 -- start start 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3184142f80 0x7f318415df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f318415e4d0 0x7f318415e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3184071c60 0x7f3184072090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f31840b48d0 con 0x7f3184071c60 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f31840b5410 con 0x7f3184142f80 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f318a848640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f31840b4b70 con 0x7f318415e4d0 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3188de5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3184071c60 0x7f3184072090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3188de5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3184071c60 0x7f3184072090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34974/0 (socket says 172.21.15.43:34974) 2024-03-30T13:11:23.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3188de5640 1 -- 172.21.15.43:0/3366002378 learned_addr learned my addr 172.21.15.43:0/3366002378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f31837fe640 1 --2- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f318415e4d0 0x7f318415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3183fff640 1 --2- 172.21.15.43:0/3366002378 >> v2:172.21.15.113:3300/0 conn(0x7f3184142f80 0x7f318415df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3183fff640 1 -- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f318415e4d0 msgr2=0x7f318415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3183fff640 1 --2- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f318415e4d0 0x7f318415e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3183fff640 1 -- 172.21.15.43:0/3366002378 >> v2:172.21.15.43:3300/0 conn(0x7f3184071c60 msgr2=0x7f3184072090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3183fff640 1 --2- 172.21.15.43:0/3366002378 >> v2:172.21.15.43:3300/0 conn(0x7f3184071c60 0x7f3184072090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.775+0000 7f3183fff640 1 -- 172.21.15.43:0/3366002378 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31840b4e50 con 0x7f3184142f80 2024-03-30T13:11:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.779+0000 7f3183fff640 1 --2- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318415df90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3184144240 tx=0x7f3174020950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9f42d4793dc8a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.779+0000 7f3182ffd640 1 -- 172.21.15.43:0/3366002378 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31740b6070 con 0x7f3184142f80 2024-03-30T13:11:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.779+0000 7f3182ffd640 1 -- 172.21.15.43:0/3366002378 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31740a97e0 con 0x7f3184142f80 2024-03-30T13:11:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.779+0000 7f3182ffd640 1 -- 172.21.15.43:0/3366002378 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31740a9bf0 con 0x7f3184142f80 2024-03-30T13:11:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.783+0000 7f318a848640 1 -- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 msgr2=0x7f318415df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.783+0000 7f318a848640 1 --2- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318415df90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3184144240 tx=0x7f3174020950 comp rx=0 tx=0).stop 2024-03-30T13:11:23.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.787+0000 7f318a848640 1 -- 172.21.15.43:0/3366002378 shutdown_connections 2024-03-30T13:11:23.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.787+0000 7f318a848640 1 --2- 172.21.15.43:0/3366002378 >> v2:172.21.15.43:3300/0 conn(0x7f3184071c60 0x7f3184072090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.787+0000 7f318a848640 1 --2- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f318415e4d0 0x7f318415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.787+0000 7f318a848640 1 --2- 172.21.15.43:0/3366002378 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318415df90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.787+0000 7f318a848640 1 -- 172.21.15.43:0/3366002378 >> 172.21.15.43:0/3366002378 conn(0x7f318406b660 msgr2=0x7f318406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.787+0000 7f318a848640 1 -- 172.21.15.43:0/3366002378 shutdown_connections 2024-03-30T13:11:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.787+0000 7f318a848640 1 -- 172.21.15.43:0/3366002378 wait complete. 2024-03-30T13:11:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 Processor -- start 2024-03-30T13:11:23.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 -- start start 2024-03-30T13:11:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3184071c60 0x7f31841377e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318413a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318415e4d0 0x7f318413f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f318406da40 con 0x7f318415e4d0 2024-03-30T13:11:23.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f318406d8c0 con 0x7f3184142f80 2024-03-30T13:11:23.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f318a848640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f318406dbc0 con 0x7f3184071c60 2024-03-30T13:11:23.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3188de5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318415e4d0 0x7f318413f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3184071c60 0x7f31841377e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f31837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318413a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f31837fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318413a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:46984/0 (socket says 172.21.15.43:46984) 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3184071c60 0x7f31841377e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37468/0 (socket says 172.21.15.43:37468) 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f31837fe640 1 -- 172.21.15.43:0/2272775579 learned_addr learned my addr 172.21.15.43:0/2272775579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3188de5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318415e4d0 0x7f318413f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34984/0 (socket says 172.21.15.43:34984) 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 -- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 msgr2=0x7f318413a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318413a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 -- 172.21.15.43:0/2272775579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318415e4d0 msgr2=0x7f318413f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318415e4d0 0x7f318413f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 -- 172.21.15.43:0/2272775579 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318413a8b0 con 0x7f3184071c60 2024-03-30T13:11:23.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3188de5640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318415e4d0 0x7f318413f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3183fff640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3184071c60 0x7f31841377e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f31740b0000 tx=0x7f317401d6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.791+0000 7f3180ff9640 1 -- 172.21.15.43:0/2272775579 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31740be020 con 0x7f3184071c60 2024-03-30T13:11:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.795+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31841392d0 con 0x7f3184071c60 2024-03-30T13:11:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.795+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3184139890 con 0x7f3184071c60 2024-03-30T13:11:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.795+0000 7f3180ff9640 1 -- 172.21.15.43:0/2272775579 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31740b1440 con 0x7f3184071c60 2024-03-30T13:11:23.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.795+0000 7f3180ff9640 1 -- 172.21.15.43:0/2272775579 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31740b6040 con 0x7f3184071c60 2024-03-30T13:11:23.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.795+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3184068890 con 0x7f3184071c60 2024-03-30T13:11:23.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.795+0000 7f3180ff9640 1 -- 172.21.15.43:0/2272775579 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f31740c1020 con 0x7f3184071c60 2024-03-30T13:11:23.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.795+0000 7f3180ff9640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f316c068120 0x7f316c06a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.799+0000 7f3180ff9640 1 -- 172.21.15.43:0/2272775579 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3174124c00 con 0x7f3184071c60 2024-03-30T13:11:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.799+0000 7f31837fe640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f316c068120 0x7f316c06a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.799+0000 7f31837fe640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f316c068120 0x7f316c06a5e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f31840b0130 tx=0x7f317c009750 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.811+0000 7f3180ff9640 1 -- 172.21.15.43:0/2272775579 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f31740c8030 con 0x7f3184071c60 2024-03-30T13:11:23.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.847+0000 7fba09fb6640 1 Processor -- start 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.847+0000 7fba09fb6640 1 -- start start 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba09fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc100e30 0x7fb9fc054230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba09fb6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb9fc007290 0x7fb9fc04f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba09fb6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9fc050140 con 0x7fb9fc04f3e0 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba09fb6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb9fc037b30 con 0x7fb9fc007290 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba09fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9fc053ae0 con 0x7fb9fc100e30 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba02ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb9fc04f3e0 0x7fb9fc104850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59706/0 (socket says 172.21.15.43:59706) 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba02ffd640 1 -- 172.21.15.43:0/3519839230 learned_addr learned my addr 172.21.15.43:0/3519839230 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba03fff640 1 --2- 172.21.15.43:0/3519839230 >> v2:172.21.15.113:3300/0 conn(0x7fb9fc007290 0x7fb9fc04f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba037fe640 1 --2- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc100e30 0x7fb9fc054230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.851+0000 7fba027fc640 1 -- 172.21.15.43:0/3519839230 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145443340 0 0) 0x7fb9fc050140 con 0x7fb9fc04f3e0 2024-03-30T13:11:23.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.855+0000 7fba027fc640 1 -- 172.21.15.43:0/3519839230 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9f4003430 con 0x7fb9fc04f3e0 2024-03-30T13:11:23.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.855+0000 7fba027fc640 1 -- 172.21.15.43:0/3519839230 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2538791435 0 0) 0x7fb9f4003430 con 0x7fb9fc04f3e0 2024-03-30T13:11:23.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.855+0000 7fba03fff640 1 -- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc100e30 msgr2=0x7fb9fc054230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.855+0000 7fba03fff640 1 --2- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc100e30 0x7fb9fc054230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.855+0000 7fba03fff640 1 -- 172.21.15.43:0/3519839230 >> v1:172.21.15.43:6789/0 conn(0x7fb9fc04f3e0 legacy=0x7fb9fc104850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.855+0000 7fba03fff640 1 -- 172.21.15.43:0/3519839230 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc053800 con 0x7fb9fc007290 2024-03-30T13:11:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.859+0000 7fba03fff640 1 --2- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc04f870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fba040512a0 tx=0x7fba04091ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8035d14e9c9094 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.859+0000 7fba027fc640 1 -- 172.21.15.43:0/3519839230 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0001680 con 0x7fb9fc007290 2024-03-30T13:11:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.859+0000 7fba027fc640 1 -- 172.21.15.43:0/3519839230 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0406faa0 con 0x7fb9fc007290 2024-03-30T13:11:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.859+0000 7fba027fc640 1 -- 172.21.15.43:0/3519839230 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba040710d0 con 0x7fb9fc007290 2024-03-30T13:11:23.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 msgr2=0x7fb9fc04f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 --2- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc04f870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fba040512a0 tx=0x7fba04091ab0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- 172.21.15.43:0/3519839230 shutdown_connections 2024-03-30T13:11:23.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 --2- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc04f870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 --2- 172.21.15.43:0/3519839230 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc100e30 0x7fb9fc054230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- 172.21.15.43:0/3519839230 >> 172.21.15.43:0/3519839230 conn(0x7fb9fc009fc0 msgr2=0x7fb9fc00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:23.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- 172.21.15.43:0/3519839230 shutdown_connections 2024-03-30T13:11:23.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- 172.21.15.43:0/3519839230 wait complete. 2024-03-30T13:11:23.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 Processor -- start 2024-03-30T13:11:23.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- start start 2024-03-30T13:11:23.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc0f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc04f3e0 0x7fb9fc196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9fc100e30 0x7fb9fc0f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc054770 con 0x7fb9fc100e30 2024-03-30T13:11:23.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc03a830 con 0x7fb9fc007290 2024-03-30T13:11:23.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.863+0000 7fba09fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9fc0533d0 con 0x7fb9fc04f3e0 2024-03-30T13:11:23.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9fc100e30 0x7fb9fc0f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba037fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc0f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba03fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9fc100e30 0x7fb9fc0f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34996/0 (socket says 172.21.15.43:34996) 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba03fff640 1 -- 172.21.15.43:0/2463612840 learned_addr learned my addr 172.21.15.43:0/2463612840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba02ffd640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc04f3e0 0x7fb9fc196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba037fe640 1 -- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc04f3e0 msgr2=0x7fb9fc196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba037fe640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc04f3e0 0x7fb9fc196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba037fe640 1 -- 172.21.15.43:0/2463612840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9fc100e30 msgr2=0x7fb9fc0f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba037fe640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9fc100e30 0x7fb9fc0f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba037fe640 1 -- 172.21.15.43:0/2463612840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc04be60 con 0x7fb9fc007290 2024-03-30T13:11:23.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba037fe640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc0f7c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800eb00 tx=0x7fb9f800ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fb9effff640 1 -- 172.21.15.43:0/2463612840 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f801c070 con 0x7fb9fc007290 2024-03-30T13:11:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc04c140 con 0x7fb9fc007290 2024-03-30T13:11:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc04c700 con 0x7fb9fc007290 2024-03-30T13:11:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fb9effff640 1 -- 172.21.15.43:0/2463612840 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9f800cab0 con 0x7fb9fc007290 2024-03-30T13:11:23.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fb9effff640 1 -- 172.21.15.43:0/2463612840 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800ce80 con 0x7fb9fc007290 2024-03-30T13:11:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9e40052a0 con 0x7fb9fc007290 2024-03-30T13:11:23.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.867+0000 7fb9effff640 1 -- 172.21.15.43:0/2463612840 <== mon.1 v2:172.21.15.113:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb9f801f070 con 0x7fb9fc007290 2024-03-30T13:11:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.871+0000 7fb9effff640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9f4068250 0x7fb9f406a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.871+0000 7fb9effff640 1 -- 172.21.15.43:0/2463612840 <== mon.1 v2:172.21.15.113:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb9f8086ba0 con 0x7fb9fc007290 2024-03-30T13:11:23.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.871+0000 7fba02ffd640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9f4068250 0x7fb9f406a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.871+0000 7fba02ffd640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9f4068250 0x7fb9f406a710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0fc190 tx=0x7fb9f0001ce0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.875+0000 7fc27f06e640 1 Processor -- start 2024-03-30T13:11:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.875+0000 7fc27f06e640 1 -- start start 2024-03-30T13:11:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.875+0000 7f49cf2ac640 1 Processor -- start 2024-03-30T13:11:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27f06e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc278158050 0x7fc2780cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27f06e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2780736e0 con 0x7fc278157bf0 2024-03-30T13:11:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27f06e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc27815b400 con 0x7fc278073300 2024-03-30T13:11:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27f06e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc27815e160 con 0x7fc278158050 2024-03-30T13:11:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27d86b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc278157bf0 0x7fc27815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59710/0 (socket says 172.21.15.43:59710) 2024-03-30T13:11:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27d86b640 1 -- 172.21.15.43:0/3677855453 learned_addr learned my addr 172.21.15.43:0/3677855453 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.875+0000 7f49cf2ac640 1 -- start start 2024-03-30T13:11:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cf2ac640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49c815a2c0 0x7f49c806f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cf2ac640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cf2ac640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c80b32f0 con 0x7f49c815a6a0 2024-03-30T13:11:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cf2ac640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f49c806f9a0 con 0x7f49c815a2c0 2024-03-30T13:11:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cf2ac640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f49c80b3e70 con 0x7f49c8072f20 2024-03-30T13:11:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27e86d640 1 --2- 172.21.15.43:0/3677855453 >> v2:172.21.15.43:3300/0 conn(0x7fc278158050 0x7fc2780cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd048640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49c815a2c0 0x7f49c806f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd048640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49c815a2c0 0x7f49c806f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:47016/0 (socket says 172.21.15.43:47016) 2024-03-30T13:11:23.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd048640 1 -- 172.21.15.43:0/108923618 learned_addr learned my addr 172.21.15.43:0/108923618 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd849640 1 --2- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd048640 1 -- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 msgr2=0x7f49c806d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd048640 1 --2- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c806d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd048640 1 -- 172.21.15.43:0/108923618 >> v1:172.21.15.43:6789/0 conn(0x7f49c815a6a0 legacy=0x7f49c8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27e86d640 1 -- 172.21.15.43:0/3677855453 >> v1:172.21.15.113:6789/0 conn(0x7fc278073300 legacy=0x7fc27815a8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:11:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27e86d640 1 -- 172.21.15.43:0/3677855453 >> v1:172.21.15.43:6789/0 conn(0x7fc278157bf0 legacy=0x7fc27815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7fc27e86d640 1 -- 172.21.15.43:0/3677855453 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27815b6e0 con 0x7fc278158050 2024-03-30T13:11:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.879+0000 7f49cd048640 1 -- 172.21.15.43:0/108923618 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c80b35d0 con 0x7f49c815a2c0 2024-03-30T13:11:23.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7f49cd048640 1 --2- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c806f460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49c80b6140 tx=0x7f49c001ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d069ad96d263d8eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27e86d640 1 --2- 172.21.15.43:0/3677855453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278158050 0x7fc2780cda10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc270009d90 tx=0x7fc27001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36144a6283eea2be server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27d06a640 1 -- 172.21.15.43:0/3677855453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2700b2070 con 0x7fc278158050 2024-03-30T13:11:23.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27d06a640 1 -- 172.21.15.43:0/3677855453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2700a58f0 con 0x7fc278158050 2024-03-30T13:11:23.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fb9effff640 1 -- 172.21.15.43:0/2463612840 <== mon.1 v2:172.21.15.113:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb9f80569b0 con 0x7fb9fc007290 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27f06e640 1 -- 172.21.15.43:0/3677855453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278158050 msgr2=0x7fc2780cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27f06e640 1 --2- 172.21.15.43:0/3677855453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278158050 0x7fc2780cda10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc270009d90 tx=0x7fc27001c850 comp rx=0 tx=0).stop 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27f06e640 1 -- 172.21.15.43:0/3677855453 shutdown_connections 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27f06e640 1 --2- 172.21.15.43:0/3677855453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc278158050 0x7fc2780cda10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27f06e640 1 -- 172.21.15.43:0/3677855453 >> 172.21.15.43:0/3677855453 conn(0x7fc27806c5a0 msgr2=0x7fc27806c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27f06e640 1 -- 172.21.15.43:0/3677855453 shutdown_connections 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7f49bffff640 1 -- 172.21.15.43:0/108923618 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00ad070 con 0x7f49c815a2c0 2024-03-30T13:11:23.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7f49bffff640 1 -- 172.21.15.43:0/108923618 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c00a7d80 con 0x7f49c815a2c0 2024-03-30T13:11:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f49cf2ac640 1 -- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 msgr2=0x7f49c806f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c806f460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49c80b6140 tx=0x7f49c001ecd0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f02345640 1 Processor -- start 2024-03-30T13:11:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f49cf2ac640 1 -- 172.21.15.43:0/108923618 shutdown_connections 2024-03-30T13:11:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c806d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/108923618 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c806f460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f49cf2ac640 1 -- 172.21.15.43:0/108923618 >> 172.21.15.43:0/108923618 conn(0x7f49c806c570 msgr2=0x7f49c806c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.883+0000 7fc27f06e640 1 -- 172.21.15.43:0/3677855453 wait complete. 2024-03-30T13:11:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 Processor -- start 2024-03-30T13:11:23.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 -- start start 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f02345640 1 -- start start 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f02345640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1efc140a70 0x7f1efc141060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f02345640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1efc0cda60 0x7f1efc1415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f02345640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1efc0cf6f0 con 0x7f1efc140690 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f02345640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1efc0cdf40 con 0x7f1efc140a70 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f02345640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1efc0ce220 con 0x7f1efc0cda60 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc278073300 0x7fc27806e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc278157bf0 0x7fc27806d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc27806dbb0 0x7fc278139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780ce0d0 con 0x7fc27806dbb0 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc2780cdf50 con 0x7fc278073300 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27f06e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc2780ce250 con 0x7fc278157bf0 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fb478cf1640 1 Processor -- start 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fb478cf1640 1 -- start start 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f01b44640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1efc0cda60 0x7f1efc1415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f01b44640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1efc0cda60 0x7f1efc1415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:47028/0 (socket says 172.21.15.43:47028) 2024-03-30T13:11:23.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f01b44640 1 -- 172.21.15.43:0/3387771157 learned_addr learned my addr 172.21.15.43:0/3387771157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc278157bf0 0x7fc27806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc278157bf0 0x7fc27806d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37492/0 (socket says 172.21.15.43:37492) 2024-03-30T13:11:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 -- 172.21.15.43:0/2801571992 learned_addr learned my addr 172.21.15.43:0/2801571992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f00b42640 1 --2- 172.21.15.43:0/3387771157 >> v2:172.21.15.43:3300/0 conn(0x7f1efc140a70 0x7f1efc141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27e86d640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc27806dbb0 0x7fc278139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27e06c640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc278073300 0x7fc27806e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 -- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc278073300 msgr2=0x7fc27806e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc278073300 0x7fc27806e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 -- 172.21.15.43:0/2801571992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc27806dbb0 msgr2=0x7fc278139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc27806dbb0 0x7fc278139ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fc27d86b640 1 -- 172.21.15.43:0/2801571992 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2780b3d00 con 0x7fc278157bf0 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fb478cf1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb46c0df390 0x7fb46c007190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fb478cf1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c04f5c0 0x7fb46c038020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fb478cf1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb46c054390 con 0x7fb46c04fa80 2024-03-30T13:11:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fb478cf1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb46c005ea0 con 0x7fb46c0df390 2024-03-30T13:11:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7fb478cf1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb46c006d00 con 0x7fb46c04f5c0 2024-03-30T13:11:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f01b44640 1 -- 172.21.15.43:0/3387771157 >> v2:172.21.15.43:3300/0 conn(0x7f1efc140a70 msgr2=0x7f1efc141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f01b44640 1 --2- 172.21.15.43:0/3387771157 >> v2:172.21.15.43:3300/0 conn(0x7f1efc140a70 0x7f1efc141060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f01b44640 1 -- 172.21.15.43:0/3387771157 >> v1:172.21.15.113:6789/0 conn(0x7f1efc140690 legacy=0x7f1efc0cefe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-30T13:11:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.887+0000 7f1f01b44640 1 -- 172.21.15.43:0/3387771157 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc0ce500 con 0x7f1efc0cda60 2024-03-30T13:11:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7f1f00b42640 1 --2- 172.21.15.43:0/3387771157 >> v2:172.21.15.43:3300/0 conn(0x7f1efc140a70 0x7f1efc141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc27d86b640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc278157bf0 0x7fc27806d670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e80 tx=0x7fc27400c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc26effd640 1 -- 172.21.15.43:0/2801571992 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274019070 con 0x7fc278157bf0 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc26effd640 1 -- 172.21.15.43:0/2801571992 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2740078f0 con 0x7fc278157bf0 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2780b3fe0 con 0x7fc278157bf0 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc26effd640 1 -- 172.21.15.43:0/2801571992 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274007ca0 con 0x7fc278157bf0 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb473fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c04f5c0 0x7fb46c038020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7fb46c04fa80 0x7fb46c04fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:44564/0 (socket says 172.21.15.43:44564) 2024-03-30T13:11:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb473fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c04f5c0 0x7fb46c038020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37506/0 (socket says 172.21.15.43:37506) 2024-03-30T13:11:23.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 -- 172.21.15.43:0/2516984881 learned_addr learned my addr 172.21.15.43:0/2516984881 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:23.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2780b45a0 con 0x7fc278157bf0 2024-03-30T13:11:23.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1730957978 0 0) 0x7fb46c054390 con 0x7fb46c04fa80 2024-03-30T13:11:23.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb464003530 con 0x7fb46c04fa80 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 <== mon.1 v1:172.21.15.113:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 406668473 0 0) 0x7fb464003530 con 0x7fb46c04fa80 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c04f5c0 msgr2=0x7fb46c038020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c04f5c0 0x7fb46c038020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 >> v2:172.21.15.113:3300/0 conn(0x7fb46c0df390 msgr2=0x7fb46c007190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 --2- 172.21.15.43:0/2516984881 >> v2:172.21.15.113:3300/0 conn(0x7fb46c0df390 0x7fb46c007190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 --> v1:172.21.15.113:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c0060f0 con 0x7fb46c04fa80 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 <== mon.1 v1:172.21.15.113:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fb4740696a0 con 0x7fb46c04fa80 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 >> v1:172.21.15.113:6789/0 conn(0x7fb46c04fa80 legacy=0x7fb46c04fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb464003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb464004110 0x7fb46400b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46400b560 0x7fb46400d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c005ea0 con 0x7fb4640037b0 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c006d00 con 0x7fb464004110 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb46400e2e0 con 0x7fb46400b560 2024-03-30T13:11:23.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb473fff640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c04f5c0 0x7fb46c038020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc27806dfc0 con 0x7fc278157bf0 2024-03-30T13:11:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb473fff640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46400b560 0x7fb46400d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb464003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb472ffd640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb464004110 0x7fb46400b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7f1f01b44640 1 --2- 172.21.15.43:0/3387771157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0cda60 0x7f1efc1415a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009bd0 tx=0x7f1ef401ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6746d09a1738ccb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7f49cf2ac640 1 -- 172.21.15.43:0/108923618 shutdown_connections 2024-03-30T13:11:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7f49cf2ac640 1 -- 172.21.15.43:0/108923618 wait complete. 2024-03-30T13:11:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7f1ef3fff640 1 -- 172.21.15.43:0/3387771157 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40b3070 con 0x7f1efc0cda60 2024-03-30T13:11:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7f1ef3fff640 1 -- 172.21.15.43:0/3387771157 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef40ae400 con 0x7f1efc0cda60 2024-03-30T13:11:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7f1ef3fff640 1 -- 172.21.15.43:0/3387771157 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40ae7d0 con 0x7f1efc0cda60 2024-03-30T13:11:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 -- 172.21.15.43:0/3387771157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0cda60 msgr2=0x7f1efc1415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 --2- 172.21.15.43:0/3387771157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0cda60 0x7f1efc1415a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009bd0 tx=0x7f1ef401ef60 comp rx=0 tx=0).stop 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 -- 172.21.15.43:0/3387771157 shutdown_connections 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 --2- 172.21.15.43:0/3387771157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc0cda60 0x7f1efc1415a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 --2- 172.21.15.43:0/3387771157 >> v2:172.21.15.43:3300/0 conn(0x7f1efc140a70 0x7f1efc141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 -- 172.21.15.43:0/3387771157 >> 172.21.15.43:0/3387771157 conn(0x7f1efc0afb60 msgr2=0x7f1efc0aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fc26effd640 1 -- 172.21.15.43:0/2801571992 <== mon.2 v2:172.21.15.113:3301/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc27401c070 con 0x7fc278157bf0 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7fc26effd640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc25c067ff0 0x7fc25c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7fc26effd640 1 -- 172.21.15.43:0/2801571992 <== mon.2 v2:172.21.15.113:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc2740889a0 con 0x7fc278157bf0 2024-03-30T13:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 -- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46400b560 msgr2=0x7fb46400d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46400b560 0x7fb46400d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 -- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb464004110 msgr2=0x7fb46400b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb464004110 0x7fb46400b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.891+0000 7fb4737fe640 1 -- 172.21.15.43:0/2516984881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c0060f0 con 0x7fb4640037b0 2024-03-30T13:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 Processor -- start 2024-03-30T13:11:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7fc27e06c640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc25c067ff0 0x7fc25c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 -- start start 2024-03-30T13:11:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7fc27e06c640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc25c067ff0 0x7fc25c06a4b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc268009980 tx=0x7fc2680090e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 -- 172.21.15.43:0/3387771157 shutdown_connections 2024-03-30T13:11:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 -- 172.21.15.43:0/3387771157 wait complete. 2024-03-30T13:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c8152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c8141620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c815a6a0 0x7f49c81569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80b2e80 con 0x7f49c815a6a0 2024-03-30T13:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80b3000 con 0x7f49c815a2c0 2024-03-30T13:11:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f49cf2ac640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f49c8167250 con 0x7f49c8072f20 2024-03-30T13:11:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.895+0000 7f1f02345640 1 Processor -- start 2024-03-30T13:11:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cc847640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c8141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd048640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c8152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cc847640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c8141620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:47066/0 (socket says 172.21.15.43:47066) 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cc847640 1 -- 172.21.15.43:0/1397302194 learned_addr learned my addr 172.21.15.43:0/1397302194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd849640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c815a6a0 0x7f49c81569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd849640 1 -- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 msgr2=0x7f49c8152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd849640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c8152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd849640 1 -- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 msgr2=0x7f49c8141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd849640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c8141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd849640 1 -- 172.21.15.43:0/1397302194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c8157b90 con 0x7f49c815a6a0 2024-03-30T13:11:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cd849640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c815a6a0 0x7f49c81569f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49c815ef70 tx=0x7f49b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f02345640 1 -- start start 2024-03-30T13:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f02345640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1efc0cda60 0x7f1efc0b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f02345640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1efc140690 0x7f1efc159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f02345640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc140a70 0x7f1efc0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f02345640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc15e6f0 con 0x7f1efc140690 2024-03-30T13:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f02345640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1efc143e90 con 0x7f1efc140a70 2024-03-30T13:11:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f02345640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1efc15e870 con 0x7f1efc0cda60 2024-03-30T13:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49bdffb640 1 -- 172.21.15.43:0/1397302194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8015070 con 0x7f49c815a6a0 2024-03-30T13:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c8157e70 con 0x7f49c815a6a0 2024-03-30T13:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c8158430 con 0x7f49c815a6a0 2024-03-30T13:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49bdffb640 1 -- 172.21.15.43:0/1397302194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49b8004220 con 0x7f49c815a6a0 2024-03-30T13:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f49bdffb640 1 -- 172.21.15.43:0/1397302194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8003620 con 0x7f49c815a6a0 2024-03-30T13:11:23.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f01b44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc140a70 0x7f1efc0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f00b42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1efc140690 0x7f1efc159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f01b44640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc140a70 0x7f1efc0aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:47068/0 (socket says 172.21.15.43:47068) 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f01b44640 1 -- 172.21.15.43:0/2832076192 learned_addr learned my addr 172.21.15.43:0/2832076192 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.899+0000 7f1f01343640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1efc0cda60 0x7f1efc0b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f1f00b42640 1 -- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1efc0cda60 msgr2=0x7f1efc0b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f1f00b42640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1efc0cda60 0x7f1efc0b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f1f00b42640 1 -- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc140a70 msgr2=0x7f1efc0aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f1f00b42640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc140a70 0x7f1efc0aff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f1f00b42640 1 -- 172.21.15.43:0/2832076192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1efc15a040 con 0x7f1efc140690 2024-03-30T13:11:23.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac0052a0 con 0x7f49c815a6a0 2024-03-30T13:11:23.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f1f00b42640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1efc140690 0x7f1efc159560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00c7f0 tx=0x7f1eec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7fb4737fe640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb464003bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb474068d20 tx=0x7fb474098310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf9ad7d23f5cb57 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f49bdffb640 1 -- 172.21.15.43:0/1397302194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f49b80037c0 con 0x7f49c815a6a0 2024-03-30T13:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f49bdffb640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49b4068100 0x7f49b406a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f49bdffb640 1 -- 172.21.15.43:0/1397302194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f49b8086340 con 0x7f49c815a6a0 2024-03-30T13:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f49cd048640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49b4068100 0x7f49b406a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.903+0000 7f49cd048640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49b4068100 0x7f49b406a5c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b2000 tx=0x7f49c00a9070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.911+0000 7fc26effd640 1 -- 172.21.15.43:0/2801571992 <== mon.2 v2:172.21.15.113:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc27806dfc0 con 0x7fc278157bf0 2024-03-30T13:11:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.911+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4740687f0 con 0x7fb4640037b0 2024-03-30T13:11:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.911+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47406d310 con 0x7fb4640037b0 2024-03-30T13:11:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.911+0000 7fb4727fc640 1 -- 172.21.15.43:0/2516984881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47406b8a0 con 0x7fb4640037b0 2024-03-30T13:11:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.911+0000 7f49bdffb640 1 -- 172.21.15.43:0/1397302194 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f49b80560c0 con 0x7f49c815a6a0 2024-03-30T13:11:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 -- 172.21.15.43:0/2516984881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 msgr2=0x7fb464003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 --2- 172.21.15.43:0/2516984881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb464003bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb474068d20 tx=0x7fb474098310 comp rx=0 tx=0).stop 2024-03-30T13:11:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb473fff640 1 -- 172.21.15.43:0/2516984881 reap_dead start 2024-03-30T13:11:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 -- 172.21.15.43:0/2516984881 shutdown_connections 2024-03-30T13:11:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 -- 172.21.15.43:0/2516984881 >> 172.21.15.43:0/2516984881 conn(0x7fb46c009fc0 msgr2=0x7fb46c00a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:23.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 -- 172.21.15.43:0/2516984881 shutdown_connections 2024-03-30T13:11:23.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 -- 172.21.15.43:0/2516984881 wait complete. 2024-03-30T13:11:23.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 Processor -- start 2024-03-30T13:11:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7fb478cf1640 1 -- start start 2024-03-30T13:11:23.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb478cf1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb46c189260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb46c189260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb46c189260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35080/0 (socket says 172.21.15.43:35080) 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 -- 172.21.15.43:0/1457701414 learned_addr learned my addr 172.21.15.43:0/1457701414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c04e3a0 0x7fb46c0dac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c0de1d0 0x7fb46c17ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c0076d0 con 0x7fb4640037b0 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c038830 con 0x7fb46c04e3a0 2024-03-30T13:11:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb472ffd640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c04e3a0 0x7fb46c0dac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb46c0db990 con 0x7fb46c0de1d0 2024-03-30T13:11:23.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb473fff640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c0de1d0 0x7fb46c17ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 -- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c0de1d0 msgr2=0x7fb46c17ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c0de1d0 0x7fb46c17ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 -- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c04e3a0 msgr2=0x7fb46c0dac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c04e3a0 0x7fb46c0dac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c0dbc70 con 0x7fb4640037b0 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7fb4737fe640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb46c189260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb47406a460 tx=0x7fb464004830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7f1ef1ffb640 1 -- 172.21.15.43:0/2832076192 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec015070 con 0x7f1efc140690 2024-03-30T13:11:23.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1efc0b3cb0 con 0x7f1efc140690 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1efc0b4270 con 0x7f1efc140690 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.915+0000 7f1ef1ffb640 1 -- 172.21.15.43:0/2832076192 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eec007a30 con 0x7f1efc140690 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7f1ef1ffb640 1 -- 172.21.15.43:0/2832076192 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eec007e40 con 0x7f1efc140690 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1edc0052a0 con 0x7f1efc140690 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7f1ef1ffb640 1 -- 172.21.15.43:0/2832076192 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1eec018070 con 0x7f1efc140690 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7f1ef1ffb640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1ee0068090 0x7f1ee006a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.919+0000 7f1ef1ffb640 1 -- 172.21.15.43:0/2832076192 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1eec086a60 con 0x7f1efc140690 2024-03-30T13:11:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.923+0000 7f1f01343640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1ee0068090 0x7f1ee006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.923+0000 7f1f01343640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1ee0068090 0x7f1ee006a550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009890 tx=0x7f1ef8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.935+0000 7f1ef1ffb640 1 -- 172.21.15.43:0/2832076192 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1eec0568f0 con 0x7f1efc140690 2024-03-30T13:11:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.935+0000 7fb45ffff640 1 -- 172.21.15.43:0/1457701414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47406d4c0 con 0x7fb4640037b0 2024-03-30T13:11:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.935+0000 7fb45ffff640 1 -- 172.21.15.43:0/1457701414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4740696a0 con 0x7fb4640037b0 2024-03-30T13:11:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.935+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c0dbf50 con 0x7fb4640037b0 2024-03-30T13:11:23.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.935+0000 7fb45ffff640 1 -- 172.21.15.43:0/1457701414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb474128a70 con 0x7fb4640037b0 2024-03-30T13:11:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.935+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c0de650 con 0x7fb4640037b0 2024-03-30T13:11:23.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.935+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb46c0053a0 con 0x7fb4640037b0 2024-03-30T13:11:23.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.939+0000 7fb45ffff640 1 -- 172.21.15.43:0/1457701414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb474128c10 con 0x7fb4640037b0 2024-03-30T13:11:23.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.939+0000 7fb45ffff640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb458068090 0x7fb45806a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.939+0000 7fb45ffff640 1 -- 172.21.15.43:0/1457701414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb47419b580 con 0x7fb4640037b0 2024-03-30T13:11:23.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.939+0000 7fb472ffd640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb458068090 0x7fb45806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.943+0000 7fb472ffd640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb458068090 0x7fb45806a550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb46c17c500 tx=0x7fb468006ef0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:23.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.951+0000 7fb45ffff640 1 -- 172.21.15.43:0/1457701414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb474136030 con 0x7fb4640037b0 2024-03-30T13:11:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa5371a9640 1 Processor -- start 2024-03-30T13:11:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa5371a9640 1 -- start start 2024-03-30T13:11:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa5371a9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa5300b50c0 0x7fa530072c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa5371a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa53006d670 0x7fa53006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa5371a9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa53006db70 con 0x7fa53016b460 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa5371a9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa530071eb0 con 0x7fa5300b50c0 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa5371a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa530072190 con 0x7fa53006d670 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa535746640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa53016b460 0x7fa53016b8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59716/0 (socket says 172.21.15.43:59716) 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa535746640 1 -- 172.21.15.43:0/3603941649 learned_addr learned my addr 172.21.15.43:0/3603941649 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa52ffff640 1 --2- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa53006d670 0x7fa53006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.975+0000 7fa534f45640 1 --2- 172.21.15.43:0/3603941649 >> v2:172.21.15.113:3300/0 conn(0x7fa5300b50c0 0x7fa530072c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.983+0000 7fa534f45640 1 -- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa53006d670 msgr2=0x7fa53006dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.983+0000 7fa534f45640 1 --2- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa53006d670 0x7fa53006dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.983+0000 7fa534f45640 1 -- 172.21.15.43:0/3603941649 >> v1:172.21.15.43:6789/0 conn(0x7fa53016b460 legacy=0x7fa53016b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.983+0000 7fa534f45640 1 -- 172.21.15.43:0/3603941649 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa530072470 con 0x7fa5300b50c0 2024-03-30T13:11:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa534f45640 1 --2- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5300b50c0 0x7fa530072c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa524006200 tx=0x7fa524024cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a29e23c36e05d35c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa52f7fe640 1 -- 172.21.15.43:0/3603941649 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5240adae0 con 0x7fa5300b50c0 2024-03-30T13:11:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa52f7fe640 1 -- 172.21.15.43:0/3603941649 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5240adc80 con 0x7fa5300b50c0 2024-03-30T13:11:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa52f7fe640 1 -- 172.21.15.43:0/3603941649 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524004250 con 0x7fa5300b50c0 2024-03-30T13:11:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 -- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5300b50c0 msgr2=0x7fa530072c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5300b50c0 0x7fa530072c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa524006200 tx=0x7fa524024cb0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 -- 172.21.15.43:0/3603941649 shutdown_connections 2024-03-30T13:11:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa53006d670 0x7fa53006dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3603941649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa5300b50c0 0x7fa530072c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 -- 172.21.15.43:0/3603941649 >> 172.21.15.43:0/3603941649 conn(0x7fa53006b660 msgr2=0x7fa53006ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 -- 172.21.15.43:0/3603941649 shutdown_connections 2024-03-30T13:11:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:23.999+0000 7fa5371a9640 1 -- 172.21.15.43:0/3603941649 wait complete. 2024-03-30T13:11:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 Processor -- start 2024-03-30T13:11:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 -- start start 2024-03-30T13:11:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53006d670 0x7fa530201a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5300b50c0 0x7fa5300a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53016b460 0x7fa53015b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa534f45640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53006d670 0x7fa530201a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa534f45640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53006d670 0x7fa530201a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:47092/0 (socket says 172.21.15.43:47092) 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5300b50c0 0x7fa5300a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53016b460 0x7fa53015b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53016b460 0x7fa53015b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35096/0 (socket says 172.21.15.43:35096) 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa52ffff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5300b50c0 0x7fa5300a9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37554/0 (socket says 172.21.15.43:37554) 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa534f45640 1 -- 172.21.15.43:0/3431525073 learned_addr learned my addr 172.21.15.43:0/3431525073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa53006e6b0 con 0x7fa53016b460 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa53006e530 con 0x7fa53006d670 2024-03-30T13:11:24.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa5300b36e0 con 0x7fa5300b50c0 2024-03-30T13:11:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 -- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5300b50c0 msgr2=0x7fa5300a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5300b50c0 0x7fa5300a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 -- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53006d670 msgr2=0x7fa530201a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53006d670 0x7fa530201a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5300aba40 con 0x7fa53016b460 2024-03-30T13:11:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.003+0000 7fa535746640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53016b460 0x7fa53015b910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa528001610 tx=0x7fa52800e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.011+0000 7fa52d7fa640 1 -- 172.21.15.43:0/3431525073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528019070 con 0x7fa53016b460 2024-03-30T13:11:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.011+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5302022f0 con 0x7fa53016b460 2024-03-30T13:11:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.011+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5302028b0 con 0x7fa53016b460 2024-03-30T13:11:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.011+0000 7fa52d7fa640 1 -- 172.21.15.43:0/3431525073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa528009480 con 0x7fa53016b460 2024-03-30T13:11:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.011+0000 7fa52d7fa640 1 -- 172.21.15.43:0/3431525073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528009870 con 0x7fa53016b460 2024-03-30T13:11:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.023+0000 7fa52d7fa640 1 -- 172.21.15.43:0/3431525073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa52801c070 con 0x7fa53016b460 2024-03-30T13:11:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.023+0000 7fa52d7fa640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa51c068090 0x7fa51c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.023+0000 7fa52d7fa640 1 -- 172.21.15.43:0/3431525073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa528088db0 con 0x7fa53016b460 2024-03-30T13:11:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.023+0000 7fa534f45640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa51c068090 0x7fa51c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.027+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5100052a0 con 0x7fa53016b460 2024-03-30T13:11:24.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.035+0000 7fa534f45640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa51c068090 0x7fa51c06a550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa524024cf0 tx=0x7fa5240af070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:24.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.035+0000 7fa52d7fa640 1 -- 172.21.15.43:0/3431525073 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa5300aba40 con 0x7fa53016b460 2024-03-30T13:11:24.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.043+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f51080cd770 con 0x7f510806c770 2024-03-30T13:11:24.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.043+0000 7f5104ff9640 1 -- 172.21.15.43:0/3829773323 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f50f80f14f0 con 0x7f510806c770 2024-03-30T13:11:24.064 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50f4068090 msgr2=0x7f50f406a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50f4068090 0x7f50f406a550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5108150da0 tx=0x7f50fc008040 comp rx=0 tx=0).stop 2024-03-30T13:11:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f510806c770 msgr2=0x7f510814ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f510806c770 0x7f510814ef40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f50f80ad040 tx=0x7f50f801acb0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 shutdown_connections 2024-03-30T13:11:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50f4068090 0x7f50f406a550 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f51080b4530 0x7f51081493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f510806f9e0 0x7f510814f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 --2- 172.21.15.43:0/3829773323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f510806c770 0x7f510814ef40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 >> 172.21.15.43:0/3829773323 conn(0x7f510806e410 msgr2=0x7f5108070510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 shutdown_connections 2024-03-30T13:11:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.079+0000 7f510e3bf640 1 -- 172.21.15.43:0/3829773323 wait complete. 2024-03-30T13:11:24.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.115+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f318415ad90 con 0x7f3184071c60 2024-03-30T13:11:24.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.2 2024-03-30T13:11:24.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.115+0000 7f3180ff9640 1 -- 172.21.15.43:0/2272775579 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f317401a5d0 con 0x7f3184071c60 2024-03-30T13:11:24.123 INFO:teuthology.orchestra.run.smithi043.stdout:64424509451 2024-03-30T13:11:24.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.115+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f316c068120 msgr2=0x7f316c06a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f316c068120 0x7f316c06a5e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f31840b0130 tx=0x7f317c009750 comp rx=0 tx=0).stop 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3184071c60 msgr2=0x7f31841377e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3184071c60 0x7f31841377e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f31740b0000 tx=0x7f317401d6f0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 shutdown_connections 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f316c068120 0x7f316c06a5e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f318415e4d0 0x7f318413f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3184142f80 0x7f318413a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 --2- 172.21.15.43:0/2272775579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3184071c60 0x7f31841377e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 >> 172.21.15.43:0/2272775579 conn(0x7f318406b660 msgr2=0x7f318406df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 shutdown_connections 2024-03-30T13:11:24.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.119+0000 7f318a848640 1 -- 172.21.15.43:0/2272775579 wait complete. 2024-03-30T13:11:24.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.1 2024-03-30T13:11:24.158 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.215+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb46c0393e0 con 0x7fb4640037b0 2024-03-30T13:11:24.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.215+0000 7fb45ffff640 1 -- 172.21.15.43:0/1457701414 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb4740921e0 con 0x7fb4640037b0 2024-03-30T13:11:24.223 INFO:teuthology.orchestra.run.smithi043.stdout:64424509451 2024-03-30T13:11:24.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb458068090 msgr2=0x7fb45806a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb458068090 0x7fb45806a550 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb46c17c500 tx=0x7fb468006ef0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 msgr2=0x7fb46c189260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb46c189260 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb47406a460 tx=0x7fb464004830 comp rx=0 tx=0).stop 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 shutdown_connections 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb458068090 0x7fb45806a550 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb46c0de1d0 0x7fb46c17ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb46c04e3a0 0x7fb46c0dac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 --2- 172.21.15.43:0/1457701414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4640037b0 0x7fb46c189260 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 >> 172.21.15.43:0/1457701414 conn(0x7fb46c009fc0 msgr2=0x7fb46c038bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 shutdown_connections 2024-03-30T13:11:24.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.219+0000 7fb478cf1640 1 -- 172.21.15.43:0/1457701414 wait complete. 2024-03-30T13:11:24.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.7 2024-03-30T13:11:24.242 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:24.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.235+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb9e4005590 con 0x7fb9fc007290 2024-03-30T13:11:24.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fb9effff640 1 -- 172.21.15.43:0/2463612840 <== mon.1 v2:172.21.15.113:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb9f805acb0 con 0x7fb9fc007290 2024-03-30T13:11:24.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f49ac005590 con 0x7f49c815a6a0 2024-03-30T13:11:24.245 INFO:teuthology.orchestra.run.smithi043.stdout:64424509452 2024-03-30T13:11:24.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7f49bdffb640 1 -- 172.21.15.43:0/1397302194 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f49b805a3c0 con 0x7f49c815a6a0 2024-03-30T13:11:24.246 INFO:teuthology.orchestra.run.smithi043.stdout:64424509451 2024-03-30T13:11:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9f4068250 msgr2=0x7fb9f406a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9f4068250 0x7fb9f406a710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0fc190 tx=0x7fb9f0001ce0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 msgr2=0x7fb9fc0f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc0f7c40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800eb00 tx=0x7fb9f800ebf0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 shutdown_connections 2024-03-30T13:11:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9f4068250 0x7fb9f406a710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9fc100e30 0x7fb9fc0f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9fc04f3e0 0x7fb9fc196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 --2- 172.21.15.43:0/2463612840 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9fc007290 0x7fb9fc0f7c40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 >> 172.21.15.43:0/2463612840 conn(0x7fb9fc009fc0 msgr2=0x7fb9fc0f4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 shutdown_connections 2024-03-30T13:11:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.239+0000 7fba09fb6640 1 -- 172.21.15.43:0/2463612840 wait complete. 2024-03-30T13:11:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49b4068100 msgr2=0x7f49b406a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49b4068100 0x7f49b406a5c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f49c00b2000 tx=0x7f49c00a9070 comp rx=0 tx=0).stop 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c815a6a0 msgr2=0x7f49c81569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c815a6a0 0x7f49c81569f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f49c815ef70 tx=0x7f49b800c8e0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 shutdown_connections 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49b4068100 0x7f49b406a5c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c815a6a0 0x7f49c81569f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c815a2c0 0x7f49c8141620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 --2- 172.21.15.43:0/1397302194 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c8072f20 0x7f49c8152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 >> 172.21.15.43:0/1397302194 conn(0x7f49c806c570 msgr2=0x7f49c80cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 shutdown_connections 2024-03-30T13:11:24.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.243+0000 7f49cf2ac640 1 -- 172.21.15.43:0/1397302194 wait complete. 2024-03-30T13:11:24.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.0 2024-03-30T13:11:24.274 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:24.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.3 2024-03-30T13:11:24.275 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc278068670 con 0x7fc278157bf0 2024-03-30T13:11:24.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc26effd640 1 -- 172.21.15.43:0/2801571992 <== mon.2 v2:172.21.15.113:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc274058720 con 0x7fc278157bf0 2024-03-30T13:11:24.286 INFO:teuthology.orchestra.run.smithi043.stdout:55834574859 2024-03-30T13:11:24.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc25c067ff0 msgr2=0x7fc25c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc25c067ff0 0x7fc25c06a4b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc268009980 tx=0x7fc2680090e0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc278157bf0 msgr2=0x7fc27806d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc278157bf0 0x7fc27806d670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e80 tx=0x7fc27400c430 comp rx=0 tx=0).stop 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 shutdown_connections 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc25c067ff0 0x7fc25c06a4b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc27806dbb0 0x7fc278139ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc278157bf0 0x7fc27806d670 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 --2- 172.21.15.43:0/2801571992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc278073300 0x7fc27806e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 >> 172.21.15.43:0/2801571992 conn(0x7fc27806c5a0 msgr2=0x7fc2780cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 shutdown_connections 2024-03-30T13:11:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7fc27f06e640 1 -- 172.21.15.43:0/2801571992 wait complete. 2024-03-30T13:11:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.279+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1edc005860 con 0x7f1efc140690 2024-03-30T13:11:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1ef1ffb640 1 -- 172.21.15.43:0/2832076192 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1eec05abf0 con 0x7f1efc140690 2024-03-30T13:11:24.291 INFO:teuthology.orchestra.run.smithi043.stdout:64424509451 2024-03-30T13:11:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1ee0068090 msgr2=0x7f1ee006a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1ee0068090 0x7f1ee006a550 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009890 tx=0x7f1ef8008040 comp rx=0 tx=0).stop 2024-03-30T13:11:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1efc140690 msgr2=0x7f1efc159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1efc140690 0x7f1efc159560 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00c7f0 tx=0x7f1eec00ccb0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 shutdown_connections 2024-03-30T13:11:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1ee0068090 0x7f1ee006a550 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1efc140a70 0x7f1efc0aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1efc140690 0x7f1efc159560 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 --2- 172.21.15.43:0/2832076192 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1efc0cda60 0x7f1efc0b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 >> 172.21.15.43:0/2832076192 conn(0x7f1efc0afb60 msgr2=0x7f1efc071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 shutdown_connections 2024-03-30T13:11:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.283+0000 7f1f02345640 1 -- 172.21.15.43:0/2832076192 wait complete. 2024-03-30T13:11:24.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-03-30T13:11:24.306 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:24.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509451 for osd.4 2024-03-30T13:11:24.316 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:24.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.327+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa510005590 con 0x7fa53016b460 2024-03-30T13:11:24.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.327+0000 7fa52d7fa640 1 -- 172.21.15.43:0/3431525073 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa528058bb0 con 0x7fa53016b460 2024-03-30T13:11:24.335 INFO:teuthology.orchestra.run.smithi043.stdout:64424509450 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa51c068090 msgr2=0x7fa51c06a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa51c068090 0x7fa51c06a550 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa524024cf0 tx=0x7fa5240af070 comp rx=0 tx=0).stop 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53016b460 msgr2=0x7fa53015b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53016b460 0x7fa53015b910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa528001610 tx=0x7fa52800e870 comp rx=0 tx=0).stop 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 shutdown_connections 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa51c068090 0x7fa51c06a550 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa53016b460 0x7fa53015b910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa5300b50c0 0x7fa5300a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 --2- 172.21.15.43:0/3431525073 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa53006d670 0x7fa530201a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:24.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 >> 172.21.15.43:0/3431525073 conn(0x7fa53006b660 msgr2=0x7fa53008b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:24.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 shutdown_connections 2024-03-30T13:11:24.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:24.331+0000 7fa5371a9640 1 -- 172.21.15.43:0/3431525073 wait complete. 2024-03-30T13:11:24.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509450 for osd.6 2024-03-30T13:11:24.352 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:25.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-30T13:11:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e26de0640 1 Processor -- start 2024-03-30T13:11:25.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e26de0640 1 -- start start 2024-03-30T13:11:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e26de0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e2015c410 0x7f5e2015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e26de0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 0x7f5e201609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e26de0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e200af590 con 0x7f5e200b5e30 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e26de0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5e20160ee0 con 0x7f5e2015c410 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e26de0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5e200cc900 con 0x7f5e200b15c0 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e24b7c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e2015c410 0x7f5e2015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e1ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5e200b5e30 0x7f5e200b0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59726/0 (socket says 172.21.15.43:59726) 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e1ffff640 1 -- 172.21.15.43:0/2301250452 learned_addr learned my addr 172.21.15.43:0/2301250452 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e2537d640 1 --2- 172.21.15.43:0/2301250452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 0x7f5e201609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e1f7fe640 1 -- 172.21.15.43:0/2301250452 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1193960080 0 0) 0x7f5e200af590 con 0x7f5e200b5e30 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e1f7fe640 1 -- 172.21.15.43:0/2301250452 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e0c003530 con 0x7f5e200b5e30 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e24b7c640 1 -- 172.21.15.43:0/2301250452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 msgr2=0x7f5e201609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e24b7c640 1 --2- 172.21.15.43:0/2301250452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 0x7f5e201609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e24b7c640 1 -- 172.21.15.43:0/2301250452 >> v1:172.21.15.43:6789/0 conn(0x7f5e200b5e30 legacy=0x7f5e200b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.235+0000 7f5e24b7c640 1 -- 172.21.15.43:0/2301250452 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2006cdf0 con 0x7f5e2015c410 2024-03-30T13:11:25.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e24b7c640 1 --2- 172.21.15.43:0/2301250452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e2015c7f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f762841bec1853fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e1f7fe640 1 -- 172.21.15.43:0/2301250452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100b2070 con 0x7f5e2015c410 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e1f7fe640 1 -- 172.21.15.43:0/2301250452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e100a5830 con 0x7f5e2015c410 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e1f7fe640 1 -- 172.21.15.43:0/2301250452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e100a5c00 con 0x7f5e2015c410 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 -- 172.21.15.43:0/2301250452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 msgr2=0x7f5e2015c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 --2- 172.21.15.43:0/2301250452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e2015c7f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009d90 tx=0x7f5e1001c970 comp rx=0 tx=0).stop 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 -- 172.21.15.43:0/2301250452 shutdown_connections 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 --2- 172.21.15.43:0/2301250452 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 0x7f5e201609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 --2- 172.21.15.43:0/2301250452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e2015c7f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 -- 172.21.15.43:0/2301250452 >> 172.21.15.43:0/2301250452 conn(0x7f5e200b5060 msgr2=0x7f5e200b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 -- 172.21.15.43:0/2301250452 shutdown_connections 2024-03-30T13:11:25.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 -- 172.21.15.43:0/2301250452 wait complete. 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 Processor -- start 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.239+0000 7f5e26de0640 1 -- start start 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 0x7f5e20158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e200b5e30 0x7f5e2009a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e20150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2006c9c0 con 0x7f5e2015c410 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2006c840 con 0x7f5e200b5e30 2024-03-30T13:11:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5e2006cb40 con 0x7f5e200b15c0 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e20150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e20150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35118/0 (socket says 172.21.15.43:35118) 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 -- 172.21.15.43:0/1982701435 learned_addr learned my addr 172.21.15.43:0/1982701435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 -- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 msgr2=0x7f5e20158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 0x7f5e20158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 -- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e200b5e30 msgr2=0x7f5e2009a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e200b5e30 0x7f5e2009a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 -- 172.21.15.43:0/1982701435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2015e3c0 con 0x7f5e2015c410 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e2537d640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e20150430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800d800 tx=0x7f5e1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e1d7fa640 1 -- 172.21.15.43:0/1982701435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800b6c0 con 0x7f5e2015c410 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e1d7fa640 1 -- 172.21.15.43:0/1982701435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e18019070 con 0x7f5e2015c410 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e1d7fa640 1 -- 172.21.15.43:0/1982701435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1800baa0 con 0x7f5e2015c410 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2009a830 con 0x7f5e2015c410 2024-03-30T13:11:25.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2015e630 con 0x7f5e2015c410 2024-03-30T13:11:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e20158fa0 con 0x7f5e2015c410 2024-03-30T13:11:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e1d7fa640 1 -- 172.21.15.43:0/1982701435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5e1800bc40 con 0x7f5e2015c410 2024-03-30T13:11:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e1d7fa640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5e00067fc0 0x7f5e0006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e1d7fa640 1 -- 172.21.15.43:0/1982701435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5e18086c80 con 0x7f5e2015c410 2024-03-30T13:11:25.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.243+0000 7f5e24b7c640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5e00067fc0 0x7f5e0006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.247+0000 7f5e24b7c640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5e00067fc0 0x7f5e0006a480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e100af000 tx=0x7f5e10004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.247+0000 7f5e1d7fa640 1 -- 172.21.15.43:0/1982701435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5e20158fa0 con 0x7f5e2015c410 2024-03-30T13:11:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5e2015e800 con 0x7f5e2015c410 2024-03-30T13:11:25.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e1d7fa640 1 -- 172.21.15.43:0/1982701435 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5e18057180 con 0x7f5e2015c410 2024-03-30T13:11:25.414 INFO:teuthology.orchestra.run.smithi043.stdout:64424509452 2024-03-30T13:11:25.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5e00067fc0 msgr2=0x7f5e0006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5e00067fc0 0x7f5e0006a480 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e100af000 tx=0x7f5e10004ac0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 msgr2=0x7f5e20150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e20150430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e1800d800 tx=0x7f5e1800dcc0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 shutdown_connections 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5e00067fc0 0x7f5e0006a480 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e2015c410 0x7f5e20150430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5e200b5e30 0x7f5e2009a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 --2- 172.21.15.43:0/1982701435 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5e200b15c0 0x7f5e20158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 >> 172.21.15.43:0/1982701435 conn(0x7f5e200b5060 msgr2=0x7f5e20159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.407+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 shutdown_connections 2024-03-30T13:11:25.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.411+0000 7f5e26de0640 1 -- 172.21.15.43:0/1982701435 wait complete. 2024-03-30T13:11:25.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509452 for osd.2 2024-03-30T13:11:25.432 DEBUG:teuthology.parallel:result is None 2024-03-30T13:11:25.432 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-30T13:11:25.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.547+0000 7fc4f87c8640 1 Processor -- start 2024-03-30T13:11:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.547+0000 7fc4f87c8640 1 -- start start 2024-03-30T13:11:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f87c8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4f006f950 0x7fc4f00b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f87c8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 0x7fc4f006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f87c8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4f00b4040 con 0x7fc4f00b3720 2024-03-30T13:11:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f87c8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc4f006dc30 con 0x7fc4f006f950 2024-03-30T13:11:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f87c8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f006def0 con 0x7fc4f006b660 2024-03-30T13:11:25.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5d63640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4f006f950 0x7fc4f00b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f6564640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc4f00b3720 0x7fc4f015f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59742/0 (socket says 172.21.15.43:59742) 2024-03-30T13:11:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f6564640 1 -- 172.21.15.43:0/227711619 learned_addr learned my addr 172.21.15.43:0/227711619 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f6d65640 1 --2- 172.21.15.43:0/227711619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 0x7fc4f006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5d63640 1 -- 172.21.15.43:0/227711619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 msgr2=0x7fc4f006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5d63640 1 --2- 172.21.15.43:0/227711619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 0x7fc4f006bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5d63640 1 -- 172.21.15.43:0/227711619 >> v1:172.21.15.43:6789/0 conn(0x7fc4f00b3720 legacy=0x7fc4f015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5d63640 1 -- 172.21.15.43:0/227711619 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f006e1d0 con 0x7fc4f006f950 2024-03-30T13:11:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5d63640 1 --2- 172.21.15.43:0/227711619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f006f950 0x7fc4f00b3b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00097c0 tx=0x7fc4e001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8547c8d8c2892c56 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5562640 1 -- 172.21.15.43:0/227711619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00a7070 con 0x7fc4f006f950 2024-03-30T13:11:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5562640 1 -- 172.21.15.43:0/227711619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e00b0070 con 0x7fc4f006f950 2024-03-30T13:11:25.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.551+0000 7fc4f5562640 1 -- 172.21.15.43:0/227711619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00ab890 con 0x7fc4f006f950 2024-03-30T13:11:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- 172.21.15.43:0/227711619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f006f950 msgr2=0x7fc4f00b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/227711619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f006f950 0x7fc4f00b3b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00097c0 tx=0x7fc4e001c470 comp rx=0 tx=0).stop 2024-03-30T13:11:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- 172.21.15.43:0/227711619 shutdown_connections 2024-03-30T13:11:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/227711619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 0x7fc4f006bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/227711619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f006f950 0x7fc4f00b3b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- 172.21.15.43:0/227711619 >> 172.21.15.43:0/227711619 conn(0x7fc4f00b2b30 msgr2=0x7fc4f00d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- 172.21.15.43:0/227711619 shutdown_connections 2024-03-30T13:11:25.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- 172.21.15.43:0/227711619 wait complete. 2024-03-30T13:11:25.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 Processor -- start 2024-03-30T13:11:25.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- start start 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 0x7fc4f0143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f006f950 0x7fc4f0152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f00b3720 0x7fc4f015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00b01c0 con 0x7fc4f00b3720 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f00b0040 con 0x7fc4f006f950 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f006c460 con 0x7fc4f006b660 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f00b3720 0x7fc4f015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f00b3720 0x7fc4f015ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35144/0 (socket says 172.21.15.43:35144) 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 -- 172.21.15.43:0/900178905 learned_addr learned my addr 172.21.15.43:0/900178905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f5d63640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f006f950 0x7fc4f0152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 -- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 msgr2=0x7fc4f0143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 0x7fc4f0143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 -- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f006f950 msgr2=0x7fc4f0152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f006f950 0x7fc4f0152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 -- 172.21.15.43:0/900178905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f01436c0 con 0x7fc4f00b3720 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f6d65640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f00b3720 0x7fc4f015ae40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007e40 tx=0x7fc4e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4e6ffd640 1 -- 172.21.15.43:0/900178905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8014620 con 0x7fc4f00b3720 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4e6ffd640 1 -- 172.21.15.43:0/900178905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e8019070 con 0x7fc4f00b3720 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f0161650 con 0x7fc4f00b3720 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4e6ffd640 1 -- 172.21.15.43:0/900178905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8014a20 con 0x7fc4f00b3720 2024-03-30T13:11:25.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.555+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f0153910 con 0x7fc4f00b3720 2024-03-30T13:11:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.559+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f00cd030 con 0x7fc4f00b3720 2024-03-30T13:11:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.559+0000 7fc4e6ffd640 1 -- 172.21.15.43:0/900178905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc4e8014bc0 con 0x7fc4f00b3720 2024-03-30T13:11:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.559+0000 7fc4e6ffd640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc4d4067fc0 0x7fc4d406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.559+0000 7fc4e6ffd640 1 -- 172.21.15.43:0/900178905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc4e8087200 con 0x7fc4f00b3720 2024-03-30T13:11:25.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.559+0000 7fc4f6564640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc4d4067fc0 0x7fc4d406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.559+0000 7fc4f6564640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc4d4067fc0 0x7fc4d406a480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec000f70 tx=0x7fc4ec007a90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.563+0000 7fc4e6ffd640 1 -- 172.21.15.43:0/900178905 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc4f01436c0 con 0x7fc4f00b3720 2024-03-30T13:11:25.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.715+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4f0068890 con 0x7fc4d4067fc0 2024-03-30T13:11:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.739+0000 7fc4e6ffd640 1 -- 172.21.15.43:0/900178905 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332529 (secure 0 0 0) 0x7fc4f0068890 con 0x7fc4d4067fc0 2024-03-30T13:11:25.749 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc4d4067fc0 msgr2=0x7fc4d406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc4d4067fc0 0x7fc4d406a480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec000f70 tx=0x7fc4ec007a90 comp rx=0 tx=0).stop 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f00b3720 msgr2=0x7fc4f015ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f00b3720 0x7fc4f015ae40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8007e40 tx=0x7fc4e800e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 shutdown_connections 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc4d4067fc0 0x7fc4d406a480 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4f00b3720 0x7fc4f015ae40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4f006f950 0x7fc4f0152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 --2- 172.21.15.43:0/900178905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4f006b660 0x7fc4f0143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 >> 172.21.15.43:0/900178905 conn(0x7fc4f00b2b30 msgr2=0x7fc4f006eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 shutdown_connections 2024-03-30T13:11:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.743+0000 7fc4f87c8640 1 -- 172.21.15.43:0/900178905 wait complete. 2024-03-30T13:11:25.766 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-03-30T13:11:23.846161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236252,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738468,"statfs":{"total":773094113280,"available":772852191232,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005326"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857033+0000","last_change":"2024-03-30T13:10:54.944419+0000","last_active":"2024-03-30T13:11:01.857033+0000","last_peered":"2024-03-30T13:11:01.857033+0000","last_clean":"2024-03-30T13:11:01.857033+0000","last_became_active":"2024-03-30T13:10:54.943176+0000","last_became_peered":"2024-03-30T13:10:54.943176+0000","last_unstale":"2024-03-30T13:11:01.857033+0000","last_undegraded":"2024-03-30T13:11:01.857033+0000","last_fullsized":"2024-03-30T13:11:01.857033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983961+0000","last_change":"2024-03-30T13:10:55.957609+0000","last_active":"2024-03-30T13:11:00.983961+0000","last_peered":"2024-03-30T13:11:00.983961+0000","last_clean":"2024-03-30T13:11:00.983961+0000","last_became_active":"2024-03-30T13:10:55.956870+0000","last_became_peered":"2024-03-30T13:10:55.956870+0000","last_unstale":"2024-03-30T13:11:00.983961+0000","last_undegraded":"2024-03-30T13:11:00.983961+0000","last_fullsized":"2024-03-30T13:11:00.983961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:04:28.315124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858409+0000","last_change":"2024-03-30T13:11:00.037601+0000","last_active":"2024-03-30T13:11:01.858409+0000","last_peered":"2024-03-30T13:11:01.858409+0000","last_clean":"2024-03-30T13:11:01.858409+0000","last_became_active":"2024-03-30T13:11:00.032050+0000","last_became_peered":"2024-03-30T13:11:00.032050+0000","last_unstale":"2024-03-30T13:11:01.858409+0000","last_undegraded":"2024-03-30T13:11:01.858409+0000","last_fullsized":"2024-03-30T13:11:01.858409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:47.447707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.984087+0000","last_change":"2024-03-30T13:10:54.940096+0000","last_active":"2024-03-30T13:11:00.984087+0000","last_peered":"2024-03-30T13:11:00.984087+0000","last_clean":"2024-03-30T13:11:00.984087+0000","last_became_active":"2024-03-30T13:10:54.939368+0000","last_became_peered":"2024-03-30T13:10:54.939368+0000","last_unstale":"2024-03-30T13:11:00.984087+0000","last_undegraded":"2024-03-30T13:11:00.984087+0000","last_fullsized":"2024-03-30T13:11:00.984087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:09:39.904016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981759+0000","last_change":"2024-03-30T13:10:55.979018+0000","last_active":"2024-03-30T13:11:00.981759+0000","last_peered":"2024-03-30T13:11:00.981759+0000","last_clean":"2024-03-30T13:11:00.981759+0000","last_became_active":"2024-03-30T13:10:55.975807+0000","last_became_peered":"2024-03-30T13:10:55.975807+0000","last_unstale":"2024-03-30T13:11:00.981759+0000","last_undegraded":"2024-03-30T13:11:00.981759+0000","last_fullsized":"2024-03-30T13:11:00.981759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:35:35.881832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857637+0000","last_change":"2024-03-30T13:11:00.020209+0000","last_active":"2024-03-30T13:11:01.857637+0000","last_peered":"2024-03-30T13:11:01.857637+0000","last_clean":"2024-03-30T13:11:01.857637+0000","last_became_active":"2024-03-30T13:11:00.019627+0000","last_became_peered":"2024-03-30T13:11:00.019627+0000","last_unstale":"2024-03-30T13:11:01.857637+0000","last_undegraded":"2024-03-30T13:11:01.857637+0000","last_fullsized":"2024-03-30T13:11:01.857637+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:31.595903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856064+0000","last_change":"2024-03-30T13:10:54.945298+0000","last_active":"2024-03-30T13:11:01.856064+0000","last_peered":"2024-03-30T13:11:01.856064+0000","last_clean":"2024-03-30T13:11:01.856064+0000","last_became_active":"2024-03-30T13:10:54.942725+0000","last_became_peered":"2024-03-30T13:10:54.942725+0000","last_unstale":"2024-03-30T13:11:01.856064+0000","last_undegraded":"2024-03-30T13:11:01.856064+0000","last_fullsized":"2024-03-30T13:11:01.856064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978049+0000","last_change":"2024-03-30T13:10:55.973824+0000","last_active":"2024-03-30T13:11:00.978049+0000","last_peered":"2024-03-30T13:11:00.978049+0000","last_clean":"2024-03-30T13:11:00.978049+0000","last_became_active":"2024-03-30T13:10:55.969728+0000","last_became_peered":"2024-03-30T13:10:55.969728+0000","last_unstale":"2024-03-30T13:11:00.978049+0000","last_undegraded":"2024-03-30T13:11:00.978049+0000","last_fullsized":"2024-03-30T13:11:00.978049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858055+0000","last_change":"2024-03-30T13:11:00.015698+0000","last_active":"2024-03-30T13:11:01.858055+0000","last_peered":"2024-03-30T13:11:01.858055+0000","last_clean":"2024-03-30T13:11:01.858055+0000","last_became_active":"2024-03-30T13:11:00.015263+0000","last_became_peered":"2024-03-30T13:11:00.015263+0000","last_unstale":"2024-03-30T13:11:01.858055+0000","last_undegraded":"2024-03-30T13:11:01.858055+0000","last_fullsized":"2024-03-30T13:11:01.858055+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856071+0000","last_change":"2024-03-30T13:10:54.941048+0000","last_active":"2024-03-30T13:11:01.856071+0000","last_peered":"2024-03-30T13:11:01.856071+0000","last_clean":"2024-03-30T13:11:01.856071+0000","last_became_active":"2024-03-30T13:10:54.940543+0000","last_became_peered":"2024-03-30T13:10:54.940543+0000","last_unstale":"2024-03-30T13:11:01.856071+0000","last_undegraded":"2024-03-30T13:11:01.856071+0000","last_fullsized":"2024-03-30T13:11:01.856071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855446+0000","last_change":"2024-03-30T13:10:55.965085+0000","last_active":"2024-03-30T13:11:01.855446+0000","last_peered":"2024-03-30T13:11:01.855446+0000","last_clean":"2024-03-30T13:11:01.855446+0000","last_became_active":"2024-03-30T13:10:55.964573+0000","last_became_peered":"2024-03-30T13:10:55.964573+0000","last_unstale":"2024-03-30T13:11:01.855446+0000","last_undegraded":"2024-03-30T13:11:01.855446+0000","last_fullsized":"2024-03-30T13:11:01.855446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:56:01.049552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857815+0000","last_change":"2024-03-30T13:11:00.020015+0000","last_active":"2024-03-30T13:11:01.857815+0000","last_peered":"2024-03-30T13:11:01.857815+0000","last_clean":"2024-03-30T13:11:01.857815+0000","last_became_active":"2024-03-30T13:11:00.019563+0000","last_became_peered":"2024-03-30T13:11:00.019563+0000","last_unstale":"2024-03-30T13:11:01.857815+0000","last_undegraded":"2024-03-30T13:11:01.857815+0000","last_fullsized":"2024-03-30T13:11:01.857815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:44.852627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854082+0000","last_change":"2024-03-30T13:10:55.972256+0000","last_active":"2024-03-30T13:11:01.854082+0000","last_peered":"2024-03-30T13:11:01.854082+0000","last_clean":"2024-03-30T13:11:01.854082+0000","last_became_active":"2024-03-30T13:10:55.971884+0000","last_became_peered":"2024-03-30T13:10:55.971884+0000","last_unstale":"2024-03-30T13:11:01.854082+0000","last_undegraded":"2024-03-30T13:11:01.854082+0000","last_fullsized":"2024-03-30T13:11:01.854082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:47:49.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854680+0000","last_change":"2024-03-30T13:10:54.932797+0000","last_active":"2024-03-30T13:11:01.854680+0000","last_peered":"2024-03-30T13:11:01.854680+0000","last_clean":"2024-03-30T13:11:01.854680+0000","last_became_active":"2024-03-30T13:10:54.932392+0000","last_became_peered":"2024-03-30T13:10:54.932392+0000","last_unstale":"2024-03-30T13:11:01.854680+0000","last_undegraded":"2024-03-30T13:11:01.854680+0000","last_fullsized":"2024-03-30T13:11:01.854680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855304+0000","last_change":"2024-03-30T13:11:00.005113+0000","last_active":"2024-03-30T13:11:01.855304+0000","last_peered":"2024-03-30T13:11:01.855304+0000","last_clean":"2024-03-30T13:11:01.855304+0000","last_became_active":"2024-03-30T13:11:00.004701+0000","last_became_peered":"2024-03-30T13:11:00.004701+0000","last_unstale":"2024-03-30T13:11:01.855304+0000","last_undegraded":"2024-03-30T13:11:01.855304+0000","last_fullsized":"2024-03-30T13:11:01.855304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:12:46.188048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859494+0000","last_change":"2024-03-30T13:10:55.993733+0000","last_active":"2024-03-30T13:11:01.859494+0000","last_peered":"2024-03-30T13:11:01.859494+0000","last_clean":"2024-03-30T13:11:01.859494+0000","last_became_active":"2024-03-30T13:10:55.993012+0000","last_became_peered":"2024-03-30T13:10:55.993012+0000","last_unstale":"2024-03-30T13:11:01.859494+0000","last_undegraded":"2024-03-30T13:11:01.859494+0000","last_fullsized":"2024-03-30T13:11:01.859494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980295+0000","last_change":"2024-03-30T13:10:54.947616+0000","last_active":"2024-03-30T13:11:00.980295+0000","last_peered":"2024-03-30T13:11:00.980295+0000","last_clean":"2024-03-30T13:11:00.980295+0000","last_became_active":"2024-03-30T13:10:54.946695+0000","last_became_peered":"2024-03-30T13:10:54.946695+0000","last_unstale":"2024-03-30T13:11:00.980295+0000","last_undegraded":"2024-03-30T13:11:00.980295+0000","last_fullsized":"2024-03-30T13:11:00.980295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855501+0000","last_change":"2024-03-30T13:11:00.019814+0000","last_active":"2024-03-30T13:11:01.855501+0000","last_peered":"2024-03-30T13:11:01.855501+0000","last_clean":"2024-03-30T13:11:01.855501+0000","last_became_active":"2024-03-30T13:11:00.019359+0000","last_became_peered":"2024-03-30T13:11:00.019359+0000","last_unstale":"2024-03-30T13:11:01.855501+0000","last_undegraded":"2024-03-30T13:11:01.855501+0000","last_fullsized":"2024-03-30T13:11:01.855501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:40:01.204781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855194+0000","last_change":"2024-03-30T13:10:56.004578+0000","last_active":"2024-03-30T13:11:01.855194+0000","last_peered":"2024-03-30T13:11:01.855194+0000","last_clean":"2024-03-30T13:11:01.855194+0000","last_became_active":"2024-03-30T13:10:56.003914+0000","last_became_peered":"2024-03-30T13:10:56.003914+0000","last_unstale":"2024-03-30T13:11:01.855194+0000","last_undegraded":"2024-03-30T13:11:01.855194+0000","last_fullsized":"2024-03-30T13:11:01.855194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856085+0000","last_change":"2024-03-30T13:10:54.948212+0000","last_active":"2024-03-30T13:11:01.856085+0000","last_peered":"2024-03-30T13:11:01.856085+0000","last_clean":"2024-03-30T13:11:01.856085+0000","last_became_active":"2024-03-30T13:10:54.947336+0000","last_became_peered":"2024-03-30T13:10:54.947336+0000","last_unstale":"2024-03-30T13:11:01.856085+0000","last_undegraded":"2024-03-30T13:11:01.856085+0000","last_fullsized":"2024-03-30T13:11:01.856085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859925+0000","last_change":"2024-03-30T13:11:00.051626+0000","last_active":"2024-03-30T13:11:01.859925+0000","last_peered":"2024-03-30T13:11:01.859925+0000","last_clean":"2024-03-30T13:11:01.859925+0000","last_became_active":"2024-03-30T13:11:00.050798+0000","last_became_peered":"2024-03-30T13:11:00.050798+0000","last_unstale":"2024-03-30T13:11:01.859925+0000","last_undegraded":"2024-03-30T13:11:01.859925+0000","last_fullsized":"2024-03-30T13:11:01.859925+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:15.648737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853752+0000","last_change":"2024-03-30T13:10:55.965723+0000","last_active":"2024-03-30T13:11:01.853752+0000","last_peered":"2024-03-30T13:11:01.853752+0000","last_clean":"2024-03-30T13:11:01.853752+0000","last_became_active":"2024-03-30T13:10:55.965433+0000","last_became_peered":"2024-03-30T13:10:55.965433+0000","last_unstale":"2024-03-30T13:11:01.853752+0000","last_undegraded":"2024-03-30T13:11:01.853752+0000","last_fullsized":"2024-03-30T13:11:01.853752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854998+0000","last_change":"2024-03-30T13:10:54.940390+0000","last_active":"2024-03-30T13:11:01.854998+0000","last_peered":"2024-03-30T13:11:01.854998+0000","last_clean":"2024-03-30T13:11:01.854998+0000","last_became_active":"2024-03-30T13:10:54.939980+0000","last_became_peered":"2024-03-30T13:10:54.939980+0000","last_unstale":"2024-03-30T13:11:01.854998+0000","last_undegraded":"2024-03-30T13:11:01.854998+0000","last_fullsized":"2024-03-30T13:11:01.854998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858329+0000","last_change":"2024-03-30T13:11:00.019922+0000","last_active":"2024-03-30T13:11:01.858329+0000","last_peered":"2024-03-30T13:11:01.858329+0000","last_clean":"2024-03-30T13:11:01.858329+0000","last_became_active":"2024-03-30T13:11:00.019491+0000","last_became_peered":"2024-03-30T13:11:00.019491+0000","last_unstale":"2024-03-30T13:11:01.858329+0000","last_undegraded":"2024-03-30T13:11:01.858329+0000","last_fullsized":"2024-03-30T13:11:01.858329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856375+0000","last_change":"2024-03-30T13:10:55.968355+0000","last_active":"2024-03-30T13:11:01.856375+0000","last_peered":"2024-03-30T13:11:01.856375+0000","last_clean":"2024-03-30T13:11:01.856375+0000","last_became_active":"2024-03-30T13:10:55.965410+0000","last_became_peered":"2024-03-30T13:10:55.965410+0000","last_unstale":"2024-03-30T13:11:01.856375+0000","last_undegraded":"2024-03-30T13:11:01.856375+0000","last_fullsized":"2024-03-30T13:11:01.856375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:47:03.240883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855058+0000","last_change":"2024-03-30T13:10:54.944101+0000","last_active":"2024-03-30T13:11:01.855058+0000","last_peered":"2024-03-30T13:11:01.855058+0000","last_clean":"2024-03-30T13:11:01.855058+0000","last_became_active":"2024-03-30T13:10:54.943588+0000","last_became_peered":"2024-03-30T13:10:54.943588+0000","last_unstale":"2024-03-30T13:11:01.855058+0000","last_undegraded":"2024-03-30T13:11:01.855058+0000","last_fullsized":"2024-03-30T13:11:01.855058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857849+0000","last_change":"2024-03-30T13:11:00.009582+0000","last_active":"2024-03-30T13:11:01.857849+0000","last_peered":"2024-03-30T13:11:01.857849+0000","last_clean":"2024-03-30T13:11:01.857849+0000","last_became_active":"2024-03-30T13:11:00.009033+0000","last_became_peered":"2024-03-30T13:11:00.009033+0000","last_unstale":"2024-03-30T13:11:01.857849+0000","last_undegraded":"2024-03-30T13:11:01.857849+0000","last_fullsized":"2024-03-30T13:11:01.857849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856879+0000","last_change":"2024-03-30T13:10:56.002735+0000","last_active":"2024-03-30T13:11:01.856879+0000","last_peered":"2024-03-30T13:11:01.856879+0000","last_clean":"2024-03-30T13:11:01.856879+0000","last_became_active":"2024-03-30T13:10:56.001973+0000","last_became_peered":"2024-03-30T13:10:56.001973+0000","last_unstale":"2024-03-30T13:11:01.856879+0000","last_undegraded":"2024-03-30T13:11:01.856879+0000","last_fullsized":"2024-03-30T13:11:01.856879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856685+0000","last_change":"2024-03-30T13:10:54.947650+0000","last_active":"2024-03-30T13:11:01.856685+0000","last_peered":"2024-03-30T13:11:01.856685+0000","last_clean":"2024-03-30T13:11:01.856685+0000","last_became_active":"2024-03-30T13:10:54.946378+0000","last_became_peered":"2024-03-30T13:10:54.946378+0000","last_unstale":"2024-03-30T13:11:01.856685+0000","last_undegraded":"2024-03-30T13:11:01.856685+0000","last_fullsized":"2024-03-30T13:11:01.856685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855829+0000","last_change":"2024-03-30T13:11:00.031425+0000","last_active":"2024-03-30T13:11:01.855829+0000","last_peered":"2024-03-30T13:11:01.855829+0000","last_clean":"2024-03-30T13:11:01.855829+0000","last_became_active":"2024-03-30T13:11:00.030689+0000","last_became_peered":"2024-03-30T13:11:00.030689+0000","last_unstale":"2024-03-30T13:11:01.855829+0000","last_undegraded":"2024-03-30T13:11:01.855829+0000","last_fullsized":"2024-03-30T13:11:01.855829+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:50.405876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981455+0000","last_change":"2024-03-30T13:10:55.991469+0000","last_active":"2024-03-30T13:11:00.981455+0000","last_peered":"2024-03-30T13:11:00.981455+0000","last_clean":"2024-03-30T13:11:00.981455+0000","last_became_active":"2024-03-30T13:10:55.990596+0000","last_became_peered":"2024-03-30T13:10:55.990596+0000","last_unstale":"2024-03-30T13:11:00.981455+0000","last_undegraded":"2024-03-30T13:11:00.981455+0000","last_fullsized":"2024-03-30T13:11:00.981455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856131+0000","last_change":"2024-03-30T13:10:54.945048+0000","last_active":"2024-03-30T13:11:01.856131+0000","last_peered":"2024-03-30T13:11:01.856131+0000","last_clean":"2024-03-30T13:11:01.856131+0000","last_became_active":"2024-03-30T13:10:54.944552+0000","last_became_peered":"2024-03-30T13:10:54.944552+0000","last_unstale":"2024-03-30T13:11:01.856131+0000","last_undegraded":"2024-03-30T13:11:01.856131+0000","last_fullsized":"2024-03-30T13:11:01.856131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:08.826146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980755+0000","last_change":"2024-03-30T13:11:00.038523+0000","last_active":"2024-03-30T13:11:00.980755+0000","last_peered":"2024-03-30T13:11:00.980755+0000","last_clean":"2024-03-30T13:11:00.980755+0000","last_became_active":"2024-03-30T13:11:00.037276+0000","last_became_peered":"2024-03-30T13:11:00.037276+0000","last_unstale":"2024-03-30T13:11:00.980755+0000","last_undegraded":"2024-03-30T13:11:00.980755+0000","last_fullsized":"2024-03-30T13:11:00.980755+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858113+0000","last_change":"2024-03-30T13:10:55.992831+0000","last_active":"2024-03-30T13:11:01.858113+0000","last_peered":"2024-03-30T13:11:01.858113+0000","last_clean":"2024-03-30T13:11:01.858113+0000","last_became_active":"2024-03-30T13:10:55.992521+0000","last_became_peered":"2024-03-30T13:10:55.992521+0000","last_unstale":"2024-03-30T13:11:01.858113+0000","last_undegraded":"2024-03-30T13:11:01.858113+0000","last_fullsized":"2024-03-30T13:11:01.858113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979972+0000","last_change":"2024-03-30T13:10:54.952220+0000","last_active":"2024-03-30T13:11:00.979972+0000","last_peered":"2024-03-30T13:11:00.979972+0000","last_clean":"2024-03-30T13:11:00.979972+0000","last_became_active":"2024-03-30T13:10:54.951484+0000","last_became_peered":"2024-03-30T13:10:54.951484+0000","last_unstale":"2024-03-30T13:11:00.979972+0000","last_undegraded":"2024-03-30T13:11:00.979972+0000","last_fullsized":"2024-03-30T13:11:00.979972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:11:24.184946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979274+0000","last_change":"2024-03-30T13:11:00.043402+0000","last_active":"2024-03-30T13:11:00.979274+0000","last_peered":"2024-03-30T13:11:00.979274+0000","last_clean":"2024-03-30T13:11:00.979274+0000","last_became_active":"2024-03-30T13:11:00.040341+0000","last_became_peered":"2024-03-30T13:11:00.040341+0000","last_unstale":"2024-03-30T13:11:00.979274+0000","last_undegraded":"2024-03-30T13:11:00.979274+0000","last_fullsized":"2024-03-30T13:11:00.979274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855368+0000","last_change":"2024-03-30T13:10:55.958140+0000","last_active":"2024-03-30T13:11:01.855368+0000","last_peered":"2024-03-30T13:11:01.855368+0000","last_clean":"2024-03-30T13:11:01.855368+0000","last_became_active":"2024-03-30T13:10:55.957860+0000","last_became_peered":"2024-03-30T13:10:55.957860+0000","last_unstale":"2024-03-30T13:11:01.855368+0000","last_undegraded":"2024-03-30T13:11:01.855368+0000","last_fullsized":"2024-03-30T13:11:01.855368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:54.477311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854336+0000","last_change":"2024-03-30T13:10:54.945433+0000","last_active":"2024-03-30T13:11:01.854336+0000","last_peered":"2024-03-30T13:11:01.854336+0000","last_clean":"2024-03-30T13:11:01.854336+0000","last_became_active":"2024-03-30T13:10:54.945133+0000","last_became_peered":"2024-03-30T13:10:54.945133+0000","last_unstale":"2024-03-30T13:11:01.854336+0000","last_undegraded":"2024-03-30T13:11:01.854336+0000","last_fullsized":"2024-03-30T13:11:01.854336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:59.514214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980453+0000","last_change":"2024-03-30T13:11:00.044030+0000","last_active":"2024-03-30T13:11:00.980453+0000","last_peered":"2024-03-30T13:11:00.980453+0000","last_clean":"2024-03-30T13:11:00.980453+0000","last_became_active":"2024-03-30T13:11:00.037702+0000","last_became_peered":"2024-03-30T13:11:00.037702+0000","last_unstale":"2024-03-30T13:11:00.980453+0000","last_undegraded":"2024-03-30T13:11:00.980453+0000","last_fullsized":"2024-03-30T13:11:00.980453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979645+0000","last_change":"2024-03-30T13:10:55.962616+0000","last_active":"2024-03-30T13:11:00.979645+0000","last_peered":"2024-03-30T13:11:00.979645+0000","last_clean":"2024-03-30T13:11:00.979645+0000","last_became_active":"2024-03-30T13:10:55.961327+0000","last_became_peered":"2024-03-30T13:10:55.961327+0000","last_unstale":"2024-03-30T13:11:00.979645+0000","last_undegraded":"2024-03-30T13:11:00.979645+0000","last_fullsized":"2024-03-30T13:11:00.979645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:17.511817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858373+0000","last_change":"2024-03-30T13:10:54.942851+0000","last_active":"2024-03-30T13:11:01.858373+0000","last_peered":"2024-03-30T13:11:01.858373+0000","last_clean":"2024-03-30T13:11:01.858373+0000","last_became_active":"2024-03-30T13:10:54.939805+0000","last_became_peered":"2024-03-30T13:10:54.939805+0000","last_unstale":"2024-03-30T13:11:01.858373+0000","last_undegraded":"2024-03-30T13:11:01.858373+0000","last_fullsized":"2024-03-30T13:11:01.858373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:24.569683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982246+0000","last_change":"2024-03-30T13:11:00.040718+0000","last_active":"2024-03-30T13:11:00.982246+0000","last_peered":"2024-03-30T13:11:00.982246+0000","last_clean":"2024-03-30T13:11:00.982246+0000","last_became_active":"2024-03-30T13:11:00.039372+0000","last_became_peered":"2024-03-30T13:11:00.039372+0000","last_unstale":"2024-03-30T13:11:00.982246+0000","last_undegraded":"2024-03-30T13:11:00.982246+0000","last_fullsized":"2024-03-30T13:11:00.982246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:26.483027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979060+0000","last_change":"2024-03-30T13:10:55.984056+0000","last_active":"2024-03-30T13:11:00.979060+0000","last_peered":"2024-03-30T13:11:00.979060+0000","last_clean":"2024-03-30T13:11:00.979060+0000","last_became_active":"2024-03-30T13:10:55.974450+0000","last_became_peered":"2024-03-30T13:10:55.974450+0000","last_unstale":"2024-03-30T13:11:00.979060+0000","last_undegraded":"2024-03-30T13:11:00.979060+0000","last_fullsized":"2024-03-30T13:11:00.979060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853918+0000","last_change":"2024-03-30T13:10:54.942183+0000","last_active":"2024-03-30T13:11:01.853918+0000","last_peered":"2024-03-30T13:11:01.853918+0000","last_clean":"2024-03-30T13:11:01.853918+0000","last_became_active":"2024-03-30T13:10:54.941923+0000","last_became_peered":"2024-03-30T13:10:54.941923+0000","last_unstale":"2024-03-30T13:11:01.853918+0000","last_undegraded":"2024-03-30T13:11:01.853918+0000","last_fullsized":"2024-03-30T13:11:01.853918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:39.803698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983242+0000","last_change":"2024-03-30T13:11:00.024121+0000","last_active":"2024-03-30T13:11:00.983242+0000","last_peered":"2024-03-30T13:11:00.983242+0000","last_clean":"2024-03-30T13:11:00.983242+0000","last_became_active":"2024-03-30T13:11:00.022137+0000","last_became_peered":"2024-03-30T13:11:00.022137+0000","last_unstale":"2024-03-30T13:11:00.983242+0000","last_undegraded":"2024-03-30T13:11:00.983242+0000","last_fullsized":"2024-03-30T13:11:00.983242+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:08:16.219711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856025+0000","last_change":"2024-03-30T13:10:55.978563+0000","last_active":"2024-03-30T13:11:01.856025+0000","last_peered":"2024-03-30T13:11:01.856025+0000","last_clean":"2024-03-30T13:11:01.856025+0000","last_became_active":"2024-03-30T13:10:55.973372+0000","last_became_peered":"2024-03-30T13:10:55.973372+0000","last_unstale":"2024-03-30T13:11:01.856025+0000","last_undegraded":"2024-03-30T13:11:01.856025+0000","last_fullsized":"2024-03-30T13:11:01.856025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854645+0000","last_change":"2024-03-30T13:10:54.932742+0000","last_active":"2024-03-30T13:11:01.854645+0000","last_peered":"2024-03-30T13:11:01.854645+0000","last_clean":"2024-03-30T13:11:01.854645+0000","last_became_active":"2024-03-30T13:10:54.932326+0000","last_became_peered":"2024-03-30T13:10:54.932326+0000","last_unstale":"2024-03-30T13:11:01.854645+0000","last_undegraded":"2024-03-30T13:11:01.854645+0000","last_fullsized":"2024-03-30T13:11:01.854645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977500+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:11:00.977500+0000","last_peered":"2024-03-30T13:11:00.977500+0000","last_clean":"2024-03-30T13:11:00.977500+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:11:00.977500+0000","last_undegraded":"2024-03-30T13:11:00.977500+0000","last_fullsized":"2024-03-30T13:11:00.977500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856097+0000","last_change":"2024-03-30T13:10:59.992352+0000","last_active":"2024-03-30T13:11:01.856097+0000","last_peered":"2024-03-30T13:11:01.856097+0000","last_clean":"2024-03-30T13:11:01.856097+0000","last_became_active":"2024-03-30T13:10:59.991915+0000","last_became_peered":"2024-03-30T13:10:59.991915+0000","last_unstale":"2024-03-30T13:11:01.856097+0000","last_undegraded":"2024-03-30T13:11:01.856097+0000","last_fullsized":"2024-03-30T13:11:01.856097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:13:07.419696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855684+0000","last_change":"2024-03-30T13:10:55.971288+0000","last_active":"2024-03-30T13:11:01.855684+0000","last_peered":"2024-03-30T13:11:01.855684+0000","last_clean":"2024-03-30T13:11:01.855684+0000","last_became_active":"2024-03-30T13:10:55.970980+0000","last_became_peered":"2024-03-30T13:10:55.970980+0000","last_unstale":"2024-03-30T13:11:01.855684+0000","last_undegraded":"2024-03-30T13:11:01.855684+0000","last_fullsized":"2024-03-30T13:11:01.855684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:57.143218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980646+0000","last_change":"2024-03-30T13:10:54.946952+0000","last_active":"2024-03-30T13:11:00.980646+0000","last_peered":"2024-03-30T13:11:00.980646+0000","last_clean":"2024-03-30T13:11:00.980646+0000","last_became_active":"2024-03-30T13:10:54.946235+0000","last_became_peered":"2024-03-30T13:10:54.946235+0000","last_unstale":"2024-03-30T13:11:00.980646+0000","last_undegraded":"2024-03-30T13:11:00.980646+0000","last_fullsized":"2024-03-30T13:11:00.980646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857845+0000","last_change":"2024-03-30T13:11:00.017769+0000","last_active":"2024-03-30T13:11:01.857845+0000","last_peered":"2024-03-30T13:11:01.857845+0000","last_clean":"2024-03-30T13:11:01.857845+0000","last_became_active":"2024-03-30T13:11:00.017317+0000","last_became_peered":"2024-03-30T13:11:00.017317+0000","last_unstale":"2024-03-30T13:11:01.857845+0000","last_undegraded":"2024-03-30T13:11:01.857845+0000","last_fullsized":"2024-03-30T13:11:01.857845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:01.211799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.963559+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.963354+0000","last_became_peered":"2024-03-30T13:10:55.963354+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981372+0000","last_change":"2024-03-30T13:10:54.942041+0000","last_active":"2024-03-30T13:11:00.981372+0000","last_peered":"2024-03-30T13:11:00.981372+0000","last_clean":"2024-03-30T13:11:00.981372+0000","last_became_active":"2024-03-30T13:10:54.940995+0000","last_became_peered":"2024-03-30T13:10:54.940995+0000","last_unstale":"2024-03-30T13:11:00.981372+0000","last_undegraded":"2024-03-30T13:11:00.981372+0000","last_fullsized":"2024-03-30T13:11:00.981372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:54:21.415837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982091+0000","last_change":"2024-03-30T13:11:00.010520+0000","last_active":"2024-03-30T13:11:00.982091+0000","last_peered":"2024-03-30T13:11:00.982091+0000","last_clean":"2024-03-30T13:11:00.982091+0000","last_became_active":"2024-03-30T13:11:00.009785+0000","last_became_peered":"2024-03-30T13:11:00.009785+0000","last_unstale":"2024-03-30T13:11:00.982091+0000","last_undegraded":"2024-03-30T13:11:00.982091+0000","last_fullsized":"2024-03-30T13:11:00.982091+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:13:10.451556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856333+0000","last_change":"2024-03-30T13:10:55.979699+0000","last_active":"2024-03-30T13:11:01.856333+0000","last_peered":"2024-03-30T13:11:01.856333+0000","last_clean":"2024-03-30T13:11:01.856333+0000","last_became_active":"2024-03-30T13:10:55.979427+0000","last_became_peered":"2024-03-30T13:10:55.979427+0000","last_unstale":"2024-03-30T13:11:01.856333+0000","last_undegraded":"2024-03-30T13:11:01.856333+0000","last_fullsized":"2024-03-30T13:11:01.856333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856586+0000","last_change":"2024-03-30T13:10:54.945823+0000","last_active":"2024-03-30T13:11:01.856586+0000","last_peered":"2024-03-30T13:11:01.856586+0000","last_clean":"2024-03-30T13:11:01.856586+0000","last_became_active":"2024-03-30T13:10:54.944624+0000","last_became_peered":"2024-03-30T13:10:54.944624+0000","last_unstale":"2024-03-30T13:11:01.856586+0000","last_undegraded":"2024-03-30T13:11:01.856586+0000","last_fullsized":"2024-03-30T13:11:01.856586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853761+0000","last_change":"2024-03-30T13:11:00.002695+0000","last_active":"2024-03-30T13:11:01.853761+0000","last_peered":"2024-03-30T13:11:01.853761+0000","last_clean":"2024-03-30T13:11:01.853761+0000","last_became_active":"2024-03-30T13:11:00.002251+0000","last_became_peered":"2024-03-30T13:11:00.002251+0000","last_unstale":"2024-03-30T13:11:01.853761+0000","last_undegraded":"2024-03-30T13:11:01.853761+0000","last_fullsized":"2024-03-30T13:11:01.853761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:23:48.725109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858389+0000","last_change":"2024-03-30T13:10:55.966115+0000","last_active":"2024-03-30T13:11:01.858389+0000","last_peered":"2024-03-30T13:11:01.858389+0000","last_clean":"2024-03-30T13:11:01.858389+0000","last_became_active":"2024-03-30T13:10:55.965541+0000","last_became_peered":"2024-03-30T13:10:55.965541+0000","last_unstale":"2024-03-30T13:11:01.858389+0000","last_undegraded":"2024-03-30T13:11:01.858389+0000","last_fullsized":"2024-03-30T13:11:01.858389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856413+0000","last_change":"2024-03-30T13:10:54.945660+0000","last_active":"2024-03-30T13:11:01.856413+0000","last_peered":"2024-03-30T13:11:01.856413+0000","last_clean":"2024-03-30T13:11:01.856413+0000","last_became_active":"2024-03-30T13:10:54.944892+0000","last_became_peered":"2024-03-30T13:10:54.944892+0000","last_unstale":"2024-03-30T13:11:01.856413+0000","last_undegraded":"2024-03-30T13:11:01.856413+0000","last_fullsized":"2024-03-30T13:11:01.856413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977650+0000","last_change":"2024-03-30T13:11:00.014434+0000","last_active":"2024-03-30T13:11:00.977650+0000","last_peered":"2024-03-30T13:11:00.977650+0000","last_clean":"2024-03-30T13:11:00.977650+0000","last_became_active":"2024-03-30T13:11:00.013995+0000","last_became_peered":"2024-03-30T13:11:00.013995+0000","last_unstale":"2024-03-30T13:11:00.977650+0000","last_undegraded":"2024-03-30T13:11:00.977650+0000","last_fullsized":"2024-03-30T13:11:00.977650+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:13.896277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853679+0000","last_change":"2024-03-30T13:10:55.960324+0000","last_active":"2024-03-30T13:11:01.853679+0000","last_peered":"2024-03-30T13:11:01.853679+0000","last_clean":"2024-03-30T13:11:01.853679+0000","last_became_active":"2024-03-30T13:10:55.958877+0000","last_became_peered":"2024-03-30T13:10:55.958877+0000","last_unstale":"2024-03-30T13:11:01.853679+0000","last_undegraded":"2024-03-30T13:11:01.853679+0000","last_fullsized":"2024-03-30T13:11:01.853679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855512+0000","last_change":"2024-03-30T13:10:54.943074+0000","last_active":"2024-03-30T13:11:01.855512+0000","last_peered":"2024-03-30T13:11:01.855512+0000","last_clean":"2024-03-30T13:11:01.855512+0000","last_became_active":"2024-03-30T13:10:54.941870+0000","last_became_peered":"2024-03-30T13:10:54.941870+0000","last_unstale":"2024-03-30T13:11:01.855512+0000","last_undegraded":"2024-03-30T13:11:01.855512+0000","last_fullsized":"2024-03-30T13:11:01.855512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983714+0000","last_change":"2024-03-30T13:11:00.023257+0000","last_active":"2024-03-30T13:11:00.983714+0000","last_peered":"2024-03-30T13:11:00.983714+0000","last_clean":"2024-03-30T13:11:00.983714+0000","last_became_active":"2024-03-30T13:11:00.020654+0000","last_became_peered":"2024-03-30T13:11:00.020654+0000","last_unstale":"2024-03-30T13:11:00.983714+0000","last_undegraded":"2024-03-30T13:11:00.983714+0000","last_fullsized":"2024-03-30T13:11:00.983714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:16:15.446866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977511+0000","last_change":"2024-03-30T13:10:55.970389+0000","last_active":"2024-03-30T13:11:00.977511+0000","last_peered":"2024-03-30T13:11:00.977511+0000","last_clean":"2024-03-30T13:11:00.977511+0000","last_became_active":"2024-03-30T13:10:55.969686+0000","last_became_peered":"2024-03-30T13:10:55.969686+0000","last_unstale":"2024-03-30T13:11:00.977511+0000","last_undegraded":"2024-03-30T13:11:00.977511+0000","last_fullsized":"2024-03-30T13:11:00.977511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855488+0000","last_change":"2024-03-30T13:10:54.945121+0000","last_active":"2024-03-30T13:11:01.855488+0000","last_peered":"2024-03-30T13:11:01.855488+0000","last_clean":"2024-03-30T13:11:01.855488+0000","last_became_active":"2024-03-30T13:10:54.944583+0000","last_became_peered":"2024-03-30T13:10:54.944583+0000","last_unstale":"2024-03-30T13:11:01.855488+0000","last_undegraded":"2024-03-30T13:11:01.855488+0000","last_fullsized":"2024-03-30T13:11:01.855488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981873+0000","last_change":"2024-03-30T13:11:00.023347+0000","last_active":"2024-03-30T13:11:00.981873+0000","last_peered":"2024-03-30T13:11:00.981873+0000","last_clean":"2024-03-30T13:11:00.981873+0000","last_became_active":"2024-03-30T13:11:00.021082+0000","last_became_peered":"2024-03-30T13:11:00.021082+0000","last_unstale":"2024-03-30T13:11:00.981873+0000","last_undegraded":"2024-03-30T13:11:00.981873+0000","last_fullsized":"2024-03-30T13:11:00.981873+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:39.699905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854454+0000","last_change":"2024-03-30T13:10:55.964652+0000","last_active":"2024-03-30T13:11:01.854454+0000","last_peered":"2024-03-30T13:11:01.854454+0000","last_clean":"2024-03-30T13:11:01.854454+0000","last_became_active":"2024-03-30T13:10:55.964363+0000","last_became_peered":"2024-03-30T13:10:55.964363+0000","last_unstale":"2024-03-30T13:11:01.854454+0000","last_undegraded":"2024-03-30T13:11:01.854454+0000","last_fullsized":"2024-03-30T13:11:01.854454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:17.023431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854282+0000","last_change":"2024-03-30T13:10:54.936477+0000","last_active":"2024-03-30T13:11:01.854282+0000","last_peered":"2024-03-30T13:11:01.854282+0000","last_clean":"2024-03-30T13:11:01.854282+0000","last_became_active":"2024-03-30T13:10:54.935689+0000","last_became_peered":"2024-03-30T13:10:54.935689+0000","last_unstale":"2024-03-30T13:11:01.854282+0000","last_undegraded":"2024-03-30T13:11:01.854282+0000","last_fullsized":"2024-03-30T13:11:01.854282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:28.044884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980099+0000","last_change":"2024-03-30T13:11:00.044614+0000","last_active":"2024-03-30T13:11:00.980099+0000","last_peered":"2024-03-30T13:11:00.980099+0000","last_clean":"2024-03-30T13:11:00.980099+0000","last_became_active":"2024-03-30T13:11:00.039909+0000","last_became_peered":"2024-03-30T13:11:00.039909+0000","last_unstale":"2024-03-30T13:11:00.980099+0000","last_undegraded":"2024-03-30T13:11:00.980099+0000","last_fullsized":"2024-03-30T13:11:00.980099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979980+0000","last_change":"2024-03-30T13:10:55.985354+0000","last_active":"2024-03-30T13:11:00.979980+0000","last_peered":"2024-03-30T13:11:00.979980+0000","last_clean":"2024-03-30T13:11:00.979980+0000","last_became_active":"2024-03-30T13:10:55.974918+0000","last_became_peered":"2024-03-30T13:10:55.974918+0000","last_unstale":"2024-03-30T13:11:00.979980+0000","last_undegraded":"2024-03-30T13:11:00.979980+0000","last_fullsized":"2024-03-30T13:11:00.979980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:29.402254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859297+0000","last_change":"2024-03-30T13:10:54.955637+0000","last_active":"2024-03-30T13:11:01.859297+0000","last_peered":"2024-03-30T13:11:01.859297+0000","last_clean":"2024-03-30T13:11:01.859297+0000","last_became_active":"2024-03-30T13:10:54.949281+0000","last_became_peered":"2024-03-30T13:10:54.949281+0000","last_unstale":"2024-03-30T13:11:01.859297+0000","last_undegraded":"2024-03-30T13:11:01.859297+0000","last_fullsized":"2024-03-30T13:11:01.859297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855707+0000","last_change":"2024-03-30T13:11:00.015834+0000","last_active":"2024-03-30T13:11:01.855707+0000","last_peered":"2024-03-30T13:11:01.855707+0000","last_clean":"2024-03-30T13:11:01.855707+0000","last_became_active":"2024-03-30T13:11:00.015344+0000","last_became_peered":"2024-03-30T13:11:00.015344+0000","last_unstale":"2024-03-30T13:11:01.855707+0000","last_undegraded":"2024-03-30T13:11:01.855707+0000","last_fullsized":"2024-03-30T13:11:01.855707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:43.288754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855890+0000","last_change":"2024-03-30T13:10:55.993807+0000","last_active":"2024-03-30T13:11:01.855890+0000","last_peered":"2024-03-30T13:11:01.855890+0000","last_clean":"2024-03-30T13:11:01.855890+0000","last_became_active":"2024-03-30T13:10:55.993589+0000","last_became_peered":"2024-03-30T13:10:55.993589+0000","last_unstale":"2024-03-30T13:11:01.855890+0000","last_undegraded":"2024-03-30T13:11:01.855890+0000","last_fullsized":"2024-03-30T13:11:01.855890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:57.150236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:06.622780+0000","last_change":"2024-03-30T13:10:54.939893+0000","last_active":"2024-03-30T13:11:06.622780+0000","last_peered":"2024-03-30T13:11:06.622780+0000","last_clean":"2024-03-30T13:11:06.622780+0000","last_became_active":"2024-03-30T13:10:54.939427+0000","last_became_peered":"2024-03-30T13:10:54.939427+0000","last_unstale":"2024-03-30T13:11:06.622780+0000","last_undegraded":"2024-03-30T13:11:06.622780+0000","last_fullsized":"2024-03-30T13:11:06.622780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:07.195072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855852+0000","last_change":"2024-03-30T13:11:00.013757+0000","last_active":"2024-03-30T13:11:01.855852+0000","last_peered":"2024-03-30T13:11:01.855852+0000","last_clean":"2024-03-30T13:11:01.855852+0000","last_became_active":"2024-03-30T13:11:00.012615+0000","last_became_peered":"2024-03-30T13:11:00.012615+0000","last_unstale":"2024-03-30T13:11:01.855852+0000","last_undegraded":"2024-03-30T13:11:01.855852+0000","last_fullsized":"2024-03-30T13:11:01.855852+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:02:00.559037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853971+0000","last_change":"2024-03-30T13:10:55.957241+0000","last_active":"2024-03-30T13:11:01.853971+0000","last_peered":"2024-03-30T13:11:01.853971+0000","last_clean":"2024-03-30T13:11:01.853971+0000","last_became_active":"2024-03-30T13:10:55.956872+0000","last_became_peered":"2024-03-30T13:10:55.956872+0000","last_unstale":"2024-03-30T13:11:01.853971+0000","last_undegraded":"2024-03-30T13:11:01.853971+0000","last_fullsized":"2024-03-30T13:11:01.853971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:00.663013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859629+0000","last_change":"2024-03-30T13:10:54.947319+0000","last_active":"2024-03-30T13:11:01.859629+0000","last_peered":"2024-03-30T13:11:01.859629+0000","last_clean":"2024-03-30T13:11:01.859629+0000","last_became_active":"2024-03-30T13:10:54.946638+0000","last_became_peered":"2024-03-30T13:10:54.946638+0000","last_unstale":"2024-03-30T13:11:01.859629+0000","last_undegraded":"2024-03-30T13:11:01.859629+0000","last_fullsized":"2024-03-30T13:11:01.859629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858359+0000","last_change":"2024-03-30T13:11:00.032095+0000","last_active":"2024-03-30T13:11:01.858359+0000","last_peered":"2024-03-30T13:11:01.858359+0000","last_clean":"2024-03-30T13:11:01.858359+0000","last_became_active":"2024-03-30T13:11:00.031651+0000","last_became_peered":"2024-03-30T13:11:00.031651+0000","last_unstale":"2024-03-30T13:11:01.858359+0000","last_undegraded":"2024-03-30T13:11:01.858359+0000","last_fullsized":"2024-03-30T13:11:01.858359+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981894+0000","last_change":"2024-03-30T13:10:55.963310+0000","last_active":"2024-03-30T13:11:00.981894+0000","last_peered":"2024-03-30T13:11:00.981894+0000","last_clean":"2024-03-30T13:11:00.981894+0000","last_became_active":"2024-03-30T13:10:55.962608+0000","last_became_peered":"2024-03-30T13:10:55.962608+0000","last_unstale":"2024-03-30T13:11:00.981894+0000","last_undegraded":"2024-03-30T13:11:00.981894+0000","last_fullsized":"2024-03-30T13:11:00.981894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:53.368510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855969+0000","last_change":"2024-03-30T13:10:54.937448+0000","last_active":"2024-03-30T13:11:01.855969+0000","last_peered":"2024-03-30T13:11:01.855969+0000","last_clean":"2024-03-30T13:11:01.855969+0000","last_became_active":"2024-03-30T13:10:54.936836+0000","last_became_peered":"2024-03-30T13:10:54.936836+0000","last_unstale":"2024-03-30T13:11:01.855969+0000","last_undegraded":"2024-03-30T13:11:01.855969+0000","last_fullsized":"2024-03-30T13:11:01.855969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858228+0000","last_change":"2024-03-30T13:11:00.039950+0000","last_active":"2024-03-30T13:11:01.858228+0000","last_peered":"2024-03-30T13:11:01.858228+0000","last_clean":"2024-03-30T13:11:01.858228+0000","last_became_active":"2024-03-30T13:11:00.038871+0000","last_became_peered":"2024-03-30T13:11:00.038871+0000","last_unstale":"2024-03-30T13:11:01.858228+0000","last_undegraded":"2024-03-30T13:11:01.858228+0000","last_fullsized":"2024-03-30T13:11:01.858228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:16.001906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858336+0000","last_change":"2024-03-30T13:10:55.962143+0000","last_active":"2024-03-30T13:11:01.858336+0000","last_peered":"2024-03-30T13:11:01.858336+0000","last_clean":"2024-03-30T13:11:01.858336+0000","last_became_active":"2024-03-30T13:10:55.961186+0000","last_became_peered":"2024-03-30T13:10:55.961186+0000","last_unstale":"2024-03-30T13:11:01.858336+0000","last_undegraded":"2024-03-30T13:11:01.858336+0000","last_fullsized":"2024-03-30T13:11:01.858336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978652+0000","last_change":"2024-03-30T13:10:54.940086+0000","last_active":"2024-03-30T13:11:00.978652+0000","last_peered":"2024-03-30T13:11:00.978652+0000","last_clean":"2024-03-30T13:11:00.978652+0000","last_became_active":"2024-03-30T13:10:54.939206+0000","last_became_peered":"2024-03-30T13:10:54.939206+0000","last_unstale":"2024-03-30T13:11:00.978652+0000","last_undegraded":"2024-03-30T13:11:00.978652+0000","last_fullsized":"2024-03-30T13:11:00.978652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855931+0000","last_change":"2024-03-30T13:11:00.034220+0000","last_active":"2024-03-30T13:11:01.855931+0000","last_peered":"2024-03-30T13:11:01.855931+0000","last_clean":"2024-03-30T13:11:01.855931+0000","last_became_active":"2024-03-30T13:11:00.033211+0000","last_became_peered":"2024-03-30T13:11:00.033211+0000","last_unstale":"2024-03-30T13:11:01.855931+0000","last_undegraded":"2024-03-30T13:11:01.855931+0000","last_fullsized":"2024-03-30T13:11:01.855931+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:22.895535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856526+0000","last_change":"2024-03-30T13:11:00.028139+0000","last_active":"2024-03-30T13:11:01.856526+0000","last_peered":"2024-03-30T13:11:01.856526+0000","last_clean":"2024-03-30T13:11:01.856526+0000","last_became_active":"2024-03-30T13:11:00.027230+0000","last_became_peered":"2024-03-30T13:11:00.027230+0000","last_unstale":"2024-03-30T13:11:01.856526+0000","last_undegraded":"2024-03-30T13:11:01.856526+0000","last_fullsized":"2024-03-30T13:11:01.856526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:04.724576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855891+0000","last_change":"2024-03-30T13:10:55.996825+0000","last_active":"2024-03-30T13:11:01.855891+0000","last_peered":"2024-03-30T13:11:01.855891+0000","last_clean":"2024-03-30T13:11:01.855891+0000","last_became_active":"2024-03-30T13:10:55.977957+0000","last_became_peered":"2024-03-30T13:10:55.977957+0000","last_unstale":"2024-03-30T13:11:01.855891+0000","last_undegraded":"2024-03-30T13:11:01.855891+0000","last_fullsized":"2024-03-30T13:11:01.855891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853813+0000","last_change":"2024-03-30T13:10:54.930843+0000","last_active":"2024-03-30T13:11:01.853813+0000","last_peered":"2024-03-30T13:11:01.853813+0000","last_clean":"2024-03-30T13:11:01.853813+0000","last_became_active":"2024-03-30T13:10:54.930056+0000","last_became_peered":"2024-03-30T13:10:54.930056+0000","last_unstale":"2024-03-30T13:11:01.853813+0000","last_undegraded":"2024-03-30T13:11:01.853813+0000","last_fullsized":"2024-03-30T13:11:01.853813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854932+0000","last_change":"2024-03-30T13:10:59.991481+0000","last_active":"2024-03-30T13:11:01.854932+0000","last_peered":"2024-03-30T13:11:01.854932+0000","last_clean":"2024-03-30T13:11:01.854932+0000","last_became_active":"2024-03-30T13:10:59.991071+0000","last_became_peered":"2024-03-30T13:10:59.991071+0000","last_unstale":"2024-03-30T13:11:01.854932+0000","last_undegraded":"2024-03-30T13:11:01.854932+0000","last_fullsized":"2024-03-30T13:11:01.854932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:29:50.731633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979852+0000","last_change":"2024-03-30T13:10:55.994215+0000","last_active":"2024-03-30T13:11:00.979852+0000","last_peered":"2024-03-30T13:11:00.979852+0000","last_clean":"2024-03-30T13:11:00.979852+0000","last_became_active":"2024-03-30T13:10:55.993553+0000","last_became_peered":"2024-03-30T13:10:55.993553+0000","last_unstale":"2024-03-30T13:11:00.979852+0000","last_undegraded":"2024-03-30T13:11:00.979852+0000","last_fullsized":"2024-03-30T13:11:00.979852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:35.609380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979309+0000","last_change":"2024-03-30T13:10:54.941904+0000","last_active":"2024-03-30T13:11:00.979309+0000","last_peered":"2024-03-30T13:11:00.979309+0000","last_clean":"2024-03-30T13:11:00.979309+0000","last_became_active":"2024-03-30T13:10:54.941223+0000","last_became_peered":"2024-03-30T13:10:54.941223+0000","last_unstale":"2024-03-30T13:11:00.979309+0000","last_undegraded":"2024-03-30T13:11:00.979309+0000","last_fullsized":"2024-03-30T13:11:00.979309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854339+0000","last_change":"2024-03-30T13:10:54.943752+0000","last_active":"2024-03-30T13:11:01.854339+0000","last_peered":"2024-03-30T13:11:01.854339+0000","last_clean":"2024-03-30T13:11:01.854339+0000","last_became_active":"2024-03-30T13:10:54.943525+0000","last_became_peered":"2024-03-30T13:10:54.943525+0000","last_unstale":"2024-03-30T13:11:01.854339+0000","last_undegraded":"2024-03-30T13:11:01.854339+0000","last_fullsized":"2024-03-30T13:11:01.854339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:47.689419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855653+0000","last_change":"2024-03-30T13:10:55.996261+0000","last_active":"2024-03-30T13:11:01.855653+0000","last_peered":"2024-03-30T13:11:01.855653+0000","last_clean":"2024-03-30T13:11:01.855653+0000","last_became_active":"2024-03-30T13:10:55.979906+0000","last_became_peered":"2024-03-30T13:10:55.979906+0000","last_unstale":"2024-03-30T13:11:01.855653+0000","last_undegraded":"2024-03-30T13:11:01.855653+0000","last_fullsized":"2024-03-30T13:11:01.855653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:11.587174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856525+0000","last_change":"2024-03-30T13:11:00.031762+0000","last_active":"2024-03-30T13:11:01.856525+0000","last_peered":"2024-03-30T13:11:01.856525+0000","last_clean":"2024-03-30T13:11:01.856525+0000","last_became_active":"2024-03-30T13:11:00.030859+0000","last_became_peered":"2024-03-30T13:11:00.030859+0000","last_unstale":"2024-03-30T13:11:01.856525+0000","last_undegraded":"2024-03-30T13:11:01.856525+0000","last_fullsized":"2024-03-30T13:11:01.856525+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:21.084995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982601+0000","last_change":"2024-03-30T13:10:54.944088+0000","last_active":"2024-03-30T13:11:00.982601+0000","last_peered":"2024-03-30T13:11:00.982601+0000","last_clean":"2024-03-30T13:11:00.982601+0000","last_became_active":"2024-03-30T13:10:54.943275+0000","last_became_peered":"2024-03-30T13:10:54.943275+0000","last_unstale":"2024-03-30T13:11:00.982601+0000","last_undegraded":"2024-03-30T13:11:00.982601+0000","last_fullsized":"2024-03-30T13:11:00.982601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:35:57.616293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977803+0000","last_change":"2024-03-30T13:10:55.970451+0000","last_active":"2024-03-30T13:11:00.977803+0000","last_peered":"2024-03-30T13:11:00.977803+0000","last_clean":"2024-03-30T13:11:00.977803+0000","last_became_active":"2024-03-30T13:10:55.969774+0000","last_became_peered":"2024-03-30T13:10:55.969774+0000","last_unstale":"2024-03-30T13:11:00.977803+0000","last_undegraded":"2024-03-30T13:11:00.977803+0000","last_fullsized":"2024-03-30T13:11:00.977803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859408+0000","last_change":"2024-03-30T13:11:00.039666+0000","last_active":"2024-03-30T13:11:01.859408+0000","last_peered":"2024-03-30T13:11:01.859408+0000","last_clean":"2024-03-30T13:11:01.859408+0000","last_became_active":"2024-03-30T13:11:00.038738+0000","last_became_peered":"2024-03-30T13:11:00.038738+0000","last_unstale":"2024-03-30T13:11:01.859408+0000","last_undegraded":"2024-03-30T13:11:01.859408+0000","last_fullsized":"2024-03-30T13:11:01.859408+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:53.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857699+0000","last_change":"2024-03-30T13:10:55.985031+0000","last_active":"2024-03-30T13:11:01.857699+0000","last_peered":"2024-03-30T13:11:01.857699+0000","last_clean":"2024-03-30T13:11:01.857699+0000","last_became_active":"2024-03-30T13:10:55.984727+0000","last_became_peered":"2024-03-30T13:10:55.984727+0000","last_unstale":"2024-03-30T13:11:01.857699+0000","last_undegraded":"2024-03-30T13:11:01.857699+0000","last_fullsized":"2024-03-30T13:11:01.857699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858193+0000","last_change":"2024-03-30T13:10:55.977928+0000","last_active":"2024-03-30T13:11:01.858193+0000","last_peered":"2024-03-30T13:11:01.858193+0000","last_clean":"2024-03-30T13:11:01.858193+0000","last_became_active":"2024-03-30T13:10:55.977153+0000","last_became_peered":"2024-03-30T13:10:55.977153+0000","last_unstale":"2024-03-30T13:11:01.858193+0000","last_undegraded":"2024-03-30T13:11:01.858193+0000","last_fullsized":"2024-03-30T13:11:01.858193+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853982+0000","last_change":"2024-03-30T13:10:55.965525+0000","last_active":"2024-03-30T13:11:01.853982+0000","last_peered":"2024-03-30T13:11:01.853982+0000","last_clean":"2024-03-30T13:11:01.853982+0000","last_became_active":"2024-03-30T13:10:55.965066+0000","last_became_peered":"2024-03-30T13:10:55.965066+0000","last_unstale":"2024-03-30T13:11:01.853982+0000","last_undegraded":"2024-03-30T13:11:01.853982+0000","last_fullsized":"2024-03-30T13:11:01.853982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982361+0000","last_change":"2024-03-30T13:10:55.961817+0000","last_active":"2024-03-30T13:11:00.982361+0000","last_peered":"2024-03-30T13:11:00.982361+0000","last_clean":"2024-03-30T13:11:00.982361+0000","last_became_active":"2024-03-30T13:10:55.960939+0000","last_became_peered":"2024-03-30T13:10:55.960939+0000","last_unstale":"2024-03-30T13:11:00.982361+0000","last_undegraded":"2024-03-30T13:11:00.982361+0000","last_fullsized":"2024-03-30T13:11:00.982361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:30:53.806860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980400+0000","last_change":"2024-03-30T13:10:55.991524+0000","last_active":"2024-03-30T13:11:00.980400+0000","last_peered":"2024-03-30T13:11:00.980400+0000","last_clean":"2024-03-30T13:11:00.980400+0000","last_became_active":"2024-03-30T13:10:55.990783+0000","last_became_peered":"2024-03-30T13:10:55.990783+0000","last_unstale":"2024-03-30T13:11:00.980400+0000","last_undegraded":"2024-03-30T13:11:00.980400+0000","last_fullsized":"2024-03-30T13:11:00.980400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:09.834713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981092+0000","last_change":"2024-03-30T13:10:55.985903+0000","last_active":"2024-03-30T13:11:00.981092+0000","last_peered":"2024-03-30T13:11:00.981092+0000","last_clean":"2024-03-30T13:11:00.981092+0000","last_became_active":"2024-03-30T13:10:55.985086+0000","last_became_peered":"2024-03-30T13:10:55.985086+0000","last_unstale":"2024-03-30T13:11:00.981092+0000","last_undegraded":"2024-03-30T13:11:00.981092+0000","last_fullsized":"2024-03-30T13:11:00.981092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858231+0000","last_change":"2024-03-30T13:10:55.961871+0000","last_active":"2024-03-30T13:11:01.858231+0000","last_peered":"2024-03-30T13:11:01.858231+0000","last_clean":"2024-03-30T13:11:01.858231+0000","last_became_active":"2024-03-30T13:10:55.961589+0000","last_became_peered":"2024-03-30T13:10:55.961589+0000","last_unstale":"2024-03-30T13:11:01.858231+0000","last_undegraded":"2024-03-30T13:11:01.858231+0000","last_fullsized":"2024-03-30T13:11:01.858231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981592+0000","last_change":"2024-03-30T13:10:55.979356+0000","last_active":"2024-03-30T13:11:00.981592+0000","last_peered":"2024-03-30T13:11:00.981592+0000","last_clean":"2024-03-30T13:11:00.981592+0000","last_became_active":"2024-03-30T13:10:55.978691+0000","last_became_peered":"2024-03-30T13:10:55.978691+0000","last_unstale":"2024-03-30T13:11:00.981592+0000","last_undegraded":"2024-03-30T13:11:00.981592+0000","last_fullsized":"2024-03-30T13:11:00.981592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:26:40.192186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853812+0000","last_change":"2024-03-30T13:10:55.965264+0000","last_active":"2024-03-30T13:11:01.853812+0000","last_peered":"2024-03-30T13:11:01.853812+0000","last_clean":"2024-03-30T13:11:01.853812+0000","last_became_active":"2024-03-30T13:10:55.964763+0000","last_became_peered":"2024-03-30T13:10:55.964763+0000","last_unstale":"2024-03-30T13:11:01.853812+0000","last_undegraded":"2024-03-30T13:11:01.853812+0000","last_fullsized":"2024-03-30T13:11:01.853812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980777+0000","last_change":"2024-03-30T13:10:55.968931+0000","last_active":"2024-03-30T13:11:00.980777+0000","last_peered":"2024-03-30T13:11:00.980777+0000","last_clean":"2024-03-30T13:11:00.980777+0000","last_became_active":"2024-03-30T13:10:55.968082+0000","last_became_peered":"2024-03-30T13:10:55.968082+0000","last_unstale":"2024-03-30T13:11:00.980777+0000","last_undegraded":"2024-03-30T13:11:00.980777+0000","last_fullsized":"2024-03-30T13:11:00.980777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:06:36.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977480+0000","last_change":"2024-03-30T13:10:55.971076+0000","last_active":"2024-03-30T13:11:00.977480+0000","last_peered":"2024-03-30T13:11:00.977480+0000","last_clean":"2024-03-30T13:11:00.977480+0000","last_became_active":"2024-03-30T13:10:55.970755+0000","last_became_peered":"2024-03-30T13:10:55.970755+0000","last_unstale":"2024-03-30T13:11:00.977480+0000","last_undegraded":"2024-03-30T13:11:00.977480+0000","last_fullsized":"2024-03-30T13:11:00.977480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855370+0000","last_change":"2024-03-30T13:10:55.996858+0000","last_active":"2024-03-30T13:11:01.855370+0000","last_peered":"2024-03-30T13:11:01.855370+0000","last_clean":"2024-03-30T13:11:01.855370+0000","last_became_active":"2024-03-30T13:10:55.980721+0000","last_became_peered":"2024-03-30T13:10:55.980721+0000","last_unstale":"2024-03-30T13:11:01.855370+0000","last_undegraded":"2024-03-30T13:11:01.855370+0000","last_fullsized":"2024-03-30T13:11:01.855370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:49.462756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855814+0000","last_change":"2024-03-30T13:10:55.976030+0000","last_active":"2024-03-30T13:11:01.855814+0000","last_peered":"2024-03-30T13:11:01.855814+0000","last_clean":"2024-03-30T13:11:01.855814+0000","last_became_active":"2024-03-30T13:10:55.975744+0000","last_became_peered":"2024-03-30T13:10:55.975744+0000","last_unstale":"2024-03-30T13:11:01.855814+0000","last_undegraded":"2024-03-30T13:11:01.855814+0000","last_fullsized":"2024-03-30T13:11:01.855814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.998425+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.998137+0000","last_became_peered":"2024-03-30T13:10:55.998137+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:11:09.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859740+0000","last_change":"2024-03-30T13:10:55.961591+0000","last_active":"2024-03-30T13:11:01.859740+0000","last_peered":"2024-03-30T13:11:01.859740+0000","last_clean":"2024-03-30T13:11:01.859740+0000","last_became_active":"2024-03-30T13:10:55.960885+0000","last_became_peered":"2024-03-30T13:10:55.960885+0000","last_unstale":"2024-03-30T13:11:01.859740+0000","last_undegraded":"2024-03-30T13:11:01.859740+0000","last_fullsized":"2024-03-30T13:11:01.859740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855050+0000","last_change":"2024-03-30T13:10:55.964798+0000","last_active":"2024-03-30T13:11:01.855050+0000","last_peered":"2024-03-30T13:11:01.855050+0000","last_clean":"2024-03-30T13:11:01.855050+0000","last_became_active":"2024-03-30T13:10:55.964471+0000","last_became_peered":"2024-03-30T13:10:55.964471+0000","last_unstale":"2024-03-30T13:11:01.855050+0000","last_undegraded":"2024-03-30T13:11:01.855050+0000","last_fullsized":"2024-03-30T13:11:01.855050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:41.557413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853680+0000","last_change":"2024-03-30T13:10:55.959954+0000","last_active":"2024-03-30T13:11:01.853680+0000","last_peered":"2024-03-30T13:11:01.853680+0000","last_clean":"2024-03-30T13:11:01.853680+0000","last_became_active":"2024-03-30T13:10:55.958856+0000","last_became_peered":"2024-03-30T13:10:55.958856+0000","last_unstale":"2024-03-30T13:11:01.853680+0000","last_undegraded":"2024-03-30T13:11:01.853680+0000","last_fullsized":"2024-03-30T13:11:01.853680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980372+0000","last_change":"2024-03-30T13:10:55.957900+0000","last_active":"2024-03-30T13:11:00.980372+0000","last_peered":"2024-03-30T13:11:00.980372+0000","last_clean":"2024-03-30T13:11:00.980372+0000","last_became_active":"2024-03-30T13:10:55.957071+0000","last_became_peered":"2024-03-30T13:10:55.957071+0000","last_unstale":"2024-03-30T13:11:00.980372+0000","last_undegraded":"2024-03-30T13:11:00.980372+0000","last_fullsized":"2024-03-30T13:11:00.980372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:16:27.043781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979715+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.979715+0000","last_peered":"2024-03-30T13:11:00.979715+0000","last_clean":"2024-03-30T13:11:00.979715+0000","last_became_active":"2024-03-30T13:10:55.990646+0000","last_became_peered":"2024-03-30T13:10:55.990646+0000","last_unstale":"2024-03-30T13:11:00.979715+0000","last_undegraded":"2024-03-30T13:11:00.979715+0000","last_fullsized":"2024-03-30T13:11:00.979715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981245+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.981245+0000","last_peered":"2024-03-30T13:11:00.981245+0000","last_clean":"2024-03-30T13:11:00.981245+0000","last_became_active":"2024-03-30T13:10:55.990666+0000","last_became_peered":"2024-03-30T13:10:55.990666+0000","last_unstale":"2024-03-30T13:11:00.981245+0000","last_undegraded":"2024-03-30T13:11:00.981245+0000","last_fullsized":"2024-03-30T13:11:00.981245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855715+0000","last_change":"2024-03-30T13:10:55.975002+0000","last_active":"2024-03-30T13:11:01.855715+0000","last_peered":"2024-03-30T13:11:01.855715+0000","last_clean":"2024-03-30T13:11:01.855715+0000","last_became_active":"2024-03-30T13:10:55.974620+0000","last_became_peered":"2024-03-30T13:10:55.974620+0000","last_unstale":"2024-03-30T13:11:01.855715+0000","last_undegraded":"2024-03-30T13:11:01.855715+0000","last_fullsized":"2024-03-30T13:11:01.855715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:33.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855796+0000","last_change":"2024-03-30T13:10:55.962912+0000","last_active":"2024-03-30T13:11:01.855796+0000","last_peered":"2024-03-30T13:11:01.855796+0000","last_clean":"2024-03-30T13:11:01.855796+0000","last_became_active":"2024-03-30T13:10:55.962404+0000","last_became_peered":"2024-03-30T13:10:55.962404+0000","last_unstale":"2024-03-30T13:11:01.855796+0000","last_undegraded":"2024-03-30T13:11:01.855796+0000","last_fullsized":"2024-03-30T13:11:01.855796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855883+0000","last_change":"2024-03-30T13:10:55.962709+0000","last_active":"2024-03-30T13:11:01.855883+0000","last_peered":"2024-03-30T13:11:01.855883+0000","last_clean":"2024-03-30T13:11:01.855883+0000","last_became_active":"2024-03-30T13:10:55.962426+0000","last_became_peered":"2024-03-30T13:10:55.962426+0000","last_unstale":"2024-03-30T13:11:01.855883+0000","last_undegraded":"2024-03-30T13:11:01.855883+0000","last_fullsized":"2024-03-30T13:11:01.855883+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978582+0000","last_change":"2024-03-30T13:10:55.973390+0000","last_active":"2024-03-30T13:11:00.978582+0000","last_peered":"2024-03-30T13:11:00.978582+0000","last_clean":"2024-03-30T13:11:00.978582+0000","last_became_active":"2024-03-30T13:10:55.969915+0000","last_became_peered":"2024-03-30T13:10:55.969915+0000","last_unstale":"2024-03-30T13:11:00.978582+0000","last_undegraded":"2024-03-30T13:11:00.978582+0000","last_fullsized":"2024-03-30T13:11:00.978582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979479+0000","last_change":"2024-03-30T13:10:55.969396+0000","last_active":"2024-03-30T13:11:00.979479+0000","last_peered":"2024-03-30T13:11:00.979479+0000","last_clean":"2024-03-30T13:11:00.979479+0000","last_became_active":"2024-03-30T13:10:55.965044+0000","last_became_peered":"2024-03-30T13:10:55.965044+0000","last_unstale":"2024-03-30T13:11:00.979479+0000","last_undegraded":"2024-03-30T13:11:00.979479+0000","last_fullsized":"2024-03-30T13:11:00.979479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:34:33.758969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855400+0000","last_change":"2024-03-30T13:10:55.974744+0000","last_active":"2024-03-30T13:11:01.855400+0000","last_peered":"2024-03-30T13:11:01.855400+0000","last_clean":"2024-03-30T13:11:01.855400+0000","last_became_active":"2024-03-30T13:10:55.974421+0000","last_became_peered":"2024-03-30T13:10:55.974421+0000","last_unstale":"2024-03-30T13:11:01.855400+0000","last_undegraded":"2024-03-30T13:11:01.855400+0000","last_fullsized":"2024-03-30T13:11:01.855400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:40.791064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857905+0000","last_change":"2024-03-30T13:10:55.998371+0000","last_active":"2024-03-30T13:11:01.857905+0000","last_peered":"2024-03-30T13:11:01.857905+0000","last_clean":"2024-03-30T13:11:01.857905+0000","last_became_active":"2024-03-30T13:10:55.997851+0000","last_became_peered":"2024-03-30T13:10:55.997851+0000","last_unstale":"2024-03-30T13:11:01.857905+0000","last_undegraded":"2024-03-30T13:11:01.857905+0000","last_fullsized":"2024-03-30T13:11:01.857905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:46.705515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855480+0000","last_change":"2024-03-30T13:10:55.967579+0000","last_active":"2024-03-30T13:11:01.855480+0000","last_peered":"2024-03-30T13:11:01.855480+0000","last_clean":"2024-03-30T13:11:01.855480+0000","last_became_active":"2024-03-30T13:10:55.967284+0000","last_became_peered":"2024-03-30T13:10:55.967284+0000","last_unstale":"2024-03-30T13:11:01.855480+0000","last_undegraded":"2024-03-30T13:11:01.855480+0000","last_fullsized":"2024-03-30T13:11:01.855480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:39.989019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858659+0000","last_change":"2024-03-30T13:10:55.998028+0000","last_active":"2024-03-30T13:11:01.858659+0000","last_peered":"2024-03-30T13:11:01.858659+0000","last_clean":"2024-03-30T13:11:01.858659+0000","last_became_active":"2024-03-30T13:10:55.997628+0000","last_became_peered":"2024-03-30T13:10:55.997628+0000","last_unstale":"2024-03-30T13:11:01.858659+0000","last_undegraded":"2024-03-30T13:11:01.858659+0000","last_fullsized":"2024-03-30T13:11:01.858659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982130+0000","last_change":"2024-03-30T13:10:55.983619+0000","last_active":"2024-03-30T13:11:00.982130+0000","last_peered":"2024-03-30T13:11:00.982130+0000","last_clean":"2024-03-30T13:11:00.982130+0000","last_became_active":"2024-03-30T13:10:55.982947+0000","last_became_peered":"2024-03-30T13:10:55.982947+0000","last_unstale":"2024-03-30T13:11:00.982130+0000","last_undegraded":"2024-03-30T13:11:00.982130+0000","last_fullsized":"2024-03-30T13:11:00.982130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:00.291625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983477+0000","last_change":"2024-03-30T13:10:55.992614+0000","last_active":"2024-03-30T13:11:00.983477+0000","last_peered":"2024-03-30T13:11:00.983477+0000","last_clean":"2024-03-30T13:11:00.983477+0000","last_became_active":"2024-03-30T13:10:55.991559+0000","last_became_peered":"2024-03-30T13:10:55.991559+0000","last_unstale":"2024-03-30T13:11:00.983477+0000","last_undegraded":"2024-03-30T13:11:00.983477+0000","last_fullsized":"2024-03-30T13:11:00.983477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:28:13.766039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856251+0000","last_change":"2024-03-30T13:10:55.960973+0000","last_active":"2024-03-30T13:11:01.856251+0000","last_peered":"2024-03-30T13:11:01.856251+0000","last_clean":"2024-03-30T13:11:01.856251+0000","last_became_active":"2024-03-30T13:10:55.960665+0000","last_became_peered":"2024-03-30T13:10:55.960665+0000","last_unstale":"2024-03-30T13:11:01.856251+0000","last_undegraded":"2024-03-30T13:11:01.856251+0000","last_fullsized":"2024-03-30T13:11:01.856251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:25.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:11:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.919+0000 7efefb696640 1 Processor -- start 2024-03-30T13:11:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.919+0000 7efefb696640 1 -- start start 2024-03-30T13:11:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efefb696640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efef41607e0 0x7efef415c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efefb696640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 0x7efef40cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efefb696640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef40cf0c0 con 0x7efef40cd140 2024-03-30T13:11:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efefb696640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efef415cc90 con 0x7efef41607e0 2024-03-30T13:11:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efefb696640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efef40cf3f0 con 0x7efef406b660 2024-03-30T13:11:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9432640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efef40cd140 0x7efef40cd520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59750/0 (socket says 172.21.15.43:59750) 2024-03-30T13:11:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9432640 1 -- 172.21.15.43:0/1135911309 learned_addr learned my addr 172.21.15.43:0/1135911309 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9c33640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 0x7efef40cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef8c31640 1 --2- 172.21.15.43:0/1135911309 >> v2:172.21.15.113:3300/0 conn(0x7efef41607e0 0x7efef415c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199059123 0 0) 0x7efef40cf0c0 con 0x7efef40cd140 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efee0003530 con 0x7efef40cd140 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2505884279 0 0) 0x7efee0003530 con 0x7efef40cd140 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 msgr2=0x7efef40cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 0x7efef40cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 >> v2:172.21.15.113:3300/0 conn(0x7efef41607e0 msgr2=0x7efef415c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 --2- 172.21.15.43:0/1135911309 >> v2:172.21.15.113:3300/0 conn(0x7efef41607e0 0x7efef415c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef4160c70 con 0x7efef40cd140 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7efee4001680 con 0x7efef40cd140 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 >> v1:172.21.15.43:6789/0 conn(0x7efef40cd140 legacy=0x7efef40cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 0x7efee00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efee0003f60 0x7efee0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee0004d50 0x7efee00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efef40cf3f0 con 0x7efee0004d50 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efee000b9d0 con 0x7efee0003f60 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efee000bb50 con 0x7efef406b660 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9c33640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee0004d50 0x7efee00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef8c31640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efee0003f60 0x7efee0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9432640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 0x7efee00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9c33640 1 -- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 msgr2=0x7efee00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9c33640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efef406b660 0x7efee00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9c33640 1 -- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efee0003f60 msgr2=0x7efee0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9c33640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efee0003f60 0x7efee0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.923+0000 7efef9c33640 1 -- 172.21.15.43:0/1135911309 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee0003530 con 0x7efee0004d50 2024-03-30T13:11:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efef9c33640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee0004d50 0x7efee00051d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e170 tx=0x7efeec01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9baf3585ac674e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0b2070 con 0x7efee0004d50 2024-03-30T13:11:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec0a5c70 con 0x7efee0004d50 2024-03-30T13:11:25.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efeebfff640 1 -- 172.21.15.43:0/1135911309 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec009c10 con 0x7efee0004d50 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 -- 172.21.15.43:0/1135911309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee0004d50 msgr2=0x7efee00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 --2- 172.21.15.43:0/1135911309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efee0004d50 0x7efee00051d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e170 tx=0x7efeec01cc60 comp rx=0 tx=0).stop 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efef9c33640 1 -- 172.21.15.43:0/1135911309 reap_dead start 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 -- 172.21.15.43:0/1135911309 shutdown_connections 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 -- 172.21.15.43:0/1135911309 >> 172.21.15.43:0/1135911309 conn(0x7efef40b2b30 msgr2=0x7efef40d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 -- 172.21.15.43:0/1135911309 shutdown_connections 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 -- 172.21.15.43:0/1135911309 wait complete. 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 Processor -- start 2024-03-30T13:11:25.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.927+0000 7efefb696640 1 -- start start 2024-03-30T13:11:25.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efee0004d50 0x7efef40ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efef40ae870 0x7efef40c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef41f1e20 0x7efef41f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efef415d480 con 0x7efef41f1e20 2024-03-30T13:11:25.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efef415d1a0 con 0x7efef40ae870 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efef406baf0 con 0x7efee0004d50 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef41f1e20 0x7efef41f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef41f1e20 0x7efef41f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35168/0 (socket says 172.21.15.43:35168) 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 -- 172.21.15.43:0/2288946404 learned_addr learned my addr 172.21.15.43:0/2288946404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9432640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efee0004d50 0x7efef40ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef8c31640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efef40ae870 0x7efef40c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 -- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efee0004d50 msgr2=0x7efef40ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efee0004d50 0x7efef40ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 -- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efef40ae870 msgr2=0x7efef40c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efef40ae870 0x7efef40c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:25.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 -- 172.21.15.43:0/2288946404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef40ad7d0 con 0x7efef41f1e20 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9c33640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef41f1e20 0x7efef41f22d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e170 tx=0x7efeec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efee9ffb640 1 -- 172.21.15.43:0/2288946404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0b2040 con 0x7efef41f1e20 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efee9ffb640 1 -- 172.21.15.43:0/2288946404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec0a5a70 con 0x7efef41f1e20 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef40adab0 con 0x7efef41f1e20 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efee9ffb640 1 -- 172.21.15.43:0/2288946404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec009c10 con 0x7efef41f1e20 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef40ae070 con 0x7efef41f1e20 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef4068890 con 0x7efef41f1e20 2024-03-30T13:11:25.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efee9ffb640 1 -- 172.21.15.43:0/2288946404 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efeec0be020 con 0x7efef41f1e20 2024-03-30T13:11:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efee9ffb640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efed8067fc0 0x7efed806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efee9ffb640 1 -- 172.21.15.43:0/2288946404 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7efeec120990 con 0x7efef41f1e20 2024-03-30T13:11:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.931+0000 7efef9432640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efed8067fc0 0x7efed806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:25.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.935+0000 7efef9432640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efed8067fc0 0x7efed806a480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efee4000f70 tx=0x7efee4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:25.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:25.943+0000 7efee9ffb640 1 -- 172.21.15.43:0/2288946404 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efeec0c5020 con 0x7efef41f1e20 2024-03-30T13:11:26.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.183+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efef40b3820 con 0x7efed8067fc0 2024-03-30T13:11:26.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.207+0000 7efee9ffb640 1 -- 172.21.15.43:0/2288946404 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332527 (secure 0 0 0) 0x7efef40b3820 con 0x7efed8067fc0 2024-03-30T13:11:26.215 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:26.218 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:11:26.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efed8067fc0 msgr2=0x7efed806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efed8067fc0 0x7efed806a480 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efee4000f70 tx=0x7efee4009040 comp rx=0 tx=0).stop 2024-03-30T13:11:26.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef41f1e20 msgr2=0x7efef41f22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef41f1e20 0x7efef41f22d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e170 tx=0x7efeec009750 comp rx=0 tx=0).stop 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 shutdown_connections 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efed8067fc0 0x7efed806a480 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efef41f1e20 0x7efef41f22d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efef40ae870 0x7efef40c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 --2- 172.21.15.43:0/2288946404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efee0004d50 0x7efef40ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 >> 172.21.15.43:0/2288946404 conn(0x7efef40b2b30 msgr2=0x7efef40b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 shutdown_connections 2024-03-30T13:11:26.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.211+0000 7efefb696640 1 -- 172.21.15.43:0/2288946404 wait complete. 2024-03-30T13:11:26.235 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2024-03-30T13:11:25.847359+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236252,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738468,"statfs":{"total":773094113280,"available":772852191232,"internally_reserved":0,"allocated":3514368,"data_stored":2074536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005614"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857033+0000","last_change":"2024-03-30T13:10:54.944419+0000","last_active":"2024-03-30T13:11:01.857033+0000","last_peered":"2024-03-30T13:11:01.857033+0000","last_clean":"2024-03-30T13:11:01.857033+0000","last_became_active":"2024-03-30T13:10:54.943176+0000","last_became_peered":"2024-03-30T13:10:54.943176+0000","last_unstale":"2024-03-30T13:11:01.857033+0000","last_undegraded":"2024-03-30T13:11:01.857033+0000","last_fullsized":"2024-03-30T13:11:01.857033+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983961+0000","last_change":"2024-03-30T13:10:55.957609+0000","last_active":"2024-03-30T13:11:00.983961+0000","last_peered":"2024-03-30T13:11:00.983961+0000","last_clean":"2024-03-30T13:11:00.983961+0000","last_became_active":"2024-03-30T13:10:55.956870+0000","last_became_peered":"2024-03-30T13:10:55.956870+0000","last_unstale":"2024-03-30T13:11:00.983961+0000","last_undegraded":"2024-03-30T13:11:00.983961+0000","last_fullsized":"2024-03-30T13:11:00.983961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:04:28.315124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858409+0000","last_change":"2024-03-30T13:11:00.037601+0000","last_active":"2024-03-30T13:11:01.858409+0000","last_peered":"2024-03-30T13:11:01.858409+0000","last_clean":"2024-03-30T13:11:01.858409+0000","last_became_active":"2024-03-30T13:11:00.032050+0000","last_became_peered":"2024-03-30T13:11:00.032050+0000","last_unstale":"2024-03-30T13:11:01.858409+0000","last_undegraded":"2024-03-30T13:11:01.858409+0000","last_fullsized":"2024-03-30T13:11:01.858409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:47.447707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.984087+0000","last_change":"2024-03-30T13:10:54.940096+0000","last_active":"2024-03-30T13:11:00.984087+0000","last_peered":"2024-03-30T13:11:00.984087+0000","last_clean":"2024-03-30T13:11:00.984087+0000","last_became_active":"2024-03-30T13:10:54.939368+0000","last_became_peered":"2024-03-30T13:10:54.939368+0000","last_unstale":"2024-03-30T13:11:00.984087+0000","last_undegraded":"2024-03-30T13:11:00.984087+0000","last_fullsized":"2024-03-30T13:11:00.984087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:09:39.904016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981759+0000","last_change":"2024-03-30T13:10:55.979018+0000","last_active":"2024-03-30T13:11:00.981759+0000","last_peered":"2024-03-30T13:11:00.981759+0000","last_clean":"2024-03-30T13:11:00.981759+0000","last_became_active":"2024-03-30T13:10:55.975807+0000","last_became_peered":"2024-03-30T13:10:55.975807+0000","last_unstale":"2024-03-30T13:11:00.981759+0000","last_undegraded":"2024-03-30T13:11:00.981759+0000","last_fullsized":"2024-03-30T13:11:00.981759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:35:35.881832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857637+0000","last_change":"2024-03-30T13:11:00.020209+0000","last_active":"2024-03-30T13:11:01.857637+0000","last_peered":"2024-03-30T13:11:01.857637+0000","last_clean":"2024-03-30T13:11:01.857637+0000","last_became_active":"2024-03-30T13:11:00.019627+0000","last_became_peered":"2024-03-30T13:11:00.019627+0000","last_unstale":"2024-03-30T13:11:01.857637+0000","last_undegraded":"2024-03-30T13:11:01.857637+0000","last_fullsized":"2024-03-30T13:11:01.857637+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:31.595903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856064+0000","last_change":"2024-03-30T13:10:54.945298+0000","last_active":"2024-03-30T13:11:01.856064+0000","last_peered":"2024-03-30T13:11:01.856064+0000","last_clean":"2024-03-30T13:11:01.856064+0000","last_became_active":"2024-03-30T13:10:54.942725+0000","last_became_peered":"2024-03-30T13:10:54.942725+0000","last_unstale":"2024-03-30T13:11:01.856064+0000","last_undegraded":"2024-03-30T13:11:01.856064+0000","last_fullsized":"2024-03-30T13:11:01.856064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978049+0000","last_change":"2024-03-30T13:10:55.973824+0000","last_active":"2024-03-30T13:11:00.978049+0000","last_peered":"2024-03-30T13:11:00.978049+0000","last_clean":"2024-03-30T13:11:00.978049+0000","last_became_active":"2024-03-30T13:10:55.969728+0000","last_became_peered":"2024-03-30T13:10:55.969728+0000","last_unstale":"2024-03-30T13:11:00.978049+0000","last_undegraded":"2024-03-30T13:11:00.978049+0000","last_fullsized":"2024-03-30T13:11:00.978049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858055+0000","last_change":"2024-03-30T13:11:00.015698+0000","last_active":"2024-03-30T13:11:01.858055+0000","last_peered":"2024-03-30T13:11:01.858055+0000","last_clean":"2024-03-30T13:11:01.858055+0000","last_became_active":"2024-03-30T13:11:00.015263+0000","last_became_peered":"2024-03-30T13:11:00.015263+0000","last_unstale":"2024-03-30T13:11:01.858055+0000","last_undegraded":"2024-03-30T13:11:01.858055+0000","last_fullsized":"2024-03-30T13:11:01.858055+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856071+0000","last_change":"2024-03-30T13:10:54.941048+0000","last_active":"2024-03-30T13:11:01.856071+0000","last_peered":"2024-03-30T13:11:01.856071+0000","last_clean":"2024-03-30T13:11:01.856071+0000","last_became_active":"2024-03-30T13:10:54.940543+0000","last_became_peered":"2024-03-30T13:10:54.940543+0000","last_unstale":"2024-03-30T13:11:01.856071+0000","last_undegraded":"2024-03-30T13:11:01.856071+0000","last_fullsized":"2024-03-30T13:11:01.856071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855446+0000","last_change":"2024-03-30T13:10:55.965085+0000","last_active":"2024-03-30T13:11:01.855446+0000","last_peered":"2024-03-30T13:11:01.855446+0000","last_clean":"2024-03-30T13:11:01.855446+0000","last_became_active":"2024-03-30T13:10:55.964573+0000","last_became_peered":"2024-03-30T13:10:55.964573+0000","last_unstale":"2024-03-30T13:11:01.855446+0000","last_undegraded":"2024-03-30T13:11:01.855446+0000","last_fullsized":"2024-03-30T13:11:01.855446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:56:01.049552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857815+0000","last_change":"2024-03-30T13:11:00.020015+0000","last_active":"2024-03-30T13:11:01.857815+0000","last_peered":"2024-03-30T13:11:01.857815+0000","last_clean":"2024-03-30T13:11:01.857815+0000","last_became_active":"2024-03-30T13:11:00.019563+0000","last_became_peered":"2024-03-30T13:11:00.019563+0000","last_unstale":"2024-03-30T13:11:01.857815+0000","last_undegraded":"2024-03-30T13:11:01.857815+0000","last_fullsized":"2024-03-30T13:11:01.857815+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:44.852627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854082+0000","last_change":"2024-03-30T13:10:55.972256+0000","last_active":"2024-03-30T13:11:01.854082+0000","last_peered":"2024-03-30T13:11:01.854082+0000","last_clean":"2024-03-30T13:11:01.854082+0000","last_became_active":"2024-03-30T13:10:55.971884+0000","last_became_peered":"2024-03-30T13:10:55.971884+0000","last_unstale":"2024-03-30T13:11:01.854082+0000","last_undegraded":"2024-03-30T13:11:01.854082+0000","last_fullsized":"2024-03-30T13:11:01.854082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:47:49.750989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854680+0000","last_change":"2024-03-30T13:10:54.932797+0000","last_active":"2024-03-30T13:11:01.854680+0000","last_peered":"2024-03-30T13:11:01.854680+0000","last_clean":"2024-03-30T13:11:01.854680+0000","last_became_active":"2024-03-30T13:10:54.932392+0000","last_became_peered":"2024-03-30T13:10:54.932392+0000","last_unstale":"2024-03-30T13:11:01.854680+0000","last_undegraded":"2024-03-30T13:11:01.854680+0000","last_fullsized":"2024-03-30T13:11:01.854680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855304+0000","last_change":"2024-03-30T13:11:00.005113+0000","last_active":"2024-03-30T13:11:01.855304+0000","last_peered":"2024-03-30T13:11:01.855304+0000","last_clean":"2024-03-30T13:11:01.855304+0000","last_became_active":"2024-03-30T13:11:00.004701+0000","last_became_peered":"2024-03-30T13:11:00.004701+0000","last_unstale":"2024-03-30T13:11:01.855304+0000","last_undegraded":"2024-03-30T13:11:01.855304+0000","last_fullsized":"2024-03-30T13:11:01.855304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:12:46.188048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859494+0000","last_change":"2024-03-30T13:10:55.993733+0000","last_active":"2024-03-30T13:11:01.859494+0000","last_peered":"2024-03-30T13:11:01.859494+0000","last_clean":"2024-03-30T13:11:01.859494+0000","last_became_active":"2024-03-30T13:10:55.993012+0000","last_became_peered":"2024-03-30T13:10:55.993012+0000","last_unstale":"2024-03-30T13:11:01.859494+0000","last_undegraded":"2024-03-30T13:11:01.859494+0000","last_fullsized":"2024-03-30T13:11:01.859494+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980295+0000","last_change":"2024-03-30T13:10:54.947616+0000","last_active":"2024-03-30T13:11:00.980295+0000","last_peered":"2024-03-30T13:11:00.980295+0000","last_clean":"2024-03-30T13:11:00.980295+0000","last_became_active":"2024-03-30T13:10:54.946695+0000","last_became_peered":"2024-03-30T13:10:54.946695+0000","last_unstale":"2024-03-30T13:11:00.980295+0000","last_undegraded":"2024-03-30T13:11:00.980295+0000","last_fullsized":"2024-03-30T13:11:00.980295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855501+0000","last_change":"2024-03-30T13:11:00.019814+0000","last_active":"2024-03-30T13:11:01.855501+0000","last_peered":"2024-03-30T13:11:01.855501+0000","last_clean":"2024-03-30T13:11:01.855501+0000","last_became_active":"2024-03-30T13:11:00.019359+0000","last_became_peered":"2024-03-30T13:11:00.019359+0000","last_unstale":"2024-03-30T13:11:01.855501+0000","last_undegraded":"2024-03-30T13:11:01.855501+0000","last_fullsized":"2024-03-30T13:11:01.855501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:40:01.204781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855194+0000","last_change":"2024-03-30T13:10:56.004578+0000","last_active":"2024-03-30T13:11:01.855194+0000","last_peered":"2024-03-30T13:11:01.855194+0000","last_clean":"2024-03-30T13:11:01.855194+0000","last_became_active":"2024-03-30T13:10:56.003914+0000","last_became_peered":"2024-03-30T13:10:56.003914+0000","last_unstale":"2024-03-30T13:11:01.855194+0000","last_undegraded":"2024-03-30T13:11:01.855194+0000","last_fullsized":"2024-03-30T13:11:01.855194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856085+0000","last_change":"2024-03-30T13:10:54.948212+0000","last_active":"2024-03-30T13:11:01.856085+0000","last_peered":"2024-03-30T13:11:01.856085+0000","last_clean":"2024-03-30T13:11:01.856085+0000","last_became_active":"2024-03-30T13:10:54.947336+0000","last_became_peered":"2024-03-30T13:10:54.947336+0000","last_unstale":"2024-03-30T13:11:01.856085+0000","last_undegraded":"2024-03-30T13:11:01.856085+0000","last_fullsized":"2024-03-30T13:11:01.856085+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859925+0000","last_change":"2024-03-30T13:11:00.051626+0000","last_active":"2024-03-30T13:11:01.859925+0000","last_peered":"2024-03-30T13:11:01.859925+0000","last_clean":"2024-03-30T13:11:01.859925+0000","last_became_active":"2024-03-30T13:11:00.050798+0000","last_became_peered":"2024-03-30T13:11:00.050798+0000","last_unstale":"2024-03-30T13:11:01.859925+0000","last_undegraded":"2024-03-30T13:11:01.859925+0000","last_fullsized":"2024-03-30T13:11:01.859925+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:15.648737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853752+0000","last_change":"2024-03-30T13:10:55.965723+0000","last_active":"2024-03-30T13:11:01.853752+0000","last_peered":"2024-03-30T13:11:01.853752+0000","last_clean":"2024-03-30T13:11:01.853752+0000","last_became_active":"2024-03-30T13:10:55.965433+0000","last_became_peered":"2024-03-30T13:10:55.965433+0000","last_unstale":"2024-03-30T13:11:01.853752+0000","last_undegraded":"2024-03-30T13:11:01.853752+0000","last_fullsized":"2024-03-30T13:11:01.853752+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854998+0000","last_change":"2024-03-30T13:10:54.940390+0000","last_active":"2024-03-30T13:11:01.854998+0000","last_peered":"2024-03-30T13:11:01.854998+0000","last_clean":"2024-03-30T13:11:01.854998+0000","last_became_active":"2024-03-30T13:10:54.939980+0000","last_became_peered":"2024-03-30T13:10:54.939980+0000","last_unstale":"2024-03-30T13:11:01.854998+0000","last_undegraded":"2024-03-30T13:11:01.854998+0000","last_fullsized":"2024-03-30T13:11:01.854998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858329+0000","last_change":"2024-03-30T13:11:00.019922+0000","last_active":"2024-03-30T13:11:01.858329+0000","last_peered":"2024-03-30T13:11:01.858329+0000","last_clean":"2024-03-30T13:11:01.858329+0000","last_became_active":"2024-03-30T13:11:00.019491+0000","last_became_peered":"2024-03-30T13:11:00.019491+0000","last_unstale":"2024-03-30T13:11:01.858329+0000","last_undegraded":"2024-03-30T13:11:01.858329+0000","last_fullsized":"2024-03-30T13:11:01.858329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856375+0000","last_change":"2024-03-30T13:10:55.968355+0000","last_active":"2024-03-30T13:11:01.856375+0000","last_peered":"2024-03-30T13:11:01.856375+0000","last_clean":"2024-03-30T13:11:01.856375+0000","last_became_active":"2024-03-30T13:10:55.965410+0000","last_became_peered":"2024-03-30T13:10:55.965410+0000","last_unstale":"2024-03-30T13:11:01.856375+0000","last_undegraded":"2024-03-30T13:11:01.856375+0000","last_fullsized":"2024-03-30T13:11:01.856375+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:47:03.240883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855058+0000","last_change":"2024-03-30T13:10:54.944101+0000","last_active":"2024-03-30T13:11:01.855058+0000","last_peered":"2024-03-30T13:11:01.855058+0000","last_clean":"2024-03-30T13:11:01.855058+0000","last_became_active":"2024-03-30T13:10:54.943588+0000","last_became_peered":"2024-03-30T13:10:54.943588+0000","last_unstale":"2024-03-30T13:11:01.855058+0000","last_undegraded":"2024-03-30T13:11:01.855058+0000","last_fullsized":"2024-03-30T13:11:01.855058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:28.393686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857849+0000","last_change":"2024-03-30T13:11:00.009582+0000","last_active":"2024-03-30T13:11:01.857849+0000","last_peered":"2024-03-30T13:11:01.857849+0000","last_clean":"2024-03-30T13:11:01.857849+0000","last_became_active":"2024-03-30T13:11:00.009033+0000","last_became_peered":"2024-03-30T13:11:00.009033+0000","last_unstale":"2024-03-30T13:11:01.857849+0000","last_undegraded":"2024-03-30T13:11:01.857849+0000","last_fullsized":"2024-03-30T13:11:01.857849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856879+0000","last_change":"2024-03-30T13:10:56.002735+0000","last_active":"2024-03-30T13:11:01.856879+0000","last_peered":"2024-03-30T13:11:01.856879+0000","last_clean":"2024-03-30T13:11:01.856879+0000","last_became_active":"2024-03-30T13:10:56.001973+0000","last_became_peered":"2024-03-30T13:10:56.001973+0000","last_unstale":"2024-03-30T13:11:01.856879+0000","last_undegraded":"2024-03-30T13:11:01.856879+0000","last_fullsized":"2024-03-30T13:11:01.856879+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856685+0000","last_change":"2024-03-30T13:10:54.947650+0000","last_active":"2024-03-30T13:11:01.856685+0000","last_peered":"2024-03-30T13:11:01.856685+0000","last_clean":"2024-03-30T13:11:01.856685+0000","last_became_active":"2024-03-30T13:10:54.946378+0000","last_became_peered":"2024-03-30T13:10:54.946378+0000","last_unstale":"2024-03-30T13:11:01.856685+0000","last_undegraded":"2024-03-30T13:11:01.856685+0000","last_fullsized":"2024-03-30T13:11:01.856685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855829+0000","last_change":"2024-03-30T13:11:00.031425+0000","last_active":"2024-03-30T13:11:01.855829+0000","last_peered":"2024-03-30T13:11:01.855829+0000","last_clean":"2024-03-30T13:11:01.855829+0000","last_became_active":"2024-03-30T13:11:00.030689+0000","last_became_peered":"2024-03-30T13:11:00.030689+0000","last_unstale":"2024-03-30T13:11:01.855829+0000","last_undegraded":"2024-03-30T13:11:01.855829+0000","last_fullsized":"2024-03-30T13:11:01.855829+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:50.405876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981455+0000","last_change":"2024-03-30T13:10:55.991469+0000","last_active":"2024-03-30T13:11:00.981455+0000","last_peered":"2024-03-30T13:11:00.981455+0000","last_clean":"2024-03-30T13:11:00.981455+0000","last_became_active":"2024-03-30T13:10:55.990596+0000","last_became_peered":"2024-03-30T13:10:55.990596+0000","last_unstale":"2024-03-30T13:11:00.981455+0000","last_undegraded":"2024-03-30T13:11:00.981455+0000","last_fullsized":"2024-03-30T13:11:00.981455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856131+0000","last_change":"2024-03-30T13:10:54.945048+0000","last_active":"2024-03-30T13:11:01.856131+0000","last_peered":"2024-03-30T13:11:01.856131+0000","last_clean":"2024-03-30T13:11:01.856131+0000","last_became_active":"2024-03-30T13:10:54.944552+0000","last_became_peered":"2024-03-30T13:10:54.944552+0000","last_unstale":"2024-03-30T13:11:01.856131+0000","last_undegraded":"2024-03-30T13:11:01.856131+0000","last_fullsized":"2024-03-30T13:11:01.856131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:08.826146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980755+0000","last_change":"2024-03-30T13:11:00.038523+0000","last_active":"2024-03-30T13:11:00.980755+0000","last_peered":"2024-03-30T13:11:00.980755+0000","last_clean":"2024-03-30T13:11:00.980755+0000","last_became_active":"2024-03-30T13:11:00.037276+0000","last_became_peered":"2024-03-30T13:11:00.037276+0000","last_unstale":"2024-03-30T13:11:00.980755+0000","last_undegraded":"2024-03-30T13:11:00.980755+0000","last_fullsized":"2024-03-30T13:11:00.980755+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858113+0000","last_change":"2024-03-30T13:10:55.992831+0000","last_active":"2024-03-30T13:11:01.858113+0000","last_peered":"2024-03-30T13:11:01.858113+0000","last_clean":"2024-03-30T13:11:01.858113+0000","last_became_active":"2024-03-30T13:10:55.992521+0000","last_became_peered":"2024-03-30T13:10:55.992521+0000","last_unstale":"2024-03-30T13:11:01.858113+0000","last_undegraded":"2024-03-30T13:11:01.858113+0000","last_fullsized":"2024-03-30T13:11:01.858113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979972+0000","last_change":"2024-03-30T13:10:54.952220+0000","last_active":"2024-03-30T13:11:00.979972+0000","last_peered":"2024-03-30T13:11:00.979972+0000","last_clean":"2024-03-30T13:11:00.979972+0000","last_became_active":"2024-03-30T13:10:54.951484+0000","last_became_peered":"2024-03-30T13:10:54.951484+0000","last_unstale":"2024-03-30T13:11:00.979972+0000","last_undegraded":"2024-03-30T13:11:00.979972+0000","last_fullsized":"2024-03-30T13:11:00.979972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:11:24.184946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979274+0000","last_change":"2024-03-30T13:11:00.043402+0000","last_active":"2024-03-30T13:11:00.979274+0000","last_peered":"2024-03-30T13:11:00.979274+0000","last_clean":"2024-03-30T13:11:00.979274+0000","last_became_active":"2024-03-30T13:11:00.040341+0000","last_became_peered":"2024-03-30T13:11:00.040341+0000","last_unstale":"2024-03-30T13:11:00.979274+0000","last_undegraded":"2024-03-30T13:11:00.979274+0000","last_fullsized":"2024-03-30T13:11:00.979274+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:58.538875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855368+0000","last_change":"2024-03-30T13:10:55.958140+0000","last_active":"2024-03-30T13:11:01.855368+0000","last_peered":"2024-03-30T13:11:01.855368+0000","last_clean":"2024-03-30T13:11:01.855368+0000","last_became_active":"2024-03-30T13:10:55.957860+0000","last_became_peered":"2024-03-30T13:10:55.957860+0000","last_unstale":"2024-03-30T13:11:01.855368+0000","last_undegraded":"2024-03-30T13:11:01.855368+0000","last_fullsized":"2024-03-30T13:11:01.855368+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:47:54.477311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854336+0000","last_change":"2024-03-30T13:10:54.945433+0000","last_active":"2024-03-30T13:11:01.854336+0000","last_peered":"2024-03-30T13:11:01.854336+0000","last_clean":"2024-03-30T13:11:01.854336+0000","last_became_active":"2024-03-30T13:10:54.945133+0000","last_became_peered":"2024-03-30T13:10:54.945133+0000","last_unstale":"2024-03-30T13:11:01.854336+0000","last_undegraded":"2024-03-30T13:11:01.854336+0000","last_fullsized":"2024-03-30T13:11:01.854336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:21:59.514214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980453+0000","last_change":"2024-03-30T13:11:00.044030+0000","last_active":"2024-03-30T13:11:00.980453+0000","last_peered":"2024-03-30T13:11:00.980453+0000","last_clean":"2024-03-30T13:11:00.980453+0000","last_became_active":"2024-03-30T13:11:00.037702+0000","last_became_peered":"2024-03-30T13:11:00.037702+0000","last_unstale":"2024-03-30T13:11:00.980453+0000","last_undegraded":"2024-03-30T13:11:00.980453+0000","last_fullsized":"2024-03-30T13:11:00.980453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:37.101347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979645+0000","last_change":"2024-03-30T13:10:55.962616+0000","last_active":"2024-03-30T13:11:00.979645+0000","last_peered":"2024-03-30T13:11:00.979645+0000","last_clean":"2024-03-30T13:11:00.979645+0000","last_became_active":"2024-03-30T13:10:55.961327+0000","last_became_peered":"2024-03-30T13:10:55.961327+0000","last_unstale":"2024-03-30T13:11:00.979645+0000","last_undegraded":"2024-03-30T13:11:00.979645+0000","last_fullsized":"2024-03-30T13:11:00.979645+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:17.511817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858373+0000","last_change":"2024-03-30T13:10:54.942851+0000","last_active":"2024-03-30T13:11:01.858373+0000","last_peered":"2024-03-30T13:11:01.858373+0000","last_clean":"2024-03-30T13:11:01.858373+0000","last_became_active":"2024-03-30T13:10:54.939805+0000","last_became_peered":"2024-03-30T13:10:54.939805+0000","last_unstale":"2024-03-30T13:11:01.858373+0000","last_undegraded":"2024-03-30T13:11:01.858373+0000","last_fullsized":"2024-03-30T13:11:01.858373+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:24.569683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982246+0000","last_change":"2024-03-30T13:11:00.040718+0000","last_active":"2024-03-30T13:11:00.982246+0000","last_peered":"2024-03-30T13:11:00.982246+0000","last_clean":"2024-03-30T13:11:00.982246+0000","last_became_active":"2024-03-30T13:11:00.039372+0000","last_became_peered":"2024-03-30T13:11:00.039372+0000","last_unstale":"2024-03-30T13:11:00.982246+0000","last_undegraded":"2024-03-30T13:11:00.982246+0000","last_fullsized":"2024-03-30T13:11:00.982246+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:26.483027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979060+0000","last_change":"2024-03-30T13:10:55.984056+0000","last_active":"2024-03-30T13:11:00.979060+0000","last_peered":"2024-03-30T13:11:00.979060+0000","last_clean":"2024-03-30T13:11:00.979060+0000","last_became_active":"2024-03-30T13:10:55.974450+0000","last_became_peered":"2024-03-30T13:10:55.974450+0000","last_unstale":"2024-03-30T13:11:00.979060+0000","last_undegraded":"2024-03-30T13:11:00.979060+0000","last_fullsized":"2024-03-30T13:11:00.979060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853918+0000","last_change":"2024-03-30T13:10:54.942183+0000","last_active":"2024-03-30T13:11:01.853918+0000","last_peered":"2024-03-30T13:11:01.853918+0000","last_clean":"2024-03-30T13:11:01.853918+0000","last_became_active":"2024-03-30T13:10:54.941923+0000","last_became_peered":"2024-03-30T13:10:54.941923+0000","last_unstale":"2024-03-30T13:11:01.853918+0000","last_undegraded":"2024-03-30T13:11:01.853918+0000","last_fullsized":"2024-03-30T13:11:01.853918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:39.803698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983242+0000","last_change":"2024-03-30T13:11:00.024121+0000","last_active":"2024-03-30T13:11:00.983242+0000","last_peered":"2024-03-30T13:11:00.983242+0000","last_clean":"2024-03-30T13:11:00.983242+0000","last_became_active":"2024-03-30T13:11:00.022137+0000","last_became_peered":"2024-03-30T13:11:00.022137+0000","last_unstale":"2024-03-30T13:11:00.983242+0000","last_undegraded":"2024-03-30T13:11:00.983242+0000","last_fullsized":"2024-03-30T13:11:00.983242+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:08:16.219711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856025+0000","last_change":"2024-03-30T13:10:55.978563+0000","last_active":"2024-03-30T13:11:01.856025+0000","last_peered":"2024-03-30T13:11:01.856025+0000","last_clean":"2024-03-30T13:11:01.856025+0000","last_became_active":"2024-03-30T13:10:55.973372+0000","last_became_peered":"2024-03-30T13:10:55.973372+0000","last_unstale":"2024-03-30T13:11:01.856025+0000","last_undegraded":"2024-03-30T13:11:01.856025+0000","last_fullsized":"2024-03-30T13:11:01.856025+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854645+0000","last_change":"2024-03-30T13:10:54.932742+0000","last_active":"2024-03-30T13:11:01.854645+0000","last_peered":"2024-03-30T13:11:01.854645+0000","last_clean":"2024-03-30T13:11:01.854645+0000","last_became_active":"2024-03-30T13:10:54.932326+0000","last_became_peered":"2024-03-30T13:10:54.932326+0000","last_unstale":"2024-03-30T13:11:01.854645+0000","last_undegraded":"2024-03-30T13:11:01.854645+0000","last_fullsized":"2024-03-30T13:11:01.854645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977500+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:11:00.977500+0000","last_peered":"2024-03-30T13:11:00.977500+0000","last_clean":"2024-03-30T13:11:00.977500+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:11:00.977500+0000","last_undegraded":"2024-03-30T13:11:00.977500+0000","last_fullsized":"2024-03-30T13:11:00.977500+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856097+0000","last_change":"2024-03-30T13:10:59.992352+0000","last_active":"2024-03-30T13:11:01.856097+0000","last_peered":"2024-03-30T13:11:01.856097+0000","last_clean":"2024-03-30T13:11:01.856097+0000","last_became_active":"2024-03-30T13:10:59.991915+0000","last_became_peered":"2024-03-30T13:10:59.991915+0000","last_unstale":"2024-03-30T13:11:01.856097+0000","last_undegraded":"2024-03-30T13:11:01.856097+0000","last_fullsized":"2024-03-30T13:11:01.856097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:13:07.419696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855684+0000","last_change":"2024-03-30T13:10:55.971288+0000","last_active":"2024-03-30T13:11:01.855684+0000","last_peered":"2024-03-30T13:11:01.855684+0000","last_clean":"2024-03-30T13:11:01.855684+0000","last_became_active":"2024-03-30T13:10:55.970980+0000","last_became_peered":"2024-03-30T13:10:55.970980+0000","last_unstale":"2024-03-30T13:11:01.855684+0000","last_undegraded":"2024-03-30T13:11:01.855684+0000","last_fullsized":"2024-03-30T13:11:01.855684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:57.143218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980646+0000","last_change":"2024-03-30T13:10:54.946952+0000","last_active":"2024-03-30T13:11:00.980646+0000","last_peered":"2024-03-30T13:11:00.980646+0000","last_clean":"2024-03-30T13:11:00.980646+0000","last_became_active":"2024-03-30T13:10:54.946235+0000","last_became_peered":"2024-03-30T13:10:54.946235+0000","last_unstale":"2024-03-30T13:11:00.980646+0000","last_undegraded":"2024-03-30T13:11:00.980646+0000","last_fullsized":"2024-03-30T13:11:00.980646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857845+0000","last_change":"2024-03-30T13:11:00.017769+0000","last_active":"2024-03-30T13:11:01.857845+0000","last_peered":"2024-03-30T13:11:01.857845+0000","last_clean":"2024-03-30T13:11:01.857845+0000","last_became_active":"2024-03-30T13:11:00.017317+0000","last_became_peered":"2024-03-30T13:11:00.017317+0000","last_unstale":"2024-03-30T13:11:01.857845+0000","last_undegraded":"2024-03-30T13:11:01.857845+0000","last_fullsized":"2024-03-30T13:11:01.857845+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:11:01.211799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.963559+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.963354+0000","last_became_peered":"2024-03-30T13:10:55.963354+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981372+0000","last_change":"2024-03-30T13:10:54.942041+0000","last_active":"2024-03-30T13:11:00.981372+0000","last_peered":"2024-03-30T13:11:00.981372+0000","last_clean":"2024-03-30T13:11:00.981372+0000","last_became_active":"2024-03-30T13:10:54.940995+0000","last_became_peered":"2024-03-30T13:10:54.940995+0000","last_unstale":"2024-03-30T13:11:00.981372+0000","last_undegraded":"2024-03-30T13:11:00.981372+0000","last_fullsized":"2024-03-30T13:11:00.981372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:54:21.415837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982091+0000","last_change":"2024-03-30T13:11:00.010520+0000","last_active":"2024-03-30T13:11:00.982091+0000","last_peered":"2024-03-30T13:11:00.982091+0000","last_clean":"2024-03-30T13:11:00.982091+0000","last_became_active":"2024-03-30T13:11:00.009785+0000","last_became_peered":"2024-03-30T13:11:00.009785+0000","last_unstale":"2024-03-30T13:11:00.982091+0000","last_undegraded":"2024-03-30T13:11:00.982091+0000","last_fullsized":"2024-03-30T13:11:00.982091+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:13:10.451556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856333+0000","last_change":"2024-03-30T13:10:55.979699+0000","last_active":"2024-03-30T13:11:01.856333+0000","last_peered":"2024-03-30T13:11:01.856333+0000","last_clean":"2024-03-30T13:11:01.856333+0000","last_became_active":"2024-03-30T13:10:55.979427+0000","last_became_peered":"2024-03-30T13:10:55.979427+0000","last_unstale":"2024-03-30T13:11:01.856333+0000","last_undegraded":"2024-03-30T13:11:01.856333+0000","last_fullsized":"2024-03-30T13:11:01.856333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856586+0000","last_change":"2024-03-30T13:10:54.945823+0000","last_active":"2024-03-30T13:11:01.856586+0000","last_peered":"2024-03-30T13:11:01.856586+0000","last_clean":"2024-03-30T13:11:01.856586+0000","last_became_active":"2024-03-30T13:10:54.944624+0000","last_became_peered":"2024-03-30T13:10:54.944624+0000","last_unstale":"2024-03-30T13:11:01.856586+0000","last_undegraded":"2024-03-30T13:11:01.856586+0000","last_fullsized":"2024-03-30T13:11:01.856586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853761+0000","last_change":"2024-03-30T13:11:00.002695+0000","last_active":"2024-03-30T13:11:01.853761+0000","last_peered":"2024-03-30T13:11:01.853761+0000","last_clean":"2024-03-30T13:11:01.853761+0000","last_became_active":"2024-03-30T13:11:00.002251+0000","last_became_peered":"2024-03-30T13:11:00.002251+0000","last_unstale":"2024-03-30T13:11:01.853761+0000","last_undegraded":"2024-03-30T13:11:01.853761+0000","last_fullsized":"2024-03-30T13:11:01.853761+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:23:48.725109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858389+0000","last_change":"2024-03-30T13:10:55.966115+0000","last_active":"2024-03-30T13:11:01.858389+0000","last_peered":"2024-03-30T13:11:01.858389+0000","last_clean":"2024-03-30T13:11:01.858389+0000","last_became_active":"2024-03-30T13:10:55.965541+0000","last_became_peered":"2024-03-30T13:10:55.965541+0000","last_unstale":"2024-03-30T13:11:01.858389+0000","last_undegraded":"2024-03-30T13:11:01.858389+0000","last_fullsized":"2024-03-30T13:11:01.858389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856413+0000","last_change":"2024-03-30T13:10:54.945660+0000","last_active":"2024-03-30T13:11:01.856413+0000","last_peered":"2024-03-30T13:11:01.856413+0000","last_clean":"2024-03-30T13:11:01.856413+0000","last_became_active":"2024-03-30T13:10:54.944892+0000","last_became_peered":"2024-03-30T13:10:54.944892+0000","last_unstale":"2024-03-30T13:11:01.856413+0000","last_undegraded":"2024-03-30T13:11:01.856413+0000","last_fullsized":"2024-03-30T13:11:01.856413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977650+0000","last_change":"2024-03-30T13:11:00.014434+0000","last_active":"2024-03-30T13:11:00.977650+0000","last_peered":"2024-03-30T13:11:00.977650+0000","last_clean":"2024-03-30T13:11:00.977650+0000","last_became_active":"2024-03-30T13:11:00.013995+0000","last_became_peered":"2024-03-30T13:11:00.013995+0000","last_unstale":"2024-03-30T13:11:00.977650+0000","last_undegraded":"2024-03-30T13:11:00.977650+0000","last_fullsized":"2024-03-30T13:11:00.977650+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:13.896277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853679+0000","last_change":"2024-03-30T13:10:55.960324+0000","last_active":"2024-03-30T13:11:01.853679+0000","last_peered":"2024-03-30T13:11:01.853679+0000","last_clean":"2024-03-30T13:11:01.853679+0000","last_became_active":"2024-03-30T13:10:55.958877+0000","last_became_peered":"2024-03-30T13:10:55.958877+0000","last_unstale":"2024-03-30T13:11:01.853679+0000","last_undegraded":"2024-03-30T13:11:01.853679+0000","last_fullsized":"2024-03-30T13:11:01.853679+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855512+0000","last_change":"2024-03-30T13:10:54.943074+0000","last_active":"2024-03-30T13:11:01.855512+0000","last_peered":"2024-03-30T13:11:01.855512+0000","last_clean":"2024-03-30T13:11:01.855512+0000","last_became_active":"2024-03-30T13:10:54.941870+0000","last_became_peered":"2024-03-30T13:10:54.941870+0000","last_unstale":"2024-03-30T13:11:01.855512+0000","last_undegraded":"2024-03-30T13:11:01.855512+0000","last_fullsized":"2024-03-30T13:11:01.855512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:02.053534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983714+0000","last_change":"2024-03-30T13:11:00.023257+0000","last_active":"2024-03-30T13:11:00.983714+0000","last_peered":"2024-03-30T13:11:00.983714+0000","last_clean":"2024-03-30T13:11:00.983714+0000","last_became_active":"2024-03-30T13:11:00.020654+0000","last_became_peered":"2024-03-30T13:11:00.020654+0000","last_unstale":"2024-03-30T13:11:00.983714+0000","last_undegraded":"2024-03-30T13:11:00.983714+0000","last_fullsized":"2024-03-30T13:11:00.983714+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:16:15.446866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977511+0000","last_change":"2024-03-30T13:10:55.970389+0000","last_active":"2024-03-30T13:11:00.977511+0000","last_peered":"2024-03-30T13:11:00.977511+0000","last_clean":"2024-03-30T13:11:00.977511+0000","last_became_active":"2024-03-30T13:10:55.969686+0000","last_became_peered":"2024-03-30T13:10:55.969686+0000","last_unstale":"2024-03-30T13:11:00.977511+0000","last_undegraded":"2024-03-30T13:11:00.977511+0000","last_fullsized":"2024-03-30T13:11:00.977511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855488+0000","last_change":"2024-03-30T13:10:54.945121+0000","last_active":"2024-03-30T13:11:01.855488+0000","last_peered":"2024-03-30T13:11:01.855488+0000","last_clean":"2024-03-30T13:11:01.855488+0000","last_became_active":"2024-03-30T13:10:54.944583+0000","last_became_peered":"2024-03-30T13:10:54.944583+0000","last_unstale":"2024-03-30T13:11:01.855488+0000","last_undegraded":"2024-03-30T13:11:01.855488+0000","last_fullsized":"2024-03-30T13:11:01.855488+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981873+0000","last_change":"2024-03-30T13:11:00.023347+0000","last_active":"2024-03-30T13:11:00.981873+0000","last_peered":"2024-03-30T13:11:00.981873+0000","last_clean":"2024-03-30T13:11:00.981873+0000","last_became_active":"2024-03-30T13:11:00.021082+0000","last_became_peered":"2024-03-30T13:11:00.021082+0000","last_unstale":"2024-03-30T13:11:00.981873+0000","last_undegraded":"2024-03-30T13:11:00.981873+0000","last_fullsized":"2024-03-30T13:11:00.981873+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:47:39.699905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854454+0000","last_change":"2024-03-30T13:10:55.964652+0000","last_active":"2024-03-30T13:11:01.854454+0000","last_peered":"2024-03-30T13:11:01.854454+0000","last_clean":"2024-03-30T13:11:01.854454+0000","last_became_active":"2024-03-30T13:10:55.964363+0000","last_became_peered":"2024-03-30T13:10:55.964363+0000","last_unstale":"2024-03-30T13:11:01.854454+0000","last_undegraded":"2024-03-30T13:11:01.854454+0000","last_fullsized":"2024-03-30T13:11:01.854454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:17.023431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854282+0000","last_change":"2024-03-30T13:10:54.936477+0000","last_active":"2024-03-30T13:11:01.854282+0000","last_peered":"2024-03-30T13:11:01.854282+0000","last_clean":"2024-03-30T13:11:01.854282+0000","last_became_active":"2024-03-30T13:10:54.935689+0000","last_became_peered":"2024-03-30T13:10:54.935689+0000","last_unstale":"2024-03-30T13:11:01.854282+0000","last_undegraded":"2024-03-30T13:11:01.854282+0000","last_fullsized":"2024-03-30T13:11:01.854282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:28.044884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980099+0000","last_change":"2024-03-30T13:11:00.044614+0000","last_active":"2024-03-30T13:11:00.980099+0000","last_peered":"2024-03-30T13:11:00.980099+0000","last_clean":"2024-03-30T13:11:00.980099+0000","last_became_active":"2024-03-30T13:11:00.039909+0000","last_became_peered":"2024-03-30T13:11:00.039909+0000","last_unstale":"2024-03-30T13:11:00.980099+0000","last_undegraded":"2024-03-30T13:11:00.980099+0000","last_fullsized":"2024-03-30T13:11:00.980099+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:44.050582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979980+0000","last_change":"2024-03-30T13:10:55.985354+0000","last_active":"2024-03-30T13:11:00.979980+0000","last_peered":"2024-03-30T13:11:00.979980+0000","last_clean":"2024-03-30T13:11:00.979980+0000","last_became_active":"2024-03-30T13:10:55.974918+0000","last_became_peered":"2024-03-30T13:10:55.974918+0000","last_unstale":"2024-03-30T13:11:00.979980+0000","last_undegraded":"2024-03-30T13:11:00.979980+0000","last_fullsized":"2024-03-30T13:11:00.979980+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:18:29.402254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859297+0000","last_change":"2024-03-30T13:10:54.955637+0000","last_active":"2024-03-30T13:11:01.859297+0000","last_peered":"2024-03-30T13:11:01.859297+0000","last_clean":"2024-03-30T13:11:01.859297+0000","last_became_active":"2024-03-30T13:10:54.949281+0000","last_became_peered":"2024-03-30T13:10:54.949281+0000","last_unstale":"2024-03-30T13:11:01.859297+0000","last_undegraded":"2024-03-30T13:11:01.859297+0000","last_fullsized":"2024-03-30T13:11:01.859297+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:01:24.807581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855707+0000","last_change":"2024-03-30T13:11:00.015834+0000","last_active":"2024-03-30T13:11:01.855707+0000","last_peered":"2024-03-30T13:11:01.855707+0000","last_clean":"2024-03-30T13:11:01.855707+0000","last_became_active":"2024-03-30T13:11:00.015344+0000","last_became_peered":"2024-03-30T13:11:00.015344+0000","last_unstale":"2024-03-30T13:11:01.855707+0000","last_undegraded":"2024-03-30T13:11:01.855707+0000","last_fullsized":"2024-03-30T13:11:01.855707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:43.288754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855890+0000","last_change":"2024-03-30T13:10:55.993807+0000","last_active":"2024-03-30T13:11:01.855890+0000","last_peered":"2024-03-30T13:11:01.855890+0000","last_clean":"2024-03-30T13:11:01.855890+0000","last_became_active":"2024-03-30T13:10:55.993589+0000","last_became_peered":"2024-03-30T13:10:55.993589+0000","last_unstale":"2024-03-30T13:11:01.855890+0000","last_undegraded":"2024-03-30T13:11:01.855890+0000","last_fullsized":"2024-03-30T13:11:01.855890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:10:57.150236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:06.622780+0000","last_change":"2024-03-30T13:10:54.939893+0000","last_active":"2024-03-30T13:11:06.622780+0000","last_peered":"2024-03-30T13:11:06.622780+0000","last_clean":"2024-03-30T13:11:06.622780+0000","last_became_active":"2024-03-30T13:10:54.939427+0000","last_became_peered":"2024-03-30T13:10:54.939427+0000","last_unstale":"2024-03-30T13:11:06.622780+0000","last_undegraded":"2024-03-30T13:11:06.622780+0000","last_fullsized":"2024-03-30T13:11:06.622780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:26:07.195072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855852+0000","last_change":"2024-03-30T13:11:00.013757+0000","last_active":"2024-03-30T13:11:01.855852+0000","last_peered":"2024-03-30T13:11:01.855852+0000","last_clean":"2024-03-30T13:11:01.855852+0000","last_became_active":"2024-03-30T13:11:00.012615+0000","last_became_peered":"2024-03-30T13:11:00.012615+0000","last_unstale":"2024-03-30T13:11:01.855852+0000","last_undegraded":"2024-03-30T13:11:01.855852+0000","last_fullsized":"2024-03-30T13:11:01.855852+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:02:00.559037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853971+0000","last_change":"2024-03-30T13:10:55.957241+0000","last_active":"2024-03-30T13:11:01.853971+0000","last_peered":"2024-03-30T13:11:01.853971+0000","last_clean":"2024-03-30T13:11:01.853971+0000","last_became_active":"2024-03-30T13:10:55.956872+0000","last_became_peered":"2024-03-30T13:10:55.956872+0000","last_unstale":"2024-03-30T13:11:01.853971+0000","last_undegraded":"2024-03-30T13:11:01.853971+0000","last_fullsized":"2024-03-30T13:11:01.853971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:00.663013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859629+0000","last_change":"2024-03-30T13:10:54.947319+0000","last_active":"2024-03-30T13:11:01.859629+0000","last_peered":"2024-03-30T13:11:01.859629+0000","last_clean":"2024-03-30T13:11:01.859629+0000","last_became_active":"2024-03-30T13:10:54.946638+0000","last_became_peered":"2024-03-30T13:10:54.946638+0000","last_unstale":"2024-03-30T13:11:01.859629+0000","last_undegraded":"2024-03-30T13:11:01.859629+0000","last_fullsized":"2024-03-30T13:11:01.859629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:00:58.056083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858359+0000","last_change":"2024-03-30T13:11:00.032095+0000","last_active":"2024-03-30T13:11:01.858359+0000","last_peered":"2024-03-30T13:11:01.858359+0000","last_clean":"2024-03-30T13:11:01.858359+0000","last_became_active":"2024-03-30T13:11:00.031651+0000","last_became_peered":"2024-03-30T13:11:00.031651+0000","last_unstale":"2024-03-30T13:11:01.858359+0000","last_undegraded":"2024-03-30T13:11:01.858359+0000","last_fullsized":"2024-03-30T13:11:01.858359+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:21.426288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981894+0000","last_change":"2024-03-30T13:10:55.963310+0000","last_active":"2024-03-30T13:11:00.981894+0000","last_peered":"2024-03-30T13:11:00.981894+0000","last_clean":"2024-03-30T13:11:00.981894+0000","last_became_active":"2024-03-30T13:10:55.962608+0000","last_became_peered":"2024-03-30T13:10:55.962608+0000","last_unstale":"2024-03-30T13:11:00.981894+0000","last_undegraded":"2024-03-30T13:11:00.981894+0000","last_fullsized":"2024-03-30T13:11:00.981894+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:53.368510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855969+0000","last_change":"2024-03-30T13:10:54.937448+0000","last_active":"2024-03-30T13:11:01.855969+0000","last_peered":"2024-03-30T13:11:01.855969+0000","last_clean":"2024-03-30T13:11:01.855969+0000","last_became_active":"2024-03-30T13:10:54.936836+0000","last_became_peered":"2024-03-30T13:10:54.936836+0000","last_unstale":"2024-03-30T13:11:01.855969+0000","last_undegraded":"2024-03-30T13:11:01.855969+0000","last_fullsized":"2024-03-30T13:11:01.855969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:50.370769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858228+0000","last_change":"2024-03-30T13:11:00.039950+0000","last_active":"2024-03-30T13:11:01.858228+0000","last_peered":"2024-03-30T13:11:01.858228+0000","last_clean":"2024-03-30T13:11:01.858228+0000","last_became_active":"2024-03-30T13:11:00.038871+0000","last_became_peered":"2024-03-30T13:11:00.038871+0000","last_unstale":"2024-03-30T13:11:01.858228+0000","last_undegraded":"2024-03-30T13:11:01.858228+0000","last_fullsized":"2024-03-30T13:11:01.858228+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:16.001906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858336+0000","last_change":"2024-03-30T13:10:55.962143+0000","last_active":"2024-03-30T13:11:01.858336+0000","last_peered":"2024-03-30T13:11:01.858336+0000","last_clean":"2024-03-30T13:11:01.858336+0000","last_became_active":"2024-03-30T13:10:55.961186+0000","last_became_peered":"2024-03-30T13:10:55.961186+0000","last_unstale":"2024-03-30T13:11:01.858336+0000","last_undegraded":"2024-03-30T13:11:01.858336+0000","last_fullsized":"2024-03-30T13:11:01.858336+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978652+0000","last_change":"2024-03-30T13:10:54.940086+0000","last_active":"2024-03-30T13:11:00.978652+0000","last_peered":"2024-03-30T13:11:00.978652+0000","last_clean":"2024-03-30T13:11:00.978652+0000","last_became_active":"2024-03-30T13:10:54.939206+0000","last_became_peered":"2024-03-30T13:10:54.939206+0000","last_unstale":"2024-03-30T13:11:00.978652+0000","last_undegraded":"2024-03-30T13:11:00.978652+0000","last_fullsized":"2024-03-30T13:11:00.978652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:39:37.916684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855931+0000","last_change":"2024-03-30T13:11:00.034220+0000","last_active":"2024-03-30T13:11:01.855931+0000","last_peered":"2024-03-30T13:11:01.855931+0000","last_clean":"2024-03-30T13:11:01.855931+0000","last_became_active":"2024-03-30T13:11:00.033211+0000","last_became_peered":"2024-03-30T13:11:00.033211+0000","last_unstale":"2024-03-30T13:11:01.855931+0000","last_undegraded":"2024-03-30T13:11:01.855931+0000","last_fullsized":"2024-03-30T13:11:01.855931+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:22.895535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856526+0000","last_change":"2024-03-30T13:11:00.028139+0000","last_active":"2024-03-30T13:11:01.856526+0000","last_peered":"2024-03-30T13:11:01.856526+0000","last_clean":"2024-03-30T13:11:01.856526+0000","last_became_active":"2024-03-30T13:11:00.027230+0000","last_became_peered":"2024-03-30T13:11:00.027230+0000","last_unstale":"2024-03-30T13:11:01.856526+0000","last_undegraded":"2024-03-30T13:11:01.856526+0000","last_fullsized":"2024-03-30T13:11:01.856526+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:52:04.724576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855891+0000","last_change":"2024-03-30T13:10:55.996825+0000","last_active":"2024-03-30T13:11:01.855891+0000","last_peered":"2024-03-30T13:11:01.855891+0000","last_clean":"2024-03-30T13:11:01.855891+0000","last_became_active":"2024-03-30T13:10:55.977957+0000","last_became_peered":"2024-03-30T13:10:55.977957+0000","last_unstale":"2024-03-30T13:11:01.855891+0000","last_undegraded":"2024-03-30T13:11:01.855891+0000","last_fullsized":"2024-03-30T13:11:01.855891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:57:58.582699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853813+0000","last_change":"2024-03-30T13:10:54.930843+0000","last_active":"2024-03-30T13:11:01.853813+0000","last_peered":"2024-03-30T13:11:01.853813+0000","last_clean":"2024-03-30T13:11:01.853813+0000","last_became_active":"2024-03-30T13:10:54.930056+0000","last_became_peered":"2024-03-30T13:10:54.930056+0000","last_unstale":"2024-03-30T13:11:01.853813+0000","last_undegraded":"2024-03-30T13:11:01.853813+0000","last_fullsized":"2024-03-30T13:11:01.853813+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:01.296857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854932+0000","last_change":"2024-03-30T13:10:59.991481+0000","last_active":"2024-03-30T13:11:01.854932+0000","last_peered":"2024-03-30T13:11:01.854932+0000","last_clean":"2024-03-30T13:11:01.854932+0000","last_became_active":"2024-03-30T13:10:59.991071+0000","last_became_peered":"2024-03-30T13:10:59.991071+0000","last_unstale":"2024-03-30T13:11:01.854932+0000","last_undegraded":"2024-03-30T13:11:01.854932+0000","last_fullsized":"2024-03-30T13:11:01.854932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:29:50.731633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979852+0000","last_change":"2024-03-30T13:10:55.994215+0000","last_active":"2024-03-30T13:11:00.979852+0000","last_peered":"2024-03-30T13:11:00.979852+0000","last_clean":"2024-03-30T13:11:00.979852+0000","last_became_active":"2024-03-30T13:10:55.993553+0000","last_became_peered":"2024-03-30T13:10:55.993553+0000","last_unstale":"2024-03-30T13:11:00.979852+0000","last_undegraded":"2024-03-30T13:11:00.979852+0000","last_fullsized":"2024-03-30T13:11:00.979852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:02:35.609380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979309+0000","last_change":"2024-03-30T13:10:54.941904+0000","last_active":"2024-03-30T13:11:00.979309+0000","last_peered":"2024-03-30T13:11:00.979309+0000","last_clean":"2024-03-30T13:11:00.979309+0000","last_became_active":"2024-03-30T13:10:54.941223+0000","last_became_peered":"2024-03-30T13:10:54.941223+0000","last_unstale":"2024-03-30T13:11:00.979309+0000","last_undegraded":"2024-03-30T13:11:00.979309+0000","last_fullsized":"2024-03-30T13:11:00.979309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:45.696948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.854339+0000","last_change":"2024-03-30T13:10:54.943752+0000","last_active":"2024-03-30T13:11:01.854339+0000","last_peered":"2024-03-30T13:11:01.854339+0000","last_clean":"2024-03-30T13:11:01.854339+0000","last_became_active":"2024-03-30T13:10:54.943525+0000","last_became_peered":"2024-03-30T13:10:54.943525+0000","last_unstale":"2024-03-30T13:11:01.854339+0000","last_undegraded":"2024-03-30T13:11:01.854339+0000","last_fullsized":"2024-03-30T13:11:01.854339+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:47.689419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855653+0000","last_change":"2024-03-30T13:10:55.996261+0000","last_active":"2024-03-30T13:11:01.855653+0000","last_peered":"2024-03-30T13:11:01.855653+0000","last_clean":"2024-03-30T13:11:01.855653+0000","last_became_active":"2024-03-30T13:10:55.979906+0000","last_became_peered":"2024-03-30T13:10:55.979906+0000","last_unstale":"2024-03-30T13:11:01.855653+0000","last_undegraded":"2024-03-30T13:11:01.855653+0000","last_fullsized":"2024-03-30T13:11:01.855653+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:54:11.587174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856525+0000","last_change":"2024-03-30T13:11:00.031762+0000","last_active":"2024-03-30T13:11:01.856525+0000","last_peered":"2024-03-30T13:11:01.856525+0000","last_clean":"2024-03-30T13:11:01.856525+0000","last_became_active":"2024-03-30T13:11:00.030859+0000","last_became_peered":"2024-03-30T13:11:00.030859+0000","last_unstale":"2024-03-30T13:11:01.856525+0000","last_undegraded":"2024-03-30T13:11:01.856525+0000","last_fullsized":"2024-03-30T13:11:01.856525+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:27:21.084995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982601+0000","last_change":"2024-03-30T13:10:54.944088+0000","last_active":"2024-03-30T13:11:00.982601+0000","last_peered":"2024-03-30T13:11:00.982601+0000","last_clean":"2024-03-30T13:11:00.982601+0000","last_became_active":"2024-03-30T13:10:54.943275+0000","last_became_peered":"2024-03-30T13:10:54.943275+0000","last_unstale":"2024-03-30T13:11:00.982601+0000","last_undegraded":"2024-03-30T13:11:00.982601+0000","last_fullsized":"2024-03-30T13:11:00.982601+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:53.890544+0000","last_clean_scrub_stamp":"2024-03-30T13:10:53.890544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:35:57.616293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977803+0000","last_change":"2024-03-30T13:10:55.970451+0000","last_active":"2024-03-30T13:11:00.977803+0000","last_peered":"2024-03-30T13:11:00.977803+0000","last_clean":"2024-03-30T13:11:00.977803+0000","last_became_active":"2024-03-30T13:10:55.969774+0000","last_became_peered":"2024-03-30T13:10:55.969774+0000","last_unstale":"2024-03-30T13:11:00.977803+0000","last_undegraded":"2024-03-30T13:11:00.977803+0000","last_fullsized":"2024-03-30T13:11:00.977803+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859408+0000","last_change":"2024-03-30T13:11:00.039666+0000","last_active":"2024-03-30T13:11:01.859408+0000","last_peered":"2024-03-30T13:11:01.859408+0000","last_clean":"2024-03-30T13:11:01.859408+0000","last_became_active":"2024-03-30T13:11:00.038738+0000","last_became_peered":"2024-03-30T13:11:00.038738+0000","last_unstale":"2024-03-30T13:11:01.859408+0000","last_undegraded":"2024-03-30T13:11:01.859408+0000","last_fullsized":"2024-03-30T13:11:01.859408+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:58.960675+0000","last_clean_scrub_stamp":"2024-03-30T13:10:58.960675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:53.524319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857699+0000","last_change":"2024-03-30T13:10:55.985031+0000","last_active":"2024-03-30T13:11:01.857699+0000","last_peered":"2024-03-30T13:11:01.857699+0000","last_clean":"2024-03-30T13:11:01.857699+0000","last_became_active":"2024-03-30T13:10:55.984727+0000","last_became_peered":"2024-03-30T13:10:55.984727+0000","last_unstale":"2024-03-30T13:11:01.857699+0000","last_undegraded":"2024-03-30T13:11:01.857699+0000","last_fullsized":"2024-03-30T13:11:01.857699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:22:00.522781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858193+0000","last_change":"2024-03-30T13:10:55.977928+0000","last_active":"2024-03-30T13:11:01.858193+0000","last_peered":"2024-03-30T13:11:01.858193+0000","last_clean":"2024-03-30T13:11:01.858193+0000","last_became_active":"2024-03-30T13:10:55.977153+0000","last_became_peered":"2024-03-30T13:10:55.977153+0000","last_unstale":"2024-03-30T13:11:01.858193+0000","last_undegraded":"2024-03-30T13:11:01.858193+0000","last_fullsized":"2024-03-30T13:11:01.858193+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853982+0000","last_change":"2024-03-30T13:10:55.965525+0000","last_active":"2024-03-30T13:11:01.853982+0000","last_peered":"2024-03-30T13:11:01.853982+0000","last_clean":"2024-03-30T13:11:01.853982+0000","last_became_active":"2024-03-30T13:10:55.965066+0000","last_became_peered":"2024-03-30T13:10:55.965066+0000","last_unstale":"2024-03-30T13:11:01.853982+0000","last_undegraded":"2024-03-30T13:11:01.853982+0000","last_fullsized":"2024-03-30T13:11:01.853982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:01:56.497473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982361+0000","last_change":"2024-03-30T13:10:55.961817+0000","last_active":"2024-03-30T13:11:00.982361+0000","last_peered":"2024-03-30T13:11:00.982361+0000","last_clean":"2024-03-30T13:11:00.982361+0000","last_became_active":"2024-03-30T13:10:55.960939+0000","last_became_peered":"2024-03-30T13:10:55.960939+0000","last_unstale":"2024-03-30T13:11:00.982361+0000","last_undegraded":"2024-03-30T13:11:00.982361+0000","last_fullsized":"2024-03-30T13:11:00.982361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:30:53.806860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980400+0000","last_change":"2024-03-30T13:10:55.991524+0000","last_active":"2024-03-30T13:11:00.980400+0000","last_peered":"2024-03-30T13:11:00.980400+0000","last_clean":"2024-03-30T13:11:00.980400+0000","last_became_active":"2024-03-30T13:10:55.990783+0000","last_became_peered":"2024-03-30T13:10:55.990783+0000","last_unstale":"2024-03-30T13:11:00.980400+0000","last_undegraded":"2024-03-30T13:11:00.980400+0000","last_fullsized":"2024-03-30T13:11:00.980400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:22:09.834713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981092+0000","last_change":"2024-03-30T13:10:55.985903+0000","last_active":"2024-03-30T13:11:00.981092+0000","last_peered":"2024-03-30T13:11:00.981092+0000","last_clean":"2024-03-30T13:11:00.981092+0000","last_became_active":"2024-03-30T13:10:55.985086+0000","last_became_peered":"2024-03-30T13:10:55.985086+0000","last_unstale":"2024-03-30T13:11:00.981092+0000","last_undegraded":"2024-03-30T13:11:00.981092+0000","last_fullsized":"2024-03-30T13:11:00.981092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:25:43.386143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858231+0000","last_change":"2024-03-30T13:10:55.961871+0000","last_active":"2024-03-30T13:11:01.858231+0000","last_peered":"2024-03-30T13:11:01.858231+0000","last_clean":"2024-03-30T13:11:01.858231+0000","last_became_active":"2024-03-30T13:10:55.961589+0000","last_became_peered":"2024-03-30T13:10:55.961589+0000","last_unstale":"2024-03-30T13:11:01.858231+0000","last_undegraded":"2024-03-30T13:11:01.858231+0000","last_fullsized":"2024-03-30T13:11:01.858231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981592+0000","last_change":"2024-03-30T13:10:55.979356+0000","last_active":"2024-03-30T13:11:00.981592+0000","last_peered":"2024-03-30T13:11:00.981592+0000","last_clean":"2024-03-30T13:11:00.981592+0000","last_became_active":"2024-03-30T13:10:55.978691+0000","last_became_peered":"2024-03-30T13:10:55.978691+0000","last_unstale":"2024-03-30T13:11:00.981592+0000","last_undegraded":"2024-03-30T13:11:00.981592+0000","last_fullsized":"2024-03-30T13:11:00.981592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:26:40.192186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853812+0000","last_change":"2024-03-30T13:10:55.965264+0000","last_active":"2024-03-30T13:11:01.853812+0000","last_peered":"2024-03-30T13:11:01.853812+0000","last_clean":"2024-03-30T13:11:01.853812+0000","last_became_active":"2024-03-30T13:10:55.964763+0000","last_became_peered":"2024-03-30T13:10:55.964763+0000","last_unstale":"2024-03-30T13:11:01.853812+0000","last_undegraded":"2024-03-30T13:11:01.853812+0000","last_fullsized":"2024-03-30T13:11:01.853812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:47:22.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980777+0000","last_change":"2024-03-30T13:10:55.968931+0000","last_active":"2024-03-30T13:11:00.980777+0000","last_peered":"2024-03-30T13:11:00.980777+0000","last_clean":"2024-03-30T13:11:00.980777+0000","last_became_active":"2024-03-30T13:10:55.968082+0000","last_became_peered":"2024-03-30T13:10:55.968082+0000","last_unstale":"2024-03-30T13:11:00.980777+0000","last_undegraded":"2024-03-30T13:11:00.980777+0000","last_fullsized":"2024-03-30T13:11:00.980777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:06:36.244970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.977480+0000","last_change":"2024-03-30T13:10:55.971076+0000","last_active":"2024-03-30T13:11:00.977480+0000","last_peered":"2024-03-30T13:11:00.977480+0000","last_clean":"2024-03-30T13:11:00.977480+0000","last_became_active":"2024-03-30T13:10:55.970755+0000","last_became_peered":"2024-03-30T13:10:55.970755+0000","last_unstale":"2024-03-30T13:11:00.977480+0000","last_undegraded":"2024-03-30T13:11:00.977480+0000","last_fullsized":"2024-03-30T13:11:00.977480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:28:02.305425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855370+0000","last_change":"2024-03-30T13:10:55.996858+0000","last_active":"2024-03-30T13:11:01.855370+0000","last_peered":"2024-03-30T13:11:01.855370+0000","last_clean":"2024-03-30T13:11:01.855370+0000","last_became_active":"2024-03-30T13:10:55.980721+0000","last_became_peered":"2024-03-30T13:10:55.980721+0000","last_unstale":"2024-03-30T13:11:01.855370+0000","last_undegraded":"2024-03-30T13:11:01.855370+0000","last_fullsized":"2024-03-30T13:11:01.855370+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:55:49.462756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855814+0000","last_change":"2024-03-30T13:10:55.976030+0000","last_active":"2024-03-30T13:11:01.855814+0000","last_peered":"2024-03-30T13:11:01.855814+0000","last_clean":"2024-03-30T13:11:01.855814+0000","last_became_active":"2024-03-30T13:10:55.975744+0000","last_became_peered":"2024-03-30T13:10:55.975744+0000","last_unstale":"2024-03-30T13:11:01.855814+0000","last_undegraded":"2024-03-30T13:11:01.855814+0000","last_fullsized":"2024-03-30T13:11:01.855814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855909+0000","last_change":"2024-03-30T13:10:55.998425+0000","last_active":"2024-03-30T13:11:01.855909+0000","last_peered":"2024-03-30T13:11:01.855909+0000","last_clean":"2024-03-30T13:11:01.855909+0000","last_became_active":"2024-03-30T13:10:55.998137+0000","last_became_peered":"2024-03-30T13:10:55.998137+0000","last_unstale":"2024-03-30T13:11:01.855909+0000","last_undegraded":"2024-03-30T13:11:01.855909+0000","last_fullsized":"2024-03-30T13:11:01.855909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:11:09.212562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.859740+0000","last_change":"2024-03-30T13:10:55.961591+0000","last_active":"2024-03-30T13:11:01.859740+0000","last_peered":"2024-03-30T13:11:01.859740+0000","last_clean":"2024-03-30T13:11:01.859740+0000","last_became_active":"2024-03-30T13:10:55.960885+0000","last_became_peered":"2024-03-30T13:10:55.960885+0000","last_unstale":"2024-03-30T13:11:01.859740+0000","last_undegraded":"2024-03-30T13:11:01.859740+0000","last_fullsized":"2024-03-30T13:11:01.859740+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:27:48.697986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855050+0000","last_change":"2024-03-30T13:10:55.964798+0000","last_active":"2024-03-30T13:11:01.855050+0000","last_peered":"2024-03-30T13:11:01.855050+0000","last_clean":"2024-03-30T13:11:01.855050+0000","last_became_active":"2024-03-30T13:10:55.964471+0000","last_became_peered":"2024-03-30T13:10:55.964471+0000","last_unstale":"2024-03-30T13:11:01.855050+0000","last_undegraded":"2024-03-30T13:11:01.855050+0000","last_fullsized":"2024-03-30T13:11:01.855050+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:08:41.557413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.853680+0000","last_change":"2024-03-30T13:10:55.959954+0000","last_active":"2024-03-30T13:11:01.853680+0000","last_peered":"2024-03-30T13:11:01.853680+0000","last_clean":"2024-03-30T13:11:01.853680+0000","last_became_active":"2024-03-30T13:10:55.958856+0000","last_became_peered":"2024-03-30T13:10:55.958856+0000","last_unstale":"2024-03-30T13:11:01.853680+0000","last_undegraded":"2024-03-30T13:11:01.853680+0000","last_fullsized":"2024-03-30T13:11:01.853680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:32:17.364725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.980372+0000","last_change":"2024-03-30T13:10:55.957900+0000","last_active":"2024-03-30T13:11:00.980372+0000","last_peered":"2024-03-30T13:11:00.980372+0000","last_clean":"2024-03-30T13:11:00.980372+0000","last_became_active":"2024-03-30T13:10:55.957071+0000","last_became_peered":"2024-03-30T13:10:55.957071+0000","last_unstale":"2024-03-30T13:11:00.980372+0000","last_undegraded":"2024-03-30T13:11:00.980372+0000","last_fullsized":"2024-03-30T13:11:00.980372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:16:27.043781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979715+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.979715+0000","last_peered":"2024-03-30T13:11:00.979715+0000","last_clean":"2024-03-30T13:11:00.979715+0000","last_became_active":"2024-03-30T13:10:55.990646+0000","last_became_peered":"2024-03-30T13:10:55.990646+0000","last_unstale":"2024-03-30T13:11:00.979715+0000","last_undegraded":"2024-03-30T13:11:00.979715+0000","last_fullsized":"2024-03-30T13:11:00.979715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:00:29.053451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.981245+0000","last_change":"2024-03-30T13:10:55.991360+0000","last_active":"2024-03-30T13:11:00.981245+0000","last_peered":"2024-03-30T13:11:00.981245+0000","last_clean":"2024-03-30T13:11:00.981245+0000","last_became_active":"2024-03-30T13:10:55.990666+0000","last_became_peered":"2024-03-30T13:10:55.990666+0000","last_unstale":"2024-03-30T13:11:00.981245+0000","last_undegraded":"2024-03-30T13:11:00.981245+0000","last_fullsized":"2024-03-30T13:11:00.981245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:58:11.940343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855715+0000","last_change":"2024-03-30T13:10:55.975002+0000","last_active":"2024-03-30T13:11:01.855715+0000","last_peered":"2024-03-30T13:11:01.855715+0000","last_clean":"2024-03-30T13:11:01.855715+0000","last_became_active":"2024-03-30T13:10:55.974620+0000","last_became_peered":"2024-03-30T13:10:55.974620+0000","last_unstale":"2024-03-30T13:11:01.855715+0000","last_undegraded":"2024-03-30T13:11:01.855715+0000","last_fullsized":"2024-03-30T13:11:01.855715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:06:33.039784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855796+0000","last_change":"2024-03-30T13:10:55.962912+0000","last_active":"2024-03-30T13:11:01.855796+0000","last_peered":"2024-03-30T13:11:01.855796+0000","last_clean":"2024-03-30T13:11:01.855796+0000","last_became_active":"2024-03-30T13:10:55.962404+0000","last_became_peered":"2024-03-30T13:10:55.962404+0000","last_unstale":"2024-03-30T13:11:01.855796+0000","last_undegraded":"2024-03-30T13:11:01.855796+0000","last_fullsized":"2024-03-30T13:11:01.855796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:43:27.534340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855883+0000","last_change":"2024-03-30T13:10:55.962709+0000","last_active":"2024-03-30T13:11:01.855883+0000","last_peered":"2024-03-30T13:11:01.855883+0000","last_clean":"2024-03-30T13:11:01.855883+0000","last_became_active":"2024-03-30T13:10:55.962426+0000","last_became_peered":"2024-03-30T13:10:55.962426+0000","last_unstale":"2024-03-30T13:11:01.855883+0000","last_undegraded":"2024-03-30T13:11:01.855883+0000","last_fullsized":"2024-03-30T13:11:01.855883+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:21:46.344312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.978582+0000","last_change":"2024-03-30T13:10:55.973390+0000","last_active":"2024-03-30T13:11:00.978582+0000","last_peered":"2024-03-30T13:11:00.978582+0000","last_clean":"2024-03-30T13:11:00.978582+0000","last_became_active":"2024-03-30T13:10:55.969915+0000","last_became_peered":"2024-03-30T13:10:55.969915+0000","last_unstale":"2024-03-30T13:11:00.978582+0000","last_undegraded":"2024-03-30T13:11:00.978582+0000","last_fullsized":"2024-03-30T13:11:00.978582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:34:40.812265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.979479+0000","last_change":"2024-03-30T13:10:55.969396+0000","last_active":"2024-03-30T13:11:00.979479+0000","last_peered":"2024-03-30T13:11:00.979479+0000","last_clean":"2024-03-30T13:11:00.979479+0000","last_became_active":"2024-03-30T13:10:55.965044+0000","last_became_peered":"2024-03-30T13:10:55.965044+0000","last_unstale":"2024-03-30T13:11:00.979479+0000","last_undegraded":"2024-03-30T13:11:00.979479+0000","last_fullsized":"2024-03-30T13:11:00.979479+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:34:33.758969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855400+0000","last_change":"2024-03-30T13:10:55.974744+0000","last_active":"2024-03-30T13:11:01.855400+0000","last_peered":"2024-03-30T13:11:01.855400+0000","last_clean":"2024-03-30T13:11:01.855400+0000","last_became_active":"2024-03-30T13:10:55.974421+0000","last_became_peered":"2024-03-30T13:10:55.974421+0000","last_unstale":"2024-03-30T13:11:01.855400+0000","last_undegraded":"2024-03-30T13:11:01.855400+0000","last_fullsized":"2024-03-30T13:11:01.855400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:35:40.791064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.857905+0000","last_change":"2024-03-30T13:10:55.998371+0000","last_active":"2024-03-30T13:11:01.857905+0000","last_peered":"2024-03-30T13:11:01.857905+0000","last_clean":"2024-03-30T13:11:01.857905+0000","last_became_active":"2024-03-30T13:10:55.997851+0000","last_became_peered":"2024-03-30T13:10:55.997851+0000","last_unstale":"2024-03-30T13:11:01.857905+0000","last_undegraded":"2024-03-30T13:11:01.857905+0000","last_fullsized":"2024-03-30T13:11:01.857905+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:46.705515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.855480+0000","last_change":"2024-03-30T13:10:55.967579+0000","last_active":"2024-03-30T13:11:01.855480+0000","last_peered":"2024-03-30T13:11:01.855480+0000","last_clean":"2024-03-30T13:11:01.855480+0000","last_became_active":"2024-03-30T13:10:55.967284+0000","last_became_peered":"2024-03-30T13:10:55.967284+0000","last_unstale":"2024-03-30T13:11:01.855480+0000","last_undegraded":"2024-03-30T13:11:01.855480+0000","last_fullsized":"2024-03-30T13:11:01.855480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:41:39.989019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.858659+0000","last_change":"2024-03-30T13:10:55.998028+0000","last_active":"2024-03-30T13:11:01.858659+0000","last_peered":"2024-03-30T13:11:01.858659+0000","last_clean":"2024-03-30T13:11:01.858659+0000","last_became_active":"2024-03-30T13:10:55.997628+0000","last_became_peered":"2024-03-30T13:10:55.997628+0000","last_unstale":"2024-03-30T13:11:01.858659+0000","last_undegraded":"2024-03-30T13:11:01.858659+0000","last_fullsized":"2024-03-30T13:11:01.858659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:26:03.062101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.982130+0000","last_change":"2024-03-30T13:10:55.983619+0000","last_active":"2024-03-30T13:11:00.982130+0000","last_peered":"2024-03-30T13:11:00.982130+0000","last_clean":"2024-03-30T13:11:00.982130+0000","last_became_active":"2024-03-30T13:10:55.982947+0000","last_became_peered":"2024-03-30T13:10:55.982947+0000","last_unstale":"2024-03-30T13:11:00.982130+0000","last_undegraded":"2024-03-30T13:11:00.982130+0000","last_fullsized":"2024-03-30T13:11:00.982130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:31:00.291625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:00.983477+0000","last_change":"2024-03-30T13:10:55.992614+0000","last_active":"2024-03-30T13:11:00.983477+0000","last_peered":"2024-03-30T13:11:00.983477+0000","last_clean":"2024-03-30T13:11:00.983477+0000","last_became_active":"2024-03-30T13:10:55.991559+0000","last_became_peered":"2024-03-30T13:10:55.991559+0000","last_unstale":"2024-03-30T13:11:00.983477+0000","last_undegraded":"2024-03-30T13:11:00.983477+0000","last_fullsized":"2024-03-30T13:11:00.983477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:28:13.766039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-03-30T13:11:01.856251+0000","last_change":"2024-03-30T13:10:55.960973+0000","last_active":"2024-03-30T13:11:01.856251+0000","last_peered":"2024-03-30T13:11:01.856251+0000","last_clean":"2024-03-30T13:11:01.856251+0000","last_became_active":"2024-03-30T13:10:55.960665+0000","last_became_peered":"2024-03-30T13:10:55.960665+0000","last_unstale":"2024-03-30T13:11:01.856251+0000","last_undegraded":"2024-03-30T13:11:01.856251+0000","last_fullsized":"2024-03-30T13:11:01.856251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:54.899111+0000","last_clean_scrub_stamp":"2024-03-30T13:10:54.899111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:57:18.833972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":115079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509452,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":111147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31464,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":294912,"data_stored":114407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":701003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":290816,"data_stored":110621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":884736,"data_stored":701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:11:26.240 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-30T13:11:26.240 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-30T13:11:26.241 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-30T13:11:26.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-30T13:11:26.261 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:26.278 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:26.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.379+0000 7ff70d6db640 1 Processor -- start 2024-03-30T13:11:26.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.379+0000 7ff70d6db640 1 -- start start 2024-03-30T13:11:26.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.379+0000 7ff70d6db640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff708145d20 0x7ff7080cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.379+0000 7ff70d6db640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 0x7ff70806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.379+0000 7ff70d6db640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7080cfc00 con 0x7ff70806ff20 2024-03-30T13:11:26.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.379+0000 7ff70d6db640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff70806e110 con 0x7ff708145d20 2024-03-30T13:11:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.379+0000 7ff70d6db640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff708070350 con 0x7ff708070a00 2024-03-30T13:11:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff70806ff20 0x7ff70815fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59762/0 (socket says 172.21.15.43:59762) 2024-03-30T13:11:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 -- 172.21.15.43:0/1174756715 learned_addr learned my addr 172.21.15.43:0/1174756715 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff706ffd640 1 --2- 172.21.15.43:0/1174756715 >> v2:172.21.15.113:3300/0 conn(0x7ff708145d20 0x7ff7080cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7067fc640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 0x7ff70806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3739011255 0 0) 0x7ff7000014a0 con 0x7ff70806ff20 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6f0003530 con 0x7ff70806ff20 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3115859140 0 0) 0x7ff6f0003530 con 0x7ff70806ff20 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 msgr2=0x7ff70806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 0x7ff70806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 >> v2:172.21.15.113:3300/0 conn(0x7ff708145d20 msgr2=0x7ff7080cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 --2- 172.21.15.43:0/1174756715 >> v2:172.21.15.113:3300/0 conn(0x7ff708145d20 0x7ff7080cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff708146100 con 0x7ff70806ff20 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ff700001bd0 con 0x7ff70806ff20 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 >> v1:172.21.15.43:6789/0 conn(0x7ff70806ff20 legacy=0x7ff70815fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 0x7ff6f00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6f0003f70 0x7ff6f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0004d50 0x7ff6f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff70806e110 con 0x7ff6f0004d50 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff708070350 con 0x7ff6f0003f70 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f000ba50 con 0x7ff708070a00 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0004d50 0x7ff6f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff706ffd640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 0x7ff6f00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7067fc640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6f0003f70 0x7ff6f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 -- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 msgr2=0x7ff6f00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 0x7ff6f00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 -- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6f0003f70 msgr2=0x7ff6f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff6f0003f70 0x7ff6f0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff7077fe640 1 -- 172.21.15.43:0/1174756715 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f0003530 con 0x7ff6f0004d50 2024-03-30T13:11:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.383+0000 7ff706ffd640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff708070a00 0x7ff6f00039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff7077fe640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0004d50 0x7ff6f00051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff70000aa50 tx=0x7ff70001fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80844100d3b3a4c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70000a670 con 0x7ff6f0004d50 2024-03-30T13:11:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7000ad070 con 0x7ff6f0004d50 2024-03-30T13:11:26.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff705ffb640 1 -- 172.21.15.43:0/1174756715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7000a9980 con 0x7ff6f0004d50 2024-03-30T13:11:26.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- 172.21.15.43:0/1174756715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0004d50 msgr2=0x7ff6f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 --2- 172.21.15.43:0/1174756715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6f0004d50 0x7ff6f00051d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff70000aa50 tx=0x7ff70001fb00 comp rx=0 tx=0).stop 2024-03-30T13:11:26.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff7077fe640 1 -- 172.21.15.43:0/1174756715 reap_dead start 2024-03-30T13:11:26.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- 172.21.15.43:0/1174756715 shutdown_connections 2024-03-30T13:11:26.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- 172.21.15.43:0/1174756715 >> 172.21.15.43:0/1174756715 conn(0x7ff70806e410 msgr2=0x7ff70806e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:26.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- 172.21.15.43:0/1174756715 shutdown_connections 2024-03-30T13:11:26.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- 172.21.15.43:0/1174756715 wait complete. 2024-03-30T13:11:26.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 Processor -- start 2024-03-30T13:11:26.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- start start 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6f0004d50 0x7ff70813e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081f24b0 0x7ff7081e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7081f2930 0x7ff7080cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080d0440 con 0x7ff7081f24b0 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080d0160 con 0x7ff7081f2930 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff70d6db640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff7080d0720 con 0x7ff6f0004d50 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff7067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081f24b0 0x7ff7081e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff7067fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081f24b0 0x7ff7081e5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35196/0 (socket says 172.21.15.43:35196) 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.387+0000 7ff7067fc640 1 -- 172.21.15.43:0/3770918861 learned_addr learned my addr 172.21.15.43:0/3770918861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff706ffd640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6f0004d50 0x7ff70813e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff7077fe640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7081f2930 0x7ff7080cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff7067fc640 1 -- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6f0004d50 msgr2=0x7ff70813e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff7067fc640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6f0004d50 0x7ff70813e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff7067fc640 1 -- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7081f2930 msgr2=0x7ff7080cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff7067fc640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7081f2930 0x7ff7080cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff7067fc640 1 -- 172.21.15.43:0/3770918861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7080c8a70 con 0x7ff7081f24b0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff7067fc640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081f24b0 0x7ff7081e5a00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8010890 tx=0x7ff6f8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff6ef7fe640 1 -- 172.21.15.43:0/3770918861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800f040 con 0x7ff7081f24b0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff6ef7fe640 1 -- 172.21.15.43:0/3770918861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f800eb60 con 0x7ff7081f24b0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7080c8d50 con 0x7ff7081f24b0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff6ef7fe640 1 -- 172.21.15.43:0/3770918861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f801c400 con 0x7ff7081f24b0 2024-03-30T13:11:26.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7080c9310 con 0x7ff7081f24b0 2024-03-30T13:11:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7080c97f0 con 0x7ff7081f24b0 2024-03-30T13:11:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff6ef7fe640 1 -- 172.21.15.43:0/3770918861 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff6f801c5a0 con 0x7ff7081f24b0 2024-03-30T13:11:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff6ef7fe640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff6f4067ff0 0x7ff6f406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff6ef7fe640 1 -- 172.21.15.43:0/3770918861 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff6f8086e40 con 0x7ff7081f24b0 2024-03-30T13:11:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff706ffd640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff6f4067ff0 0x7ff6f406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.391+0000 7ff706ffd640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff6f4067ff0 0x7ff6f406a4b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009a60 tx=0x7ff6fc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:26.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.395+0000 7ff6ef7fe640 1 -- 172.21.15.43:0/3770918861 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff7080c97f0 con 0x7ff7081f24b0 2024-03-30T13:11:26.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff708068890 con 0x7ff7081f24b0 2024-03-30T13:11:26.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff6ef7fe640 1 -- 172.21.15.43:0/3770918861 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff6f8018030 con 0x7ff7081f24b0 2024-03-30T13:11:26.595 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:26.595 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-30T13:11:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff6f4067ff0 msgr2=0x7ff6f406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff6f4067ff0 0x7ff6f406a4b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009a60 tx=0x7ff6fc008040 comp rx=0 tx=0).stop 2024-03-30T13:11:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081f24b0 msgr2=0x7ff7081e5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081f24b0 0x7ff7081e5a00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8010890 tx=0x7ff6f8010d50 comp rx=0 tx=0).stop 2024-03-30T13:11:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 shutdown_connections 2024-03-30T13:11:26.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff6f4067ff0 0x7ff6f406a4b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7081f2930 0x7ff7080cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7081f24b0 0x7ff7081e5a00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 --2- 172.21.15.43:0/3770918861 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff6f0004d50 0x7ff70813e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 >> 172.21.15.43:0/3770918861 conn(0x7ff70806e410 msgr2=0x7ff7080c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 shutdown_connections 2024-03-30T13:11:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.587+0000 7ff70d6db640 1 -- 172.21.15.43:0/3770918861 wait complete. 2024-03-30T13:11:26.611 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-30T13:11:26.612 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:26.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1266312640 1 Processor -- start 2024-03-30T13:11:26.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1266312640 1 -- start start 2024-03-30T13:11:26.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1266312640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f12600d0470 0x7f126015fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1266312640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12600ccba0 con 0x7f126015c1c0 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1266312640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12600b26f0 con 0x7f12600d0900 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1266312640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f12600ce3a0 con 0x7f12600d0470 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f126015c1c0 0x7f12600cfd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59764/0 (socket says 172.21.15.43:59764) 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125ffff640 1 -- 172.21.15.43:0/3736701646 learned_addr learned my addr 172.21.15.43:0/3736701646 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1257fff640 1 --2- 172.21.15.43:0/3736701646 >> v2:172.21.15.113:3300/0 conn(0x7f12600d0470 0x7f126015fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4279643531 0 0) 0x7f12600ccba0 con 0x7f126015c1c0 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f124c003530 con 0x7f126015c1c0 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1924420394 0 0) 0x7f124c003530 con 0x7f126015c1c0 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 >> v2:172.21.15.113:3300/0 conn(0x7f12600d0470 msgr2=0x7f126015fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 --2- 172.21.15.43:0/3736701646 >> v2:172.21.15.113:3300/0 conn(0x7f12600d0470 0x7f126015fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 >> v1:172.21.15.113:6789/0 conn(0x7f12600d0900 legacy=0x7f12601603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12600b29d0 con 0x7f126015c1c0 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f1250001680 con 0x7f126015c1c0 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 >> v1:172.21.15.43:6789/0 conn(0x7f126015c1c0 legacy=0x7f12600cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0039c0 0x7f124c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124c004810 0x7f124c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c0051d0 0x7f124c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f124c00c040 con 0x7f124c0051d0 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f124c00c1f0 con 0x7f124c0039c0 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f124c00c3f0 con 0x7f124c004810 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f12648af640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c0051d0 0x7f124c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f125ffff640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0039c0 0x7f124c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1257fff640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124c004810 0x7f124c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f12648af640 1 -- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124c004810 msgr2=0x7f124c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f12648af640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124c004810 0x7f124c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f12648af640 1 -- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0039c0 msgr2=0x7f124c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:26.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f12648af640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0039c0 0x7f124c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f12648af640 1 -- 172.21.15.43:0/3736701646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c003530 con 0x7f124c0051d0 2024-03-30T13:11:26.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.759+0000 7f1257fff640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f124c004810 0x7f124c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f12648af640 1 -- 172.21.15.43:0/3736701646 reap_dead start 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f12648af640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c0051d0 0x7f124c00b9d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1258001e40 tx=0x7f125801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca78e553644f64d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258001a90 con 0x7f124c0051d0 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1258003b80 con 0x7f124c0051d0 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f125f7fe640 1 -- 172.21.15.43:0/3736701646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258005020 con 0x7f124c0051d0 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- 172.21.15.43:0/3736701646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c0051d0 msgr2=0x7f124c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c0051d0 0x7f124c00b9d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1258001e40 tx=0x7f125801c760 comp rx=0 tx=0).stop 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- 172.21.15.43:0/3736701646 shutdown_connections 2024-03-30T13:11:26.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 --2- 172.21.15.43:0/3736701646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f124c0051d0 0x7f124c00b9d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- 172.21.15.43:0/3736701646 >> 172.21.15.43:0/3736701646 conn(0x7f12600b3d60 msgr2=0x7f12600b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:26.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- 172.21.15.43:0/3736701646 shutdown_connections 2024-03-30T13:11:26.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- 172.21.15.43:0/3736701646 wait complete. 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 Processor -- start 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- start start 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0051d0 0x7f12600cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12601544a0 0x7f12600c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12600c97a0 0x7f12600d2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f12600b2cb0 con 0x7f12600c97a0 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f12600cce80 con 0x7f124c0051d0 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f1266312640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f12600cc7e0 con 0x7f12601544a0 2024-03-30T13:11:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.763+0000 7f12648af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12600c97a0 0x7f12600d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12600c97a0 0x7f12600d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35200/0 (socket says 172.21.15.43:35200) 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 -- 172.21.15.43:0/2325415207 learned_addr learned my addr 172.21.15.43:0/2325415207 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125ffff640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0051d0 0x7f12600cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f1257fff640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12601544a0 0x7f12600c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 -- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12601544a0 msgr2=0x7f12600c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12601544a0 0x7f12600c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 -- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0051d0 msgr2=0x7f12600cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0051d0 0x7f12600cc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 -- 172.21.15.43:0/2325415207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12600c6080 con 0x7f12600c97a0 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125ffff640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0051d0 0x7f12600cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f1257fff640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12601544a0 0x7f12600c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f12648af640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12600c97a0 0x7f12600d2090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f125801d000 tx=0x7f124c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125d7fa640 1 -- 172.21.15.43:0/2325415207 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258001a90 con 0x7f12600c97a0 2024-03-30T13:11:26.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125d7fa640 1 -- 172.21.15.43:0/2325415207 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12580ad070 con 0x7f12600c97a0 2024-03-30T13:11:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125d7fa640 1 -- 172.21.15.43:0/2325415207 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1258007dc0 con 0x7f12600c97a0 2024-03-30T13:11:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12600c6360 con 0x7f12600c97a0 2024-03-30T13:11:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12600c6920 con 0x7f12600c97a0 2024-03-30T13:11:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1260068890 con 0x7f12600c97a0 2024-03-30T13:11:26.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125d7fa640 1 -- 172.21.15.43:0/2325415207 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1258007030 con 0x7f12600c97a0 2024-03-30T13:11:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125d7fa640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1240067fc0 0x7f124006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125d7fa640 1 -- 172.21.15.43:0/2325415207 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f125811ff70 con 0x7f12600c97a0 2024-03-30T13:11:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125ffff640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1240067fc0 0x7f124006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:26.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.767+0000 7f125ffff640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1240067fc0 0x7f124006a480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1250001210 tx=0x7f1250000f60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:26.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.771+0000 7f125d7fa640 1 -- 172.21.15.43:0/2325415207 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1258016470 con 0x7f12600c97a0 2024-03-30T13:11:26.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.931+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12600679e0 con 0x7f12600c97a0 2024-03-30T13:11:26.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.931+0000 7f125d7fa640 1 -- 172.21.15.43:0/2325415207 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7f12580f03e0 con 0x7f12600c97a0 2024-03-30T13:11:26.940 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:26.940 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1240067fc0 msgr2=0x7f124006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1240067fc0 0x7f124006a480 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1250001210 tx=0x7f1250000f60 comp rx=0 tx=0).stop 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12600c97a0 msgr2=0x7f12600d2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12600c97a0 0x7f12600d2090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f125801d000 tx=0x7f124c005840 comp rx=0 tx=0).stop 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 shutdown_connections 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1240067fc0 0x7f124006a480 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12600c97a0 0x7f12600d2090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f12601544a0 0x7f12600c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 --2- 172.21.15.43:0/2325415207 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f124c0051d0 0x7f12600cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 >> 172.21.15.43:0/2325415207 conn(0x7f12600b3d60 msgr2=0x7f12601598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 shutdown_connections 2024-03-30T13:11:26.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:26.935+0000 7f1266312640 1 -- 172.21.15.43:0/2325415207 wait complete. 2024-03-30T13:11:26.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:26.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:27.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.063+0000 7f5762ad6640 1 Processor -- start 2024-03-30T13:11:27.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.063+0000 7f5762ad6640 1 -- start start 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5762ad6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f575c06c070 0x7f575c06c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5762ad6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c0adbc0 0x7f575c0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5762ad6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f575c0b5470 con 0x7f575c147e00 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5762ad6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f575c06ca10 con 0x7f575c06c070 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5762ad6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f575c0b5750 con 0x7f575c0adbc0 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5761073640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c0adbc0 0x7f575c0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f575bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f575c06c070 0x7f575c06c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5761073640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c0adbc0 0x7f575c0ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37662/0 (socket says 172.21.15.43:37662) 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f575bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f575c06c070 0x7f575c06c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:47204/0 (socket says 172.21.15.43:47204) 2024-03-30T13:11:27.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5761073640 1 -- 172.21.15.43:0/2579538228 learned_addr learned my addr 172.21.15.43:0/2579538228 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:27.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5761073640 1 -- 172.21.15.43:0/2579538228 >> v2:172.21.15.113:3300/0 conn(0x7f575c06c070 msgr2=0x7f575c06c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:27.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5761073640 1 --2- 172.21.15.43:0/2579538228 >> v2:172.21.15.113:3300/0 conn(0x7f575c06c070 0x7f575c06c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5761073640 1 -- 172.21.15.43:0/2579538228 >> v1:172.21.15.113:6789/0 conn(0x7f575c147e00 legacy=0x7f575c0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f5761073640 1 -- 172.21.15.43:0/2579538228 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c0b5a30 con 0x7f575c0adbc0 2024-03-30T13:11:27.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.067+0000 7f575bfff640 1 --2- 172.21.15.43:0/2579538228 >> v2:172.21.15.113:3300/0 conn(0x7f575c06c070 0x7f575c06c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:27.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.071+0000 7f5761073640 1 --2- 172.21.15.43:0/2579538228 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c0adbc0 0x7f575c0ae030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5754009e50 tx=0x7f575401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec264f59ddde1f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f575b7fe640 1 -- 172.21.15.43:0/2579538228 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540a5700 con 0x7f575c0adbc0 2024-03-30T13:11:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f575b7fe640 1 -- 172.21.15.43:0/2579538228 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57540a58a0 con 0x7f575c0adbc0 2024-03-30T13:11:27.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f575b7fe640 1 -- 172.21.15.43:0/2579538228 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540a5c70 con 0x7f575c0adbc0 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- 172.21.15.43:0/2579538228 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c0adbc0 msgr2=0x7f575c0ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2579538228 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c0adbc0 0x7f575c0ae030 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5754009e50 tx=0x7f575401ca30 comp rx=0 tx=0).stop 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- 172.21.15.43:0/2579538228 shutdown_connections 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2579538228 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c0adbc0 0x7f575c0ae030 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2579538228 >> v2:172.21.15.113:3300/0 conn(0x7f575c06c070 0x7f575c06c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- 172.21.15.43:0/2579538228 >> 172.21.15.43:0/2579538228 conn(0x7f575c0d1070 msgr2=0x7f575c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- 172.21.15.43:0/2579538228 shutdown_connections 2024-03-30T13:11:27.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- 172.21.15.43:0/2579538228 wait complete. 2024-03-30T13:11:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 Processor -- start 2024-03-30T13:11:27.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- start start 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c06c070 0x7f575c0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0adbc0 0x7f575c167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f575c147e00 0x7f575c160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f575c16b2a0 con 0x7f575c0adbc0 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f575c16b120 con 0x7f575c147e00 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f5762ad6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f575c16b420 con 0x7f575c06c070 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f575bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0adbc0 0x7f575c167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f575bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0adbc0 0x7f575c167ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35212/0 (socket says 172.21.15.43:35212) 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.075+0000 7f575bfff640 1 -- 172.21.15.43:0/2002186169 learned_addr learned my addr 172.21.15.43:0/2002186169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f5760872640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c06c070 0x7f575c0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f5761073640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f575c147e00 0x7f575c160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f575bfff640 1 -- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c06c070 msgr2=0x7f575c0cfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f575bfff640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c06c070 0x7f575c0cfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f575bfff640 1 -- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f575c147e00 msgr2=0x7f575c160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f575bfff640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f575c147e00 0x7f575c160e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f575bfff640 1 -- 172.21.15.43:0/2002186169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c0cd9e0 con 0x7f575c0adbc0 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f5761073640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f575c147e00 0x7f575c160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f575bfff640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0adbc0 0x7f575c167ec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f575000e980 tx=0x7f575000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f57597fa640 1 -- 172.21.15.43:0/2002186169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5750017070 con 0x7f575c0adbc0 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f57597fa640 1 -- 172.21.15.43:0/2002186169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f575000ca00 con 0x7f575c0adbc0 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c0cea40 con 0x7f575c0adbc0 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f57597fa640 1 -- 172.21.15.43:0/2002186169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575000cdd0 con 0x7f575c0adbc0 2024-03-30T13:11:27.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c0ce240 con 0x7f575c0adbc0 2024-03-30T13:11:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f575c068900 con 0x7f575c0adbc0 2024-03-30T13:11:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f57597fa640 1 -- 172.21.15.43:0/2002186169 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f575001a070 con 0x7f575c0adbc0 2024-03-30T13:11:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f57597fa640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5748067fc0 0x7f574806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f57597fa640 1 -- 172.21.15.43:0/2002186169 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5750086770 con 0x7f575c0adbc0 2024-03-30T13:11:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.079+0000 7f5760872640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5748067fc0 0x7f574806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.083+0000 7f5760872640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5748067fc0 0x7f574806a480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f574c001650 tx=0x7f574c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.083+0000 7f57597fa640 1 -- 172.21.15.43:0/2002186169 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5750056be0 con 0x7f575c0adbc0 2024-03-30T13:11:27.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.239+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f575c070a90 con 0x7f575c0adbc0 2024-03-30T13:11:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.239+0000 7f57597fa640 1 -- 172.21.15.43:0/2002186169 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f57500564e0 con 0x7f575c0adbc0 2024-03-30T13:11:27.247 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5748067fc0 msgr2=0x7f574806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5748067fc0 0x7f574806a480 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f574c001650 tx=0x7f574c009040 comp rx=0 tx=0).stop 2024-03-30T13:11:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0adbc0 msgr2=0x7f575c167ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0adbc0 0x7f575c167ec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f575000e980 tx=0x7f575000ee40 comp rx=0 tx=0).stop 2024-03-30T13:11:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 shutdown_connections 2024-03-30T13:11:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5748067fc0 0x7f574806a480 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f575c147e00 0x7f575c160e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f575c0adbc0 0x7f575c167ec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 --2- 172.21.15.43:0/2002186169 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f575c06c070 0x7f575c0cfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 >> 172.21.15.43:0/2002186169 conn(0x7f575c0d1070 msgr2=0x7f575c0af930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 shutdown_connections 2024-03-30T13:11:27.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.243+0000 7f5762ad6640 1 -- 172.21.15.43:0/2002186169 wait complete. 2024-03-30T13:11:27.264 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f162d3df640 1 Processor -- start 2024-03-30T13:11:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f162d3df640 1 -- start start 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f162d3df640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f162806cfe0 0x7f162806d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f162d3df640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162815d190 con 0x7f162806f9e0 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f162d3df640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f162806bf40 con 0x7f16280d0bc0 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f162d3df640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f162815cf70 con 0x7f162806cfe0 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f16277fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f162806cfe0 0x7f162806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f1626ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f162806f9e0 0x7f162806b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59776/0 (socket says 172.21.15.43:59776) 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f1626ffd640 1 -- 172.21.15.43:0/1963097917 learned_addr learned my addr 172.21.15.43:0/1963097917 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:27.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f16277fe640 1 -- 172.21.15.43:0/1963097917 >> v1:172.21.15.113:6789/0 conn(0x7f16280d0bc0 legacy=0x7f162815c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f16277fe640 1 -- 172.21.15.43:0/1963097917 >> v1:172.21.15.43:6789/0 conn(0x7f162806f9e0 legacy=0x7f162806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f16277fe640 1 -- 172.21.15.43:0/1963097917 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162806c220 con 0x7f162806cfe0 2024-03-30T13:11:27.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.375+0000 7f16267fc640 1 -- 172.21.15.43:0/1963097917 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948629371 0 0) 0x7f162815d190 con 0x7f162806f9e0 2024-03-30T13:11:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f16277fe640 1 --2- 172.21.15.43:0/1963097917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806cfe0 0x7f162806d470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1620009d90 tx=0x7f162001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c49120eb15e5b02 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f16267fc640 1 -- 172.21.15.43:0/1963097917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200b2070 con 0x7f162806cfe0 2024-03-30T13:11:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f16267fc640 1 -- 172.21.15.43:0/1963097917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16200a5830 con 0x7f162806cfe0 2024-03-30T13:11:27.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f16267fc640 1 -- 172.21.15.43:0/1963097917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16200a5c00 con 0x7f162806cfe0 2024-03-30T13:11:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 -- 172.21.15.43:0/1963097917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806cfe0 msgr2=0x7f162806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 --2- 172.21.15.43:0/1963097917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806cfe0 0x7f162806d470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1620009d90 tx=0x7f162001c970 comp rx=0 tx=0).stop 2024-03-30T13:11:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 -- 172.21.15.43:0/1963097917 shutdown_connections 2024-03-30T13:11:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 --2- 172.21.15.43:0/1963097917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806cfe0 0x7f162806d470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 -- 172.21.15.43:0/1963097917 >> 172.21.15.43:0/1963097917 conn(0x7f16280d1070 msgr2=0x7f16280d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 -- 172.21.15.43:0/1963097917 shutdown_connections 2024-03-30T13:11:27.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 -- 172.21.15.43:0/1963097917 wait complete. 2024-03-30T13:11:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 Processor -- start 2024-03-30T13:11:27.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.379+0000 7f162d3df640 1 -- start start 2024-03-30T13:11:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f162806cfe0 0x7f1628159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806f9e0 0x7f162813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16280d0bc0 0x7f16280b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1628169eb0 con 0x7f162806f9e0 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f16280707e0 con 0x7f16280d0bc0 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f162816a030 con 0x7f162806cfe0 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806f9e0 0x7f162813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806f9e0 0x7f162813db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35226/0 (socket says 172.21.15.43:35226) 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 -- 172.21.15.43:0/1735956271 learned_addr learned my addr 172.21.15.43:0/1735956271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f1626ffd640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f162806cfe0 0x7f1628159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f16277fe640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16280d0bc0 0x7f16280b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 -- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f162806cfe0 msgr2=0x7f1628159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f162806cfe0 0x7f1628159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 -- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16280d0bc0 msgr2=0x7f16280b1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16280d0bc0 0x7f16280b1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 -- 172.21.15.43:0/1735956271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1628158530 con 0x7f162806f9e0 2024-03-30T13:11:27.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f1626ffd640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f162806cfe0 0x7f1628159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f16277fe640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16280d0bc0 0x7f16280b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161ffff640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806f9e0 0x7f162813db30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f161000c530 tx=0x7f161000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161f7fe640 1 -- 172.21.15.43:0/1735956271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610016020 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161f7fe640 1 -- 172.21.15.43:0/1735956271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1610005220 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1628157490 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161f7fe640 1 -- 172.21.15.43:0/1735956271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1610007030 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1628157a50 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1628068900 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161f7fe640 1 -- 172.21.15.43:0/1735956271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f16100071d0 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161f7fe640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1608067fc0 0x7f160806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f161f7fe640 1 -- 172.21.15.43:0/1735956271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1610086200 con 0x7f162806f9e0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.383+0000 7f1626ffd640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1608067fc0 0x7f160806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.387+0000 7f1626ffd640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1608067fc0 0x7f160806a480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1618000f70 tx=0x7f1618009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.387+0000 7f161f7fe640 1 -- 172.21.15.43:0/1735956271 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1610020020 con 0x7f162806f9e0 2024-03-30T13:11:27.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.547+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f162806b660 con 0x7f162806f9e0 2024-03-30T13:11:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.547+0000 7f161f7fe640 1 -- 172.21.15.43:0/1735956271 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f16100074c0 con 0x7f162806f9e0 2024-03-30T13:11:27.555 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1608067fc0 msgr2=0x7f160806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1608067fc0 0x7f160806a480 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1618000f70 tx=0x7f1618009040 comp rx=0 tx=0).stop 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806f9e0 msgr2=0x7f162813db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806f9e0 0x7f162813db30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f161000c530 tx=0x7f161000c9f0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 shutdown_connections 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1608067fc0 0x7f160806a480 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16280d0bc0 0x7f16280b1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162806f9e0 0x7f162813db30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 --2- 172.21.15.43:0/1735956271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f162806cfe0 0x7f1628159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 >> 172.21.15.43:0/1735956271 conn(0x7f16280d1070 msgr2=0x7f16281512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 shutdown_connections 2024-03-30T13:11:27.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.551+0000 7f162d3df640 1 -- 172.21.15.43:0/1735956271 wait complete. 2024-03-30T13:11:27.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:27.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0c80a640 1 Processor -- start 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0c80a640 1 -- start start 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0c80a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 0x7fba0806eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0c80a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba0806dc00 con 0x7fba080afb90 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0c80a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba0806dee0 con 0x7fba0815bd70 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0c80a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fba0806e1c0 con 0x7fba0806fd20 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fba080afb90 0x7fba0806f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59778/0 (socket says 172.21.15.43:59778) 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 -- 172.21.15.43:0/340963848 learned_addr learned my addr 172.21.15.43:0/340963848 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba06d9d640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 0x7fba0806eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865777474 0 0) 0x7fba0806dc00 con 0x7fba080afb90 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9f0003530 con 0x7fba080afb90 2024-03-30T13:11:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3448921529 0 0) 0x7fb9f0003530 con 0x7fba080afb90 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 msgr2=0x7fba0806eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 0x7fba0806eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 >> v1:172.21.15.113:6789/0 conn(0x7fba0815bd70 legacy=0x7fba0815c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba0806e4a0 con 0x7fba080afb90 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fb9fc001680 con 0x7fba080afb90 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 >> v1:172.21.15.43:6789/0 conn(0x7fba080afb90 legacy=0x7fba0806f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00039f0 0x7fb9f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9f00043a0 0x7fb9f0004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 0x7fb9f00050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba0806e1c0 con 0x7fb9f00043a0 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f000b9d0 con 0x7fb9f00039f0 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f000dde0 con 0x7fba0806fd20 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9f00043a0 0x7fb9f0004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba0659c640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00039f0 0x7fb9f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba06d9d640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 0x7fb9f00050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 -- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 msgr2=0x7fb9f00050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 0x7fb9f00050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 -- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00039f0 msgr2=0x7fb9f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00039f0 0x7fb9f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba05d9b640 1 -- 172.21.15.43:0/340963848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f0003530 con 0x7fb9f00043a0 2024-03-30T13:11:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.679+0000 7fba06d9d640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba0806fd20 0x7fb9f00050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:27.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba05d9b640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9f00043a0 0x7fb9f0004870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c160 tx=0x7fb9fc01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7481dae1a08c696f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc007cd0 con 0x7fb9f00043a0 2024-03-30T13:11:27.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc0a9070 con 0x7fb9f00043a0 2024-03-30T13:11:27.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0559a640 1 -- 172.21.15.43:0/340963848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc005d20 con 0x7fb9f00043a0 2024-03-30T13:11:27.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 -- 172.21.15.43:0/340963848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9f00043a0 msgr2=0x7fb9f0004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 --2- 172.21.15.43:0/340963848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9f00043a0 0x7fb9f0004870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c160 tx=0x7fb9fc01cbe0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba06d9d640 1 -- 172.21.15.43:0/340963848 reap_dead start 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 -- 172.21.15.43:0/340963848 shutdown_connections 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 -- 172.21.15.43:0/340963848 >> 172.21.15.43:0/340963848 conn(0x7fba080d0c90 msgr2=0x7fba080d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 -- 172.21.15.43:0/340963848 shutdown_connections 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 -- 172.21.15.43:0/340963848 wait complete. 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 Processor -- start 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.683+0000 7fba0c80a640 1 -- start start 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00043a0 0x7fba08158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba080cff30 0x7fba08146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba080ce1e0 0x7fba08156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba0806ee20 con 0x7fba080cff30 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba0806eca0 con 0x7fb9f00043a0 2024-03-30T13:11:27.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fba0806efa0 con 0x7fba080ce1e0 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba080cff30 0x7fba08146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba080cff30 0x7fba08146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35254/0 (socket says 172.21.15.43:35254) 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 -- 172.21.15.43:0/3155282722 learned_addr learned my addr 172.21.15.43:0/3155282722 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0659c640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00043a0 0x7fba08158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba06d9d640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba080ce1e0 0x7fba08156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 -- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba080ce1e0 msgr2=0x7fba08156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba080ce1e0 0x7fba08156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 -- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00043a0 msgr2=0x7fba08158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00043a0 0x7fba08158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 -- 172.21.15.43:0/3155282722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba08146c40 con 0x7fba080cff30 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0659c640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00043a0 0x7fba08158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba05d9b640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba080cff30 0x7fba08146700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc01c650 tx=0x7fb9f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3155282722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0a5af0 con 0x7fba080cff30 2024-03-30T13:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3155282722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9fc0a5c90 con 0x7fba080cff30 2024-03-30T13:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba080d04a0 con 0x7fba080cff30 2024-03-30T13:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3155282722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc004a30 con 0x7fba080cff30 2024-03-30T13:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba08156110 con 0x7fba080cff30 2024-03-30T13:11:27.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba08068900 con 0x7fba080cff30 2024-03-30T13:11:27.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3155282722 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb9fc0b7030 con 0x7fba080cff30 2024-03-30T13:11:27.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fb9f6ffd640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9e8067fc0 0x7fb9e806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3155282722 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb9fc11fce0 con 0x7fba080cff30 2024-03-30T13:11:27.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0659c640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9e8067fc0 0x7fb9e806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.687+0000 7fba0659c640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9e8067fc0 0x7fb9e806a480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8007860 tx=0x7fb9f80077f0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.691+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3155282722 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb9fc0168f0 con 0x7fba080cff30 2024-03-30T13:11:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.847+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fba0815c2d0 con 0x7fba080cff30 2024-03-30T13:11:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.847+0000 7fb9f6ffd640 1 -- 172.21.15.43:0/3155282722 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fb9fc0f01e0 con 0x7fba080cff30 2024-03-30T13:11:27.856 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9e8067fc0 msgr2=0x7fb9e806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9e8067fc0 0x7fb9e806a480 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8007860 tx=0x7fb9f80077f0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba080cff30 msgr2=0x7fba08146700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba080cff30 0x7fba08146700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc01c650 tx=0x7fb9f0005840 comp rx=0 tx=0).stop 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 shutdown_connections 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9e8067fc0 0x7fb9e806a480 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba080ce1e0 0x7fba08156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba080cff30 0x7fba08146700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 --2- 172.21.15.43:0/3155282722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9f00043a0 0x7fba08158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 >> 172.21.15.43:0/3155282722 conn(0x7fba080d0c90 msgr2=0x7fba080b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 shutdown_connections 2024-03-30T13:11:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.851+0000 7fba0c80a640 1 -- 172.21.15.43:0/3155282722 wait complete. 2024-03-30T13:11:27.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc07dd56640 1 Processor -- start 2024-03-30T13:11:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc07dd56640 1 -- start start 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc07dd56640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc07815f880 0x7fc0780b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc07dd56640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc078070640 0x7fc07806e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc07dd56640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0780ce4d0 con 0x7fc07806eec0 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc07dd56640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc078070ad0 con 0x7fc078070640 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc07dd56640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc07806f310 con 0x7fc07815f880 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc076ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc078070640 0x7fc07806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc076ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc078070640 0x7fc07806e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35268/0 (socket says 172.21.15.43:35268) 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc076ffd640 1 -- 172.21.15.43:0/2169274828 learned_addr learned my addr 172.21.15.43:0/2169274828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc0777fe640 1 --2- 172.21.15.43:0/2169274828 >> v2:172.21.15.113:3300/0 conn(0x7fc07815f880 0x7fc0780b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc076ffd640 1 -- 172.21.15.43:0/2169274828 >> v2:172.21.15.113:3300/0 conn(0x7fc07815f880 msgr2=0x7fc0780b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc076ffd640 1 --2- 172.21.15.43:0/2169274828 >> v2:172.21.15.113:3300/0 conn(0x7fc07815f880 0x7fc0780b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc076ffd640 1 -- 172.21.15.43:0/2169274828 >> v1:172.21.15.113:6789/0 conn(0x7fc07806eec0 legacy=0x7fc0780ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:27.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.979+0000 7fc076ffd640 1 -- 172.21.15.43:0/2169274828 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0780cc7f0 con 0x7fc078070640 2024-03-30T13:11:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc076ffd640 1 --2- 172.21.15.43:0/2169274828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc078070640 0x7fc07806e980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc0780b4860 tx=0x7fc06c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de524c52e0dbfd87 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc0767fc640 1 -- 172.21.15.43:0/2169274828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b0070 con 0x7fc078070640 2024-03-30T13:11:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc0767fc640 1 -- 172.21.15.43:0/2169274828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06c0ab600 con 0x7fc078070640 2024-03-30T13:11:27.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc0767fc640 1 -- 172.21.15.43:0/2169274828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0ab9d0 con 0x7fc078070640 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 -- 172.21.15.43:0/2169274828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc078070640 msgr2=0x7fc07806e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 --2- 172.21.15.43:0/2169274828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc078070640 0x7fc07806e980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc0780b4860 tx=0x7fc06c01c3c0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 -- 172.21.15.43:0/2169274828 shutdown_connections 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 --2- 172.21.15.43:0/2169274828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc078070640 0x7fc07806e980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 --2- 172.21.15.43:0/2169274828 >> v2:172.21.15.113:3300/0 conn(0x7fc07815f880 0x7fc0780b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 -- 172.21.15.43:0/2169274828 >> 172.21.15.43:0/2169274828 conn(0x7fc0780b3cf0 msgr2=0x7fc0780b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 -- 172.21.15.43:0/2169274828 shutdown_connections 2024-03-30T13:11:27.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 -- 172.21.15.43:0/2169274828 wait complete. 2024-03-30T13:11:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 Processor -- start 2024-03-30T13:11:27.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.983+0000 7fc07dd56640 1 -- start start 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07806eec0 0x7fc078156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc078070640 0x7fc078146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07815f880 0x7fc078146560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc0780ccf30 con 0x7fc07806eec0 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc0780ccdb0 con 0x7fc078070640 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc0780cf4d0 con 0x7fc07815f880 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07806eec0 0x7fc078156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07806eec0 0x7fc078156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35274/0 (socket says 172.21.15.43:35274) 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 -- 172.21.15.43:0/1684619462 learned_addr learned my addr 172.21.15.43:0/1684619462 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc076ffd640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc078070640 0x7fc078146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 -- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07815f880 msgr2=0x7fc078146560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc077fff640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07815f880 0x7fc078146560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07815f880 0x7fc078146560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 -- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc078070640 msgr2=0x7fc078146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc078070640 0x7fc078146020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 -- 172.21.15.43:0/1684619462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc078166040 con 0x7fc07806eec0 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc076ffd640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc078070640 0x7fc078146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc0777fe640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07806eec0 0x7fc078156e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc06800d620 tx=0x7fc06800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc05ffff640 1 -- 172.21.15.43:0/1684619462 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800b620 con 0x7fc07806eec0 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc05ffff640 1 -- 172.21.15.43:0/1684619462 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc068019070 con 0x7fc07806eec0 2024-03-30T13:11:27.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc078166320 con 0x7fc07806eec0 2024-03-30T13:11:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc05ffff640 1 -- 172.21.15.43:0/1684619462 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800b9e0 con 0x7fc07806eec0 2024-03-30T13:11:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0781668e0 con 0x7fc07806eec0 2024-03-30T13:11:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc078068900 con 0x7fc07806eec0 2024-03-30T13:11:27.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.987+0000 7fc05ffff640 1 -- 172.21.15.43:0/1684619462 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc06800bb80 con 0x7fc07806eec0 2024-03-30T13:11:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.991+0000 7fc05ffff640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc064067ff0 0x7fc06406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.991+0000 7fc05ffff640 1 -- 172.21.15.43:0/1684619462 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc068086ca0 con 0x7fc07806eec0 2024-03-30T13:11:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.991+0000 7fc076ffd640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc064067ff0 0x7fc06406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:27.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.991+0000 7fc076ffd640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc064067ff0 0x7fc06406a4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0ad000 tx=0x7fc06c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:28.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:27.995+0000 7fc05ffff640 1 -- 172.21.15.43:0/1684619462 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc068057090 con 0x7fc07806eec0 2024-03-30T13:11:28.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.179+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc0780b3ad0 con 0x7fc07806eec0 2024-03-30T13:11:28.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc05ffff640 1 -- 172.21.15.43:0/1684619462 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc068056990 con 0x7fc07806eec0 2024-03-30T13:11:28.189 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:28.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc064067ff0 msgr2=0x7fc06406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc064067ff0 0x7fc06406a4b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0ad000 tx=0x7fc06c008040 comp rx=0 tx=0).stop 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07806eec0 msgr2=0x7fc078156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07806eec0 0x7fc078156e30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc06800d620 tx=0x7fc06800dae0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 shutdown_connections 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc064067ff0 0x7fc06406a4b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07815f880 0x7fc078146560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc078070640 0x7fc078146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 --2- 172.21.15.43:0/1684619462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc07806eec0 0x7fc078156e30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 >> 172.21.15.43:0/1684619462 conn(0x7fc0780b3cf0 msgr2=0x7fc07815fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 shutdown_connections 2024-03-30T13:11:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.183+0000 7fc07dd56640 1 -- 172.21.15.43:0/1684619462 wait complete. 2024-03-30T13:11:28.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.311+0000 7f9d29160640 1 Processor -- start 2024-03-30T13:11:28.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.311+0000 7f9d29160640 1 -- start start 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d29160640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d2406f390 0x7f9d241604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d29160640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d2415c610 con 0x7f9d2415c230 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d29160640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d240cc9f0 con 0x7f9d240cfd20 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d29160640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9d240b4fa0 con 0x7f9d2406f390 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d2259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d2406f390 0x7f9d241604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d2259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d2406f390 0x7f9d241604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35276/0 (socket says 172.21.15.43:35276) 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d22d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9d2415c230 0x7f9d2415fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59792/0 (socket says 172.21.15.43:59792) 2024-03-30T13:11:28.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d2259c640 1 -- 172.21.15.43:0/2730839599 learned_addr learned my addr 172.21.15.43:0/2730839599 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d2259c640 1 -- 172.21.15.43:0/2730839599 >> v1:172.21.15.113:6789/0 conn(0x7f9d240cfd20 legacy=0x7f9d240d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d2259c640 1 -- 172.21.15.43:0/2730839599 >> v1:172.21.15.43:6789/0 conn(0x7f9d2415c230 legacy=0x7f9d2415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d2259c640 1 -- 172.21.15.43:0/2730839599 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d240cccd0 con 0x7f9d2406f390 2024-03-30T13:11:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d2259c640 1 --2- 172.21.15.43:0/2730839599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d241604b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009870 tx=0x7f9d0c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a564b26c56bea5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d21d9b640 1 -- 172.21.15.43:0/2730839599 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0b1070 con 0x7f9d2406f390 2024-03-30T13:11:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d21d9b640 1 -- 172.21.15.43:0/2730839599 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0c0a8050 con 0x7f9d2406f390 2024-03-30T13:11:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.315+0000 7f9d21d9b640 1 -- 172.21.15.43:0/2730839599 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0ac860 con 0x7f9d2406f390 2024-03-30T13:11:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- 172.21.15.43:0/2730839599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 msgr2=0x7f9d241604b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 --2- 172.21.15.43:0/2730839599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d241604b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009870 tx=0x7f9d0c01c4d0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- 172.21.15.43:0/2730839599 shutdown_connections 2024-03-30T13:11:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 --2- 172.21.15.43:0/2730839599 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d241604b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- 172.21.15.43:0/2730839599 >> 172.21.15.43:0/2730839599 conn(0x7f9d240b3c80 msgr2=0x7f9d240b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- 172.21.15.43:0/2730839599 shutdown_connections 2024-03-30T13:11:28.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- 172.21.15.43:0/2730839599 wait complete. 2024-03-30T13:11:28.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 Processor -- start 2024-03-30T13:11:28.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- start start 2024-03-30T13:11:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d24151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d240cfd20 0x7f9d240c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2415c230 0x7f9d240c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d240cc7a0 con 0x7f9d2406f390 2024-03-30T13:11:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d240cf4f0 con 0x7f9d240cfd20 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2406f820 con 0x7f9d2415c230 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d24151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d24151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35290/0 (socket says 172.21.15.43:35290) 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 -- 172.21.15.43:0/3819047497 learned_addr learned my addr 172.21.15.43:0/3819047497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d2259c640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d240cfd20 0x7f9d240c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d2359e640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2415c230 0x7f9d240c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 -- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2415c230 msgr2=0x7f9d240c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2415c230 0x7f9d240c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 -- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d240cfd20 msgr2=0x7f9d240c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d240cfd20 0x7f9d240c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 -- 172.21.15.43:0/3819047497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d24160730 con 0x7f9d2406f390 2024-03-30T13:11:28.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d2359e640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2415c230 0x7f9d240c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d2259c640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d240cfd20 0x7f9d240c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d22d9d640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d24151430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400c4c0 tx=0x7f9d1400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d1b7fe640 1 -- 172.21.15.43:0/3819047497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d14018070 con 0x7f9d2406f390 2024-03-30T13:11:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d1b7fe640 1 -- 172.21.15.43:0/3819047497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d14007480 con 0x7f9d2406f390 2024-03-30T13:11:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d24160a10 con 0x7f9d2406f390 2024-03-30T13:11:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d1b7fe640 1 -- 172.21.15.43:0/3819047497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140047f0 con 0x7f9d2406f390 2024-03-30T13:11:28.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.319+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d241591d0 con 0x7f9d2406f390 2024-03-30T13:11:28.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.323+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d24068890 con 0x7f9d2406f390 2024-03-30T13:11:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.323+0000 7f9d1b7fe640 1 -- 172.21.15.43:0/3819047497 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9d1401b070 con 0x7f9d2406f390 2024-03-30T13:11:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.323+0000 7f9d1b7fe640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d04067fc0 0x7f9d0406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.323+0000 7f9d1b7fe640 1 -- 172.21.15.43:0/3819047497 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9d14085af0 con 0x7f9d2406f390 2024-03-30T13:11:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.323+0000 7f9d2259c640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d04067fc0 0x7f9d0406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.323+0000 7f9d2259c640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d04067fc0 0x7f9d0406a480 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0ae000 tx=0x7f9d0c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.327+0000 7f9d1b7fe640 1 -- 172.21.15.43:0/3819047497 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9d14055f60 con 0x7f9d2406f390 2024-03-30T13:11:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d24070230 con 0x7f9d2406f390 2024-03-30T13:11:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d1b7fe640 1 -- 172.21.15.43:0/3819047497 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f9d14055860 con 0x7f9d2406f390 2024-03-30T13:11:28.611 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:28.611 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:11:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:11:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d04067fc0 msgr2=0x7f9d0406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d04067fc0 0x7f9d0406a480 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0ae000 tx=0x7f9d0c008040 comp rx=0 tx=0).stop 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 msgr2=0x7f9d24151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d24151430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400c4c0 tx=0x7f9d1400c980 comp rx=0 tx=0).stop 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 shutdown_connections 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d04067fc0 0x7f9d0406a480 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d2415c230 0x7f9d240c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d240cfd20 0x7f9d240c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 --2- 172.21.15.43:0/3819047497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2406f390 0x7f9d24151430 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 >> 172.21.15.43:0/3819047497 conn(0x7f9d240b3c80 msgr2=0x7f9d240b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 shutdown_connections 2024-03-30T13:11:28.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.603+0000 7f9d29160640 1 -- 172.21.15.43:0/3819047497 wait complete. 2024-03-30T13:11:28.628 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:28.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.727+0000 7f8cabbf0640 1 Processor -- start 2024-03-30T13:11:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.727+0000 7f8cabbf0640 1 -- start start 2024-03-30T13:11:28.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8cabbf0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ca406c850 0x7f8ca415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8cabbf0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ca40b4dc0 0x7f8ca406dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8cabbf0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca40afb90 con 0x7f8ca40b49e0 2024-03-30T13:11:28.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8cabbf0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca40afe70 con 0x7f8ca40b4dc0 2024-03-30T13:11:28.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8cabbf0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca40b0150 con 0x7f8ca406c850 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ca40b4dc0 0x7f8ca406dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ca40b4dc0 0x7f8ca406dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35302/0 (socket says 172.21.15.43:35302) 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 -- 172.21.15.43:0/47237195 learned_addr learned my addr 172.21.15.43:0/47237195 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8ca918b640 1 --2- 172.21.15.43:0/47237195 >> v2:172.21.15.113:3300/0 conn(0x7f8ca406c850 0x7f8ca415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 -- 172.21.15.43:0/47237195 >> v2:172.21.15.113:3300/0 conn(0x7f8ca406c850 msgr2=0x7f8ca415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 --2- 172.21.15.43:0/47237195 >> v2:172.21.15.113:3300/0 conn(0x7f8ca406c850 0x7f8ca415c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 -- 172.21.15.43:0/47237195 >> v1:172.21.15.113:6789/0 conn(0x7f8ca40b49e0 legacy=0x7f8ca40b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 -- 172.21.15.43:0/47237195 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca40b37c0 con 0x7f8ca40b4dc0 2024-03-30T13:11:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8caa18d640 1 --2- 172.21.15.43:0/47237195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca406dad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40b5e10 tx=0x7f8c9c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953c46167b4c2aa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8ca898a640 1 -- 172.21.15.43:0/47237195 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0a55b0 con 0x7f8ca40b4dc0 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8ca898a640 1 -- 172.21.15.43:0/47237195 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c0a5750 con 0x7f8ca40b4dc0 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8ca898a640 1 -- 172.21.15.43:0/47237195 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0a5b20 con 0x7f8ca40b4dc0 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8cabbf0640 1 -- 172.21.15.43:0/47237195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b4dc0 msgr2=0x7f8ca406dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.731+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/47237195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca406dad0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8ca40b5e10 tx=0x7f8c9c01c8f0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- 172.21.15.43:0/47237195 shutdown_connections 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/47237195 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca406dad0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/47237195 >> v2:172.21.15.113:3300/0 conn(0x7f8ca406c850 0x7f8ca415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- 172.21.15.43:0/47237195 >> 172.21.15.43:0/47237195 conn(0x7f8ca40b2b30 msgr2=0x7f8ca40d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- 172.21.15.43:0/47237195 shutdown_connections 2024-03-30T13:11:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- 172.21.15.43:0/47237195 wait complete. 2024-03-30T13:11:28.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 Processor -- start 2024-03-30T13:11:28.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- start start 2024-03-30T13:11:28.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406c850 0x7f8ca4166db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b49e0 0x7f8ca4150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca41488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca406f3d0 con 0x7f8ca40b49e0 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca406f250 con 0x7f8ca40b4dc0 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca4165da0 con 0x7f8ca406c850 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b49e0 0x7f8ca4150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b49e0 0x7f8ca4150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35306/0 (socket says 172.21.15.43:35306) 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 -- 172.21.15.43:0/937063022 learned_addr learned my addr 172.21.15.43:0/937063022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca998c640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406c850 0x7f8ca4166db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8caa18d640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca41488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 -- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406c850 msgr2=0x7f8ca4166db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406c850 0x7f8ca4166db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 -- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca40b4dc0 msgr2=0x7f8ca41488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca41488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 -- 172.21.15.43:0/937063022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca4154ea0 con 0x7f8ca40b49e0 2024-03-30T13:11:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8caa18d640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca41488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8ca918b640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b49e0 0x7f8ca4150ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c7f0 tx=0x7f8c9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8c9a7fc640 1 -- 172.21.15.43:0/937063022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94015070 con 0x7f8ca40b49e0 2024-03-30T13:11:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8c9a7fc640 1 -- 172.21.15.43:0/937063022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c94004240 con 0x7f8ca40b49e0 2024-03-30T13:11:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8c9a7fc640 1 -- 172.21.15.43:0/937063022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94003620 con 0x7f8ca40b49e0 2024-03-30T13:11:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca4155180 con 0x7f8ca40b49e0 2024-03-30T13:11:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.735+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca4155740 con 0x7f8ca40b49e0 2024-03-30T13:11:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.739+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca4068890 con 0x7f8ca40b49e0 2024-03-30T13:11:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.739+0000 7f8c9a7fc640 1 -- 172.21.15.43:0/937063022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8c940039d0 con 0x7f8ca40b49e0 2024-03-30T13:11:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.739+0000 7f8c9a7fc640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:28.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.739+0000 7f8c9a7fc640 1 -- 172.21.15.43:0/937063022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8c94086d80 con 0x7f8ca40b49e0 2024-03-30T13:11:28.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.739+0000 7f8ca998c640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:28.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.739+0000 7f8ca998c640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0001600 tx=0x7f8ca0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:28.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.743+0000 7f8c9a7fc640 1 -- 172.21.15.43:0/937063022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8c94057260 con 0x7f8ca40b49e0 2024-03-30T13:11:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.939+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ca4074ed0 con 0x7f8ca40b49e0 2024-03-30T13:11:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8c9a7fc640 1 -- 172.21.15.43:0/937063022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7f8c94056b60 con 0x7f8ca40b49e0 2024-03-30T13:11:28.949 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:28.950 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 msgr2=0x7f8c8806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0001600 tx=0x7f8ca0009040 comp rx=0 tx=0).stop 2024-03-30T13:11:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b49e0 msgr2=0x7f8ca4150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:28.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b49e0 0x7f8ca4150ef0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c7f0 tx=0x7f8c9400ccb0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 shutdown_connections 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca40b4dc0 0x7f8ca41488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca40b49e0 0x7f8ca4150ef0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 --2- 172.21.15.43:0/937063022 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406c850 0x7f8ca4166db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 >> 172.21.15.43:0/937063022 conn(0x7f8ca40b2b30 msgr2=0x7f8ca4165580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 shutdown_connections 2024-03-30T13:11:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:28.943+0000 7f8cabbf0640 1 -- 172.21.15.43:0/937063022 wait complete. 2024-03-30T13:11:28.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:28.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:29.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.075+0000 7f07b889e640 1 Processor -- start 2024-03-30T13:11:29.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b889e640 1 -- start start 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b889e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f07b415afc0 0x7f07b40b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b889e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f07b40b4520 0x7f07b406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b889e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b40b4ba0 con 0x7f07b406ff20 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b889e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f07b4070c30 con 0x7f07b415afc0 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b889e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f07b40b4960 con 0x7f07b40b4520 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f07b415afc0 0x7f07b40b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f07b406ff20 0x7f07b406c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59794/0 (socket says 172.21.15.43:59794) 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b2d9d640 1 -- 172.21.15.43:0/3317229291 learned_addr learned my addr 172.21.15.43:0/3317229291 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b1d9b640 1 --2- 172.21.15.43:0/3317229291 >> v2:172.21.15.113:3300/0 conn(0x7f07b40b4520 0x7f07b406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b259c640 1 -- 172.21.15.43:0/3317229291 >> v2:172.21.15.113:3300/0 conn(0x7f07b40b4520 msgr2=0x7f07b406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b259c640 1 --2- 172.21.15.43:0/3317229291 >> v2:172.21.15.113:3300/0 conn(0x7f07b40b4520 0x7f07b406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b259c640 1 -- 172.21.15.43:0/3317229291 >> v1:172.21.15.43:6789/0 conn(0x7f07b406ff20 legacy=0x7f07b406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.079+0000 7f07b259c640 1 -- 172.21.15.43:0/3317229291 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b415ec60 con 0x7f07b415afc0 2024-03-30T13:11:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b259c640 1 --2- 172.21.15.43:0/3317229291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b40b3fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f07b40737e0 tx=0x7f07a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74675aef896a9760 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b159a640 1 -- 172.21.15.43:0/3317229291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40b0070 con 0x7f07b415afc0 2024-03-30T13:11:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b159a640 1 -- 172.21.15.43:0/3317229291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a40ab600 con 0x7f07b415afc0 2024-03-30T13:11:29.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b159a640 1 -- 172.21.15.43:0/3317229291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a40ab9d0 con 0x7f07b415afc0 2024-03-30T13:11:29.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- 172.21.15.43:0/3317229291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 msgr2=0x7f07b40b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 --2- 172.21.15.43:0/3317229291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b40b3fe0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f07b40737e0 tx=0x7f07a401c480 comp rx=0 tx=0).stop 2024-03-30T13:11:29.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- 172.21.15.43:0/3317229291 shutdown_connections 2024-03-30T13:11:29.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 --2- 172.21.15.43:0/3317229291 >> v2:172.21.15.113:3300/0 conn(0x7f07b40b4520 0x7f07b406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 --2- 172.21.15.43:0/3317229291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b40b3fe0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- 172.21.15.43:0/3317229291 >> 172.21.15.43:0/3317229291 conn(0x7f07b40d1070 msgr2=0x7f07b40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:29.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- 172.21.15.43:0/3317229291 shutdown_connections 2024-03-30T13:11:29.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- 172.21.15.43:0/3317229291 wait complete. 2024-03-30T13:11:29.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 Processor -- start 2024-03-30T13:11:29.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- start start 2024-03-30T13:11:29.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07b406ff20 0x7f07b413e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b40b4520 0x7f07b415fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b41593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f07b415fa20 con 0x7f07b415afc0 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f07b40cf7c0 con 0x7f07b40b4520 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b889e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f07b415a1b0 con 0x7f07b406ff20 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b41593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b41593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35326/0 (socket says 172.21.15.43:35326) 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b2d9d640 1 -- 172.21.15.43:0/4051822342 learned_addr learned my addr 172.21.15.43:0/4051822342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b1d9b640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b40b4520 0x7f07b415fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.083+0000 7f07b259c640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07b406ff20 0x7f07b413e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b2d9d640 1 -- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07b406ff20 msgr2=0x7f07b413e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b2d9d640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07b406ff20 0x7f07b413e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b2d9d640 1 -- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b40b4520 msgr2=0x7f07b415fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b2d9d640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b40b4520 0x7f07b415fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b2d9d640 1 -- 172.21.15.43:0/4051822342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b415a490 con 0x7f07b415afc0 2024-03-30T13:11:29.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b2d9d640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b41593c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00ca80 tx=0x7f07ac00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07a2ffd640 1 -- 172.21.15.43:0/4051822342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac017070 con 0x7f07b415afc0 2024-03-30T13:11:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07a2ffd640 1 -- 172.21.15.43:0/4051822342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ac00ac30 con 0x7f07b415afc0 2024-03-30T13:11:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b41517b0 con 0x7f07b415afc0 2024-03-30T13:11:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07a2ffd640 1 -- 172.21.15.43:0/4051822342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac012600 con 0x7f07b415afc0 2024-03-30T13:11:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b415a770 con 0x7f07b415afc0 2024-03-30T13:11:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4068900 con 0x7f07b415afc0 2024-03-30T13:11:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07a2ffd640 1 -- 172.21.15.43:0/4051822342 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f07ac01a070 con 0x7f07b415afc0 2024-03-30T13:11:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07a2ffd640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0794067ff0 0x7f079406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07a2ffd640 1 -- 172.21.15.43:0/4051822342 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f07ac086420 con 0x7f07b415afc0 2024-03-30T13:11:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b259c640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0794067ff0 0x7f079406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.087+0000 7f07b259c640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0794067ff0 0x7f079406a4b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07a40ad000 tx=0x7f07a40a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.091+0000 7f07a2ffd640 1 -- 172.21.15.43:0/4051822342 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f07ac00ae20 con 0x7f07b415afc0 2024-03-30T13:11:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.359+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07b40706c0 con 0x7f07b415afc0 2024-03-30T13:11:29.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.359+0000 7f07a2ffd640 1 -- 172.21.15.43:0/4051822342 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f07ac056240 con 0x7f07b415afc0 2024-03-30T13:11:29.398 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0794067ff0 msgr2=0x7f079406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0794067ff0 0x7f079406a4b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07a40ad000 tx=0x7f07a40a6000 comp rx=0 tx=0).stop 2024-03-30T13:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 msgr2=0x7f07b41593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b41593c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00ca80 tx=0x7f07ac00cf40 comp rx=0 tx=0).stop 2024-03-30T13:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 shutdown_connections 2024-03-30T13:11:29.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0794067ff0 0x7f079406a4b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07b415afc0 0x7f07b41593c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f07b40b4520 0x7f07b415fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 --2- 172.21.15.43:0/4051822342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f07b406ff20 0x7f07b413e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 >> 172.21.15.43:0/4051822342 conn(0x7f07b40d1070 msgr2=0x7f07b40d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 shutdown_connections 2024-03-30T13:11:29.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.363+0000 7f07b889e640 1 -- 172.21.15.43:0/4051822342 wait complete. 2024-03-30T13:11:29.400 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.511+0000 7f894160e640 1 Processor -- start 2024-03-30T13:11:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.511+0000 7f894160e640 1 -- start start 2024-03-30T13:11:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f894160e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f893c147e00 0x7f893c0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f894160e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c06c780 0x7f893c0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f894160e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893c0b5470 con 0x7f893c0b4f30 2024-03-30T13:11:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f894160e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f893c0b5750 con 0x7f893c147e00 2024-03-30T13:11:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f894160e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f893c0b5a30 con 0x7f893c06c780 2024-03-30T13:11:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f893c147e00 0x7f893c0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f893c0b4f30 0x7f893c06c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59806/0 (socket says 172.21.15.43:59806) 2024-03-30T13:11:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893a7fc640 1 -- 172.21.15.43:0/1364340876 learned_addr learned my addr 172.21.15.43:0/1364340876 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893b7fe640 1 --2- 172.21.15.43:0/1364340876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c06c780 0x7f893c0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893affd640 1 -- 172.21.15.43:0/1364340876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c06c780 msgr2=0x7f893c0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893affd640 1 --2- 172.21.15.43:0/1364340876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c06c780 0x7f893c0adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893affd640 1 -- 172.21.15.43:0/1364340876 >> v1:172.21.15.43:6789/0 conn(0x7f893c0b4f30 legacy=0x7f893c06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893affd640 1 -- 172.21.15.43:0/1364340876 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c0b5d10 con 0x7f893c147e00 2024-03-30T13:11:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f8939ffb640 1 -- 172.21.15.43:0/1364340876 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3552226306 0 0) 0x7f893c0b5470 con 0x7f893c0b4f30 2024-03-30T13:11:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f893affd640 1 --2- 172.21.15.43:0/1364340876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c147e00 0x7f893c0b49f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f892c0097c0 tx=0x7f892c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b60ad43a29e2bd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f8939ffb640 1 -- 172.21.15.43:0/1364340876 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0b0070 con 0x7f893c147e00 2024-03-30T13:11:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f8939ffb640 1 -- 172.21.15.43:0/1364340876 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c0ab600 con 0x7f893c147e00 2024-03-30T13:11:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.515+0000 7f8939ffb640 1 -- 172.21.15.43:0/1364340876 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0ab9d0 con 0x7f893c147e00 2024-03-30T13:11:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/1364340876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c147e00 msgr2=0x7f893c0b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 --2- 172.21.15.43:0/1364340876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c147e00 0x7f893c0b49f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f892c0097c0 tx=0x7f892c01c4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/1364340876 shutdown_connections 2024-03-30T13:11:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 --2- 172.21.15.43:0/1364340876 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c06c780 0x7f893c0adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 --2- 172.21.15.43:0/1364340876 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c147e00 0x7f893c0b49f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/1364340876 >> 172.21.15.43:0/1364340876 conn(0x7f893c0d1070 msgr2=0x7f893c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/1364340876 shutdown_connections 2024-03-30T13:11:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/1364340876 wait complete. 2024-03-30T13:11:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 Processor -- start 2024-03-30T13:11:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- start start 2024-03-30T13:11:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c06c780 0x7f893c146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c0b4f30 0x7f893c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f893c147e00 0x7f893c167a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f893c0aeca0 con 0x7f893c06c780 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f893c0aeb20 con 0x7f893c147e00 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f893c0aee20 con 0x7f893c0b4f30 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c06c780 0x7f893c146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c06c780 0x7f893c146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35354/0 (socket says 172.21.15.43:35354) 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 -- 172.21.15.43:0/4138239460 learned_addr learned my addr 172.21.15.43:0/4138239460 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893a7fc640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c0b4f30 0x7f893c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893b7fe640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f893c147e00 0x7f893c167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 -- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c0b4f30 msgr2=0x7f893c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c0b4f30 0x7f893c158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 -- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f893c147e00 msgr2=0x7f893c167a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f893c147e00 0x7f893c167a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 -- 172.21.15.43:0/4138239460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c163260 con 0x7f893c06c780 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893a7fc640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c0b4f30 0x7f893c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893b7fe640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f893c147e00 0x7f893c167a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f893affd640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c06c780 0x7f893c146100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f892c01a0d0 tx=0x7f892c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f89237fe640 1 -- 172.21.15.43:0/4138239460 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0b0050 con 0x7f893c06c780 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f89237fe640 1 -- 172.21.15.43:0/4138239460 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c0bc070 con 0x7f893c06c780 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c163540 con 0x7f893c06c780 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f89237fe640 1 -- 172.21.15.43:0/4138239460 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0aac00 con 0x7f893c06c780 2024-03-30T13:11:29.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c163b00 con 0x7f893c06c780 2024-03-30T13:11:29.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.519+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c068900 con 0x7f893c06c780 2024-03-30T13:11:29.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.523+0000 7f89237fe640 1 -- 172.21.15.43:0/4138239460 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f892c0c1460 con 0x7f893c06c780 2024-03-30T13:11:29.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.523+0000 7f89237fe640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8928067fc0 0x7f892806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.523+0000 7f89237fe640 1 -- 172.21.15.43:0/4138239460 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f892c121e40 con 0x7f893c06c780 2024-03-30T13:11:29.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.523+0000 7f893a7fc640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8928067fc0 0x7f892806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.523+0000 7f893a7fc640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8928067fc0 0x7f892806a480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f893c15c3b0 tx=0x7f8930009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.527+0000 7f89237fe640 1 -- 172.21.15.43:0/4138239460 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f892c016090 con 0x7f893c06c780 2024-03-30T13:11:29.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.711+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f893c147270 con 0x7f893c06c780 2024-03-30T13:11:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f89237fe640 1 -- 172.21.15.43:0/4138239460 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f892c0b63b0 con 0x7f893c06c780 2024-03-30T13:11:29.721 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8928067fc0 msgr2=0x7f892806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8928067fc0 0x7f892806a480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f893c15c3b0 tx=0x7f8930009040 comp rx=0 tx=0).stop 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c06c780 msgr2=0x7f893c146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c06c780 0x7f893c146100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f892c01a0d0 tx=0x7f892c01ace0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 shutdown_connections 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8928067fc0 0x7f892806a480 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f893c147e00 0x7f893c167a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f893c0b4f30 0x7f893c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 --2- 172.21.15.43:0/4138239460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f893c06c780 0x7f893c146100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 >> 172.21.15.43:0/4138239460 conn(0x7f893c0d1070 msgr2=0x7f893c0b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 shutdown_connections 2024-03-30T13:11:29.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.715+0000 7f894160e640 1 -- 172.21.15.43:0/4138239460 wait complete. 2024-03-30T13:11:29.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:29.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee9e85640 1 Processor -- start 2024-03-30T13:11:29.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee9e85640 1 -- start start 2024-03-30T13:11:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee9e85640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee9e85640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcee4070640 0x7fcee406e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee9e85640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee4070aa0 con 0x7fcee406eec0 2024-03-30T13:11:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee9e85640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fcee40ce4d0 con 0x7fcee4070640 2024-03-30T13:11:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee9e85640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcee40cc7f0 con 0x7fcee415f880 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcee4070640 0x7fcee406e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcee406eec0 0x7fcee40ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59816/0 (socket says 172.21.15.43:59816) 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcee4070640 0x7fcee406e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35364/0 (socket says 172.21.15.43:35364) 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee3fff640 1 -- 172.21.15.43:0/3943648905 learned_addr learned my addr 172.21.15.43:0/3943648905 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee37fe640 1 --2- 172.21.15.43:0/3943648905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee2ffd640 1 -- 172.21.15.43:0/3943648905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 msgr2=0x7fcee40b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee2ffd640 1 --2- 172.21.15.43:0/3943648905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee2ffd640 1 -- 172.21.15.43:0/3943648905 >> v1:172.21.15.43:6789/0 conn(0x7fcee406eec0 legacy=0x7fcee40ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee2ffd640 1 -- 172.21.15.43:0/3943648905 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee40ccad0 con 0x7fcee4070640 2024-03-30T13:11:29.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.859+0000 7fcee27fc640 1 -- 172.21.15.43:0/3943648905 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214896923 0 0) 0x7fcee4070aa0 con 0x7fcee406eec0 2024-03-30T13:11:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee2ffd640 1 --2- 172.21.15.43:0/3943648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4070640 0x7fcee406e980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0097c0 tx=0x7fcecc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be8be340a9ceb7e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee27fc640 1 -- 172.21.15.43:0/3943648905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0b0070 con 0x7fcee4070640 2024-03-30T13:11:29.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee27fc640 1 -- 172.21.15.43:0/3943648905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc0ab600 con 0x7fcee4070640 2024-03-30T13:11:29.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee27fc640 1 -- 172.21.15.43:0/3943648905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0ab9d0 con 0x7fcee4070640 2024-03-30T13:11:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 -- 172.21.15.43:0/3943648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4070640 msgr2=0x7fcee406e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 --2- 172.21.15.43:0/3943648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4070640 0x7fcee406e980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0097c0 tx=0x7fcecc01c380 comp rx=0 tx=0).stop 2024-03-30T13:11:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 -- 172.21.15.43:0/3943648905 shutdown_connections 2024-03-30T13:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 --2- 172.21.15.43:0/3943648905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee4070640 0x7fcee406e980 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 --2- 172.21.15.43:0/3943648905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 -- 172.21.15.43:0/3943648905 >> 172.21.15.43:0/3943648905 conn(0x7fcee40b3cf0 msgr2=0x7fcee40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 -- 172.21.15.43:0/3943648905 shutdown_connections 2024-03-30T13:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 -- 172.21.15.43:0/3943648905 wait complete. 2024-03-30T13:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 Processor -- start 2024-03-30T13:11:29.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.863+0000 7fcee9e85640 1 -- start start 2024-03-30T13:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee406eec0 0x7fcee40ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee4070640 0x7fcee4158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40cf650 con 0x7fcee406eec0 2024-03-30T13:11:29.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcee40cf4d0 con 0x7fcee4070640 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcee4160850 con 0x7fcee415f880 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee406eec0 0x7fcee40ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee406eec0 0x7fcee40ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35380/0 (socket says 172.21.15.43:35380) 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 -- 172.21.15.43:0/3151839422 learned_addr learned my addr 172.21.15.43:0/3151839422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee3fff640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee2ffd640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee4070640 0x7fcee4158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 -- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 msgr2=0x7fcee40cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 -- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee4070640 msgr2=0x7fcee4158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee4070640 0x7fcee4158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 -- 172.21.15.43:0/3151839422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee40c6fb0 con 0x7fcee406eec0 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee37fe640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee406eec0 0x7fcee40ca220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcedc007ef0 tx=0x7fcedc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcedbfff640 1 -- 172.21.15.43:0/3151839422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc014620 con 0x7fcee406eec0 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcedbfff640 1 -- 172.21.15.43:0/3151839422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcedc019070 con 0x7fcee406eec0 2024-03-30T13:11:29.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 -- 172.21.15.43:0/3151839422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee40c7290 con 0x7fcee406eec0 2024-03-30T13:11:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcedbfff640 1 -- 172.21.15.43:0/3151839422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcedc014a00 con 0x7fcee406eec0 2024-03-30T13:11:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcee9e85640 1 -- 172.21.15.43:0/3151839422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee40c7850 con 0x7fcee406eec0 2024-03-30T13:11:29.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcebc0052a0 con 0x7fcee406eec0 2024-03-30T13:11:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.867+0000 7fcedbfff640 1 -- 172.21.15.43:0/3151839422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcedc014ba0 con 0x7fcee406eec0 2024-03-30T13:11:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.871+0000 7fcedbfff640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcec4067fc0 0x7fcec406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.871+0000 7fcedbfff640 1 -- 172.21.15.43:0/3151839422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fcedc087150 con 0x7fcee406eec0 2024-03-30T13:11:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.871+0000 7fcee2ffd640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcec4067fc0 0x7fcec406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.871+0000 7fcee2ffd640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcec4067fc0 0x7fcec406a480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0ad000 tx=0x7fcecc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:29.875+0000 7fcedbfff640 1 -- 172.21.15.43:0/3151839422 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcedc057650 con 0x7fcee406eec0 2024-03-30T13:11:30.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.035+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcebc005860 con 0x7fcee406eec0 2024-03-30T13:11:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fcedbfff640 1 -- 172.21.15.43:0/3151839422 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fcedc056f50 con 0x7fcee406eec0 2024-03-30T13:11:30.045 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcec4067fc0 msgr2=0x7fcec406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcec4067fc0 0x7fcec406a480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcecc0ad000 tx=0x7fcecc008040 comp rx=0 tx=0).stop 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee406eec0 msgr2=0x7fcee40ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee406eec0 0x7fcee40ca220 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fcedc007ef0 tx=0x7fcedc00e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 shutdown_connections 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcec4067fc0 0x7fcec406a480 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcee415f880 0x7fcee40cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcee4070640 0x7fcee4158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 --2- 172.21.15.43:0/3151839422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcee406eec0 0x7fcee40ca220 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 >> 172.21.15.43:0/3151839422 conn(0x7fcee40b3cf0 msgr2=0x7fcee4159900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 shutdown_connections 2024-03-30T13:11:30.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.039+0000 7fced9ffb640 1 -- 172.21.15.43:0/3151839422 wait complete. 2024-03-30T13:11:30.064 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99570dd640 1 Processor -- start 2024-03-30T13:11:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99570dd640 1 -- start start 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99570dd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f99500cc750 0x7f99500ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99570dd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9950158f20 con 0x7f99500b6c50 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99570dd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99501620f0 con 0x7f9950158b40 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99570dd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9950162d00 con 0x7f99500cc750 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f995567a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f99500b6c50 0x7f99500b7070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59824/0 (socket says 172.21.15.43:59824) 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f995567a640 1 -- 172.21.15.43:0/3389338570 learned_addr learned my addr 172.21.15.43:0/3389338570 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9947fff640 1 --2- 172.21.15.43:0/3389338570 >> v2:172.21.15.113:3300/0 conn(0x7f99500cc750 0x7f99500ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20184561 0 0) 0x7f9950158f20 con 0x7f99500b6c50 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9938003530 con 0x7f99500b6c50 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2071057503 0 0) 0x7f9938003530 con 0x7f99500b6c50 2024-03-30T13:11:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 >> v2:172.21.15.113:3300/0 conn(0x7f99500cc750 msgr2=0x7f99500ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 --2- 172.21.15.43:0/3389338570 >> v2:172.21.15.113:3300/0 conn(0x7f99500cc750 0x7f99500ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 >> v1:172.21.15.113:6789/0 conn(0x7f9950158b40 legacy=0x7f99500b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99501623d0 con 0x7f99500b6c50 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f9948001a00 con 0x7f99500b6c50 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 >> v1:172.21.15.43:6789/0 conn(0x7f99500b6c50 legacy=0x7f99500b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99380039c0 0x7f9938003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938004810 0x7f9938004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99380051d0 0x7f993800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9950162d00 con 0x7f9938004810 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f993800c170 con 0x7f99380051d0 2024-03-30T13:11:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f993800c370 con 0x7f99380039c0 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9947fff640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938004810 0x7f9938004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9954e79640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99380039c0 0x7f9938003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f995567a640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99380051d0 0x7f993800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9947fff640 1 -- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99380039c0 msgr2=0x7f9938003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9947fff640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f99380039c0 0x7f9938003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9947fff640 1 -- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99380051d0 msgr2=0x7f993800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9947fff640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99380051d0 0x7f993800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f9947fff640 1 -- 172.21.15.43:0/3389338570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9938003530 con 0x7f9938004810 2024-03-30T13:11:30.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.187+0000 7f995567a640 1 -- 172.21.15.43:0/3389338570 reap_dead start 2024-03-30T13:11:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f9947fff640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938004810 0x7f9938004c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9940009ae0 tx=0x7f994001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39fb7108704191 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99400b2070 con 0x7f9938004810 2024-03-30T13:11:30.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99400a5610 con 0x7f9938004810 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99477fe640 1 -- 172.21.15.43:0/3389338570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99400096b0 con 0x7f9938004810 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99570dd640 1 -- 172.21.15.43:0/3389338570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938004810 msgr2=0x7f9938004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99570dd640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938004810 0x7f9938004c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9940009ae0 tx=0x7f994001ca40 comp rx=0 tx=0).stop 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99570dd640 1 -- 172.21.15.43:0/3389338570 shutdown_connections 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99570dd640 1 --2- 172.21.15.43:0/3389338570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9938004810 0x7f9938004c90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99570dd640 1 -- 172.21.15.43:0/3389338570 >> 172.21.15.43:0/3389338570 conn(0x7f99500b3cf0 msgr2=0x7f99500b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99570dd640 1 -- 172.21.15.43:0/3389338570 shutdown_connections 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.191+0000 7f99570dd640 1 -- 172.21.15.43:0/3389338570 wait complete. 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 Processor -- start 2024-03-30T13:11:30.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 -- start start 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9938004810 0x7f995014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99501fae60 0x7f99501fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9950151990 0x7f9950151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f99500d1eb0 con 0x7f9950151990 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f99500d1d30 con 0x7f99501fae60 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f99500d0000 con 0x7f9938004810 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9950151990 0x7f9950151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9950151990 0x7f9950151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35402/0 (socket says 172.21.15.43:35402) 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 -- 172.21.15.43:0/4002998566 learned_addr learned my addr 172.21.15.43:0/4002998566 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f9954e79640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9938004810 0x7f995014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f9947fff640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99501fae60 0x7f99501fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 -- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9938004810 msgr2=0x7f995014c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9938004810 0x7f995014c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 -- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99501fae60 msgr2=0x7f99501fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99501fae60 0x7f99501fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 -- 172.21.15.43:0/4002998566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995014c7b0 con 0x7f9950151990 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f995567a640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9950151990 0x7f9950151e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f99500b4810 tx=0x7f9948007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99457fa640 1 -- 172.21.15.43:0/4002998566 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994800e9c0 con 0x7f9950151990 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99457fa640 1 -- 172.21.15.43:0/4002998566 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994800eb60 con 0x7f9950151990 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9950152350 con 0x7f9950151990 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99457fa640 1 -- 172.21.15.43:0/4002998566 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994800ed50 con 0x7f9950151990 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99501528c0 con 0x7f9950151990 2024-03-30T13:11:30.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f995015ea90 con 0x7f9950151990 2024-03-30T13:11:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.195+0000 7f99457fa640 1 -- 172.21.15.43:0/4002998566 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f994801a020 con 0x7f9950151990 2024-03-30T13:11:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.199+0000 7f99457fa640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f993c067fc0 0x7f993c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.199+0000 7f99457fa640 1 -- 172.21.15.43:0/4002998566 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9948086c40 con 0x7f9950151990 2024-03-30T13:11:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.199+0000 7f9954e79640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f993c067fc0 0x7f993c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.199+0000 7f9954e79640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f993c067fc0 0x7f993c06a480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9950074a70 tx=0x7f994c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.203+0000 7f99457fa640 1 -- 172.21.15.43:0/4002998566 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f995015ea90 con 0x7f9950151990 2024-03-30T13:11:30.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.379+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9950068900 con 0x7f9950151990 2024-03-30T13:11:30.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.379+0000 7f99457fa640 1 -- 172.21.15.43:0/4002998566 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f9948021020 con 0x7f9950151990 2024-03-30T13:11:30.388 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f993c067fc0 msgr2=0x7f993c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f993c067fc0 0x7f993c06a480 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9950074a70 tx=0x7f994c009040 comp rx=0 tx=0).stop 2024-03-30T13:11:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9950151990 msgr2=0x7f9950151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9950151990 0x7f9950151e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f99500b4810 tx=0x7f9948007ad0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 shutdown_connections 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f993c067fc0 0x7f993c06a480 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9950151990 0x7f9950151e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f99501fae60 0x7f99501fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 --2- 172.21.15.43:0/4002998566 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9938004810 0x7f995014c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 >> 172.21.15.43:0/4002998566 conn(0x7f99500b3cf0 msgr2=0x7f99500b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 shutdown_connections 2024-03-30T13:11:30.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.383+0000 7f99570dd640 1 -- 172.21.15.43:0/4002998566 wait complete. 2024-03-30T13:11:30.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:30.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.531+0000 7f136a502640 1 Processor -- start 2024-03-30T13:11:30.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.531+0000 7f136a502640 1 -- start start 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f136a502640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f13640b3720 0x7f136415f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f136a502640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f136406f950 0x7f13640b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f136a502640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f136415f870 con 0x7f1364161ea0 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f136a502640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f136406c670 con 0x7f136406f950 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f136a502640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f13640b4040 con 0x7f13640b3720 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f13637fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f136406f950 0x7f13640b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f1368a9f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1364161ea0 0x7f1364162350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59836/0 (socket says 172.21.15.43:59836) 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f13637fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f136406f950 0x7f13640b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35414/0 (socket says 172.21.15.43:35414) 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f1368a9f640 1 -- 172.21.15.43:0/1281708843 learned_addr learned my addr 172.21.15.43:0/1281708843 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f1363fff640 1 --2- 172.21.15.43:0/1281708843 >> v2:172.21.15.113:3300/0 conn(0x7f13640b3720 0x7f136415f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f13637fe640 1 -- 172.21.15.43:0/1281708843 >> v2:172.21.15.113:3300/0 conn(0x7f13640b3720 msgr2=0x7f136415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:30.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f13637fe640 1 --2- 172.21.15.43:0/1281708843 >> v2:172.21.15.113:3300/0 conn(0x7f13640b3720 0x7f136415f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f13637fe640 1 -- 172.21.15.43:0/1281708843 >> v1:172.21.15.43:6789/0 conn(0x7f1364161ea0 legacy=0x7f1364162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f13637fe640 1 -- 172.21.15.43:0/1281708843 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136406dc30 con 0x7f136406f950 2024-03-30T13:11:30.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.535+0000 7f1362ffd640 1 -- 172.21.15.43:0/1281708843 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865285119 0 0) 0x7f1354001120 con 0x7f1364161ea0 2024-03-30T13:11:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 --2- 172.21.15.43:0/1281708843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f136406f950 0x7f13640b3b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13640680f0 tx=0x7f134c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212a269727ef11f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f1362ffd640 1 -- 172.21.15.43:0/1281708843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0ab650 con 0x7f136406f950 2024-03-30T13:11:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f1362ffd640 1 -- 172.21.15.43:0/1281708843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134c0b0070 con 0x7f136406f950 2024-03-30T13:11:30.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f1362ffd640 1 -- 172.21.15.43:0/1281708843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c0aba10 con 0x7f136406f950 2024-03-30T13:11:30.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- 172.21.15.43:0/1281708843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f136406f950 msgr2=0x7f13640b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 --2- 172.21.15.43:0/1281708843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f136406f950 0x7f13640b3b00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13640680f0 tx=0x7f134c01c450 comp rx=0 tx=0).stop 2024-03-30T13:11:30.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- 172.21.15.43:0/1281708843 shutdown_connections 2024-03-30T13:11:30.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 --2- 172.21.15.43:0/1281708843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f136406f950 0x7f13640b3b00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 --2- 172.21.15.43:0/1281708843 >> v2:172.21.15.113:3300/0 conn(0x7f13640b3720 0x7f136415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- 172.21.15.43:0/1281708843 >> 172.21.15.43:0/1281708843 conn(0x7f13640b2b30 msgr2=0x7f13640d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:30.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- 172.21.15.43:0/1281708843 shutdown_connections 2024-03-30T13:11:30.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- 172.21.15.43:0/1281708843 wait complete. 2024-03-30T13:11:30.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 Processor -- start 2024-03-30T13:11:30.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- start start 2024-03-30T13:11:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f136406f950 0x7f13640af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13640b3720 0x7f136415d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1364161ea0 0x7f1364154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f136406ecb0 con 0x7f13640b3720 2024-03-30T13:11:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f136406e790 con 0x7f1364161ea0 2024-03-30T13:11:30.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f136a502640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f136406ee30 con 0x7f136406f950 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13640b3720 0x7f136415d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13640b3720 0x7f136415d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35420/0 (socket says 172.21.15.43:35420) 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 -- 172.21.15.43:0/3215237544 learned_addr learned my addr 172.21.15.43:0/3215237544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f1363fff640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f136406f950 0x7f13640af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f1368a9f640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1364161ea0 0x7f1364154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 -- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f136406f950 msgr2=0x7f13640af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f136406f950 0x7f13640af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 -- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1364161ea0 msgr2=0x7f1364154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1364161ea0 0x7f1364154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f13637fe640 1 -- 172.21.15.43:0/3215237544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13640ae540 con 0x7f13640b3720 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.539+0000 7f1368a9f640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1364161ea0 0x7f1364154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f13637fe640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13640b3720 0x7f136415d780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f134c019060 tx=0x7f134c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1360ff9640 1 -- 172.21.15.43:0/3215237544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c006e20 con 0x7f13640b3720 2024-03-30T13:11:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1360ff9640 1 -- 172.21.15.43:0/3215237544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f134c004230 con 0x7f13640b3720 2024-03-30T13:11:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1360ff9640 1 -- 172.21.15.43:0/3215237544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c003620 con 0x7f13640b3720 2024-03-30T13:11:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13640ae820 con 0x7f13640b3720 2024-03-30T13:11:30.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1364144f30 con 0x7f13640b3720 2024-03-30T13:11:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1360ff9640 1 -- 172.21.15.43:0/3215237544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f134c0037c0 con 0x7f13640b3720 2024-03-30T13:11:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1364068890 con 0x7f13640b3720 2024-03-30T13:11:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1360ff9640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1344067fc0 0x7f134406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1360ff9640 1 -- 172.21.15.43:0/3215237544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f134c121370 con 0x7f13640b3720 2024-03-30T13:11:30.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1363fff640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1344067fc0 0x7f134406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.543+0000 7f1363fff640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1344067fc0 0x7f134406a480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f135c003fd0 tx=0x7f135c0039a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.547+0000 7f1360ff9640 1 -- 172.21.15.43:0/3215237544 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f134c016070 con 0x7f13640b3720 2024-03-30T13:11:30.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.743+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1364161650 con 0x7f13640b3720 2024-03-30T13:11:30.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.743+0000 7f1360ff9640 1 -- 172.21.15.43:0/3215237544 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f134c0f1850 con 0x7f13640b3720 2024-03-30T13:11:30.752 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:30.752 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:11:30.752 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1344067fc0 msgr2=0x7f134406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1344067fc0 0x7f134406a480 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f135c003fd0 tx=0x7f135c0039a0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13640b3720 msgr2=0x7f136415d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13640b3720 0x7f136415d780 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f134c019060 tx=0x7f134c01ac90 comp rx=0 tx=0).stop 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 shutdown_connections 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1344067fc0 0x7f134406a480 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1364161ea0 0x7f1364154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13640b3720 0x7f136415d780 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 --2- 172.21.15.43:0/3215237544 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f136406f950 0x7f13640af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 >> 172.21.15.43:0/3215237544 conn(0x7f13640b2b30 msgr2=0x7f136406efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:30.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 shutdown_connections 2024-03-30T13:11:30.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.747+0000 7f136a502640 1 -- 172.21.15.43:0/3215237544 wait complete. 2024-03-30T13:11:30.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:30.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 Processor -- start 2024-03-30T13:11:30.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 -- start start 2024-03-30T13:11:30.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f133c15b0f0 0x7f133c0b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f133c06c770 0x7f133c0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f133c0b5760 con 0x7f133c06c770 2024-03-30T13:11:30.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f133c06e660 con 0x7f133c15b0f0 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f1342ae2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f133c06e940 con 0x7f133c0afc20 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f133c06c770 0x7f133c0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f133c06c770 0x7f133c0b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35432/0 (socket says 172.21.15.43:35432) 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 -- 172.21.15.43:0/881314229 learned_addr learned my addr 172.21.15.43:0/881314229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134087e640 1 --2- 172.21.15.43:0/881314229 >> v2:172.21.15.113:3300/0 conn(0x7f133c15b0f0 0x7f133c0b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f133bfff640 1 --2- 172.21.15.43:0/881314229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 -- 172.21.15.43:0/881314229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 msgr2=0x7f133c0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 --2- 172.21.15.43:0/881314229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c0b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 -- 172.21.15.43:0/881314229 >> v2:172.21.15.113:3300/0 conn(0x7f133c15b0f0 msgr2=0x7f133c0b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 --2- 172.21.15.43:0/881314229 >> v2:172.21.15.113:3300/0 conn(0x7f133c15b0f0 0x7f133c0b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.883+0000 7f134107f640 1 -- 172.21.15.43:0/881314229 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c06ec20 con 0x7f133c06c770 2024-03-30T13:11:30.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f134107f640 1 --2- 172.21.15.43:0/881314229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06c770 0x7f133c0b3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f133c0b2f70 tx=0x7f133401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd200b84912c393a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f133b7fe640 1 -- 172.21.15.43:0/881314229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340a5580 con 0x7f133c06c770 2024-03-30T13:11:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f133b7fe640 1 -- 172.21.15.43:0/881314229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13340a5720 con 0x7f133c06c770 2024-03-30T13:11:30.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f133b7fe640 1 -- 172.21.15.43:0/881314229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340a5af0 con 0x7f133c06c770 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- 172.21.15.43:0/881314229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06c770 msgr2=0x7f133c0b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 --2- 172.21.15.43:0/881314229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06c770 0x7f133c0b3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f133c0b2f70 tx=0x7f133401c700 comp rx=0 tx=0).stop 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- 172.21.15.43:0/881314229 shutdown_connections 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 --2- 172.21.15.43:0/881314229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06c770 0x7f133c0b3680 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 --2- 172.21.15.43:0/881314229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 --2- 172.21.15.43:0/881314229 >> v2:172.21.15.113:3300/0 conn(0x7f133c15b0f0 0x7f133c0b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- 172.21.15.43:0/881314229 >> 172.21.15.43:0/881314229 conn(0x7f133c0b2980 msgr2=0x7f133c0d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- 172.21.15.43:0/881314229 shutdown_connections 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- 172.21.15.43:0/881314229 wait complete. 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 Processor -- start 2024-03-30T13:11:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- start start 2024-03-30T13:11:30.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f133c06c770 0x7f133c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c15b0f0 0x7f133c14dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f133c0ce370 con 0x7f133c15b0f0 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f133c15b4d0 con 0x7f133c06c770 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.887+0000 7f1342ae2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f133c0ce4f0 con 0x7f133c0afc20 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c15b0f0 0x7f133c14dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c15b0f0 0x7f133c14dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35438/0 (socket says 172.21.15.43:35438) 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 -- 172.21.15.43:0/864489035 learned_addr learned my addr 172.21.15.43:0/864489035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f133bfff640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134087e640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f133c06c770 0x7f133c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 -- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 msgr2=0x7f133c153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 -- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f133c06c770 msgr2=0x7f133c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:30.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f133c06c770 0x7f133c158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 -- 172.21.15.43:0/864489035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c153860 con 0x7f133c15b0f0 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134087e640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f133c06c770 0x7f133c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134107f640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c15b0f0 0x7f133c14dc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13340af040 tx=0x7f13340194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f13397fa640 1 -- 172.21.15.43:0/864489035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340bc020 con 0x7f133c15b0f0 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f13397fa640 1 -- 172.21.15.43:0/864489035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13340b8670 con 0x7f133c15b0f0 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c153b00 con 0x7f133c15b0f0 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f13397fa640 1 -- 172.21.15.43:0/864489035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334009510 con 0x7f133c15b0f0 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1540c0 con 0x7f133c15b0f0 2024-03-30T13:11:30.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c068890 con 0x7f133c15b0f0 2024-03-30T13:11:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f13397fa640 1 -- 172.21.15.43:0/864489035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f13340bf020 con 0x7f133c15b0f0 2024-03-30T13:11:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f13397fa640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1320067fc0 0x7f132006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f13397fa640 1 -- 172.21.15.43:0/864489035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1334121c80 con 0x7f133c15b0f0 2024-03-30T13:11:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.891+0000 7f134087e640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1320067fc0 0x7f132006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:30.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.895+0000 7f134087e640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1320067fc0 0x7f132006a480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f133c0b55a0 tx=0x7f1330008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:30.903+0000 7f13397fa640 1 -- 172.21.15.43:0/864489035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f13340c6030 con 0x7f133c15b0f0 2024-03-30T13:11:31.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.063+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f133c155910 con 0x7f133c15b0f0 2024-03-30T13:11:31.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.063+0000 7f13397fa640 1 -- 172.21.15.43:0/864489035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18494 (secure 0 0 0) 0x7f13340165e0 con 0x7f133c15b0f0 2024-03-30T13:11:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:31.072 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:00.364769+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:31.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1320067fc0 msgr2=0x7f132006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1320067fc0 0x7f132006a480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f133c0b55a0 tx=0x7f1330008040 comp rx=0 tx=0).stop 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c15b0f0 msgr2=0x7f133c14dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c15b0f0 0x7f133c14dc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f13340af040 tx=0x7f13340194a0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 shutdown_connections 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1320067fc0 0x7f132006a480 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c15b0f0 0x7f133c14dc60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133c0afc20 0x7f133c153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 --2- 172.21.15.43:0/864489035 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f133c06c770 0x7f133c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 >> 172.21.15.43:0/864489035 conn(0x7f133c0b2980 msgr2=0x7f133c0cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 shutdown_connections 2024-03-30T13:11:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.067+0000 7f1342ae2640 1 -- 172.21.15.43:0/864489035 wait complete. 2024-03-30T13:11:31.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:11:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.211+0000 7f9a4da3e640 1 Processor -- start 2024-03-30T13:11:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.211+0000 7f9a4da3e640 1 -- start start 2024-03-30T13:11:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a4da3e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 0x7f9a4815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a4da3e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9a4815bf00 0x7f9a480b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a4da3e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4815bd00 con 0x7f9a480b4600 2024-03-30T13:11:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a4da3e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9a480b4a70 con 0x7f9a4815bf00 2024-03-30T13:11:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a4da3e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9a480b39e0 con 0x7f9a480d00c0 2024-03-30T13:11:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9a480b4600 0x7f9a48157a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32980/0 (socket says 172.21.15.43:32980) 2024-03-30T13:11:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 -- 172.21.15.43:0/2239345973 learned_addr learned my addr 172.21.15.43:0/2239345973 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a46ffd640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 0x7f9a4815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877896343 0 0) 0x7f9a38001120 con 0x7f9a480b4600 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a467fc640 1 --2- 172.21.15.43:0/2239345973 >> v2:172.21.15.113:3300/0 conn(0x7f9a4815bf00 0x7f9a480b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a30003530 con 0x7f9a480b4600 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3770218616 0 0) 0x7f9a30003530 con 0x7f9a480b4600 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 msgr2=0x7f9a4815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 0x7f9a4815b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 >> v2:172.21.15.113:3300/0 conn(0x7f9a4815bf00 msgr2=0x7f9a480b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 --2- 172.21.15.43:0/2239345973 >> v2:172.21.15.113:3300/0 conn(0x7f9a4815bf00 0x7f9a480b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480b3cc0 con 0x7f9a480b4600 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f9a38001850 con 0x7f9a480b4600 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 >> v1:172.21.15.43:6789/0 conn(0x7f9a480b4600 legacy=0x7f9a48157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 0x7f9a300039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a30003f40 0x7f9a30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a30004d50 0x7f9a300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480b4a70 con 0x7f9a30004d50 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480b39e0 con 0x7f9a30003f40 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3000dbe0 con 0x7f9a480d00c0 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a30004d50 0x7f9a300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a46ffd640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 0x7f9a300039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a467fc640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a30003f40 0x7f9a30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 -- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 msgr2=0x7f9a300039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a480d00c0 0x7f9a300039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 -- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a30003f40 msgr2=0x7f9a30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a30003f40 0x7f9a30004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.215+0000 7f9a477fe640 1 -- 172.21.15.43:0/2239345973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a30003530 con 0x7f9a30004d50 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a477fe640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a30004d50 0x7f9a300051d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800c4c0 tx=0x7f9a3801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f70e07a998098ee server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3800c0e0 con 0x7f9a30004d50 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a380a9020 con 0x7f9a30004d50 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a45ffb640 1 -- 172.21.15.43:0/2239345973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a380a5bc0 con 0x7f9a30004d50 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- 172.21.15.43:0/2239345973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a30004d50 msgr2=0x7f9a300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 --2- 172.21.15.43:0/2239345973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a30004d50 0x7f9a300051d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800c4c0 tx=0x7f9a3801cc60 comp rx=0 tx=0).stop 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a477fe640 1 -- 172.21.15.43:0/2239345973 reap_dead start 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- 172.21.15.43:0/2239345973 shutdown_connections 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- 172.21.15.43:0/2239345973 >> 172.21.15.43:0/2239345973 conn(0x7f9a480b29b0 msgr2=0x7f9a480d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:31.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- 172.21.15.43:0/2239345973 shutdown_connections 2024-03-30T13:11:31.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- 172.21.15.43:0/2239345973 wait complete. 2024-03-30T13:11:31.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 Processor -- start 2024-03-30T13:11:31.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- start start 2024-03-30T13:11:31.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a48140b00 0x7f9a480ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a480ce6c0 0x7f9a48141490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a481419d0 0x7f9a481568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4806bcc0 con 0x7f9a48140b00 2024-03-30T13:11:31.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4806b9e0 con 0x7f9a480ce6c0 2024-03-30T13:11:31.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9a480b36b0 con 0x7f9a481419d0 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a48140b00 0x7f9a480ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a48140b00 0x7f9a480ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32886/0 (socket says 172.21.15.43:32886) 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a477fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a481419d0 0x7f9a481568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 -- 172.21.15.43:0/4034610589 learned_addr learned my addr 172.21.15.43:0/4034610589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a467fc640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a480ce6c0 0x7f9a48141490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 -- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a481419d0 msgr2=0x7f9a481568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a481419d0 0x7f9a481568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 -- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a480ce6c0 msgr2=0x7f9a48141490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a480ce6c0 0x7f9a48141490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 -- 172.21.15.43:0/4034610589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a48142200 con 0x7f9a48140b00 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a46ffd640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a48140b00 0x7f9a480ce180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a40007f90 tx=0x7f9a40011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a2ffff640 1 -- 172.21.15.43:0/4034610589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4001a020 con 0x7f9a48140b00 2024-03-30T13:11:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a2ffff640 1 -- 172.21.15.43:0/4034610589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a40011e90 con 0x7f9a48140b00 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a481424e0 con 0x7f9a48140b00 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a2ffff640 1 -- 172.21.15.43:0/4034610589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a400168c0 con 0x7f9a48140b00 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.219+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a48142aa0 con 0x7f9a48140b00 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.223+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a48068f00 con 0x7f9a48140b00 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.223+0000 7f9a2ffff640 1 -- 172.21.15.43:0/4034610589 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9a4001d020 con 0x7f9a48140b00 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.223+0000 7f9a2ffff640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9a34067ff0 0x7f9a3406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.223+0000 7f9a2ffff640 1 -- 172.21.15.43:0/4034610589 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9a40087600 con 0x7f9a48140b00 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.223+0000 7f9a467fc640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9a34067ff0 0x7f9a3406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.223+0000 7f9a467fc640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9a34067ff0 0x7f9a3406a4b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a48144460 tx=0x7f9a3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.227+0000 7f9a2ffff640 1 -- 172.21.15.43:0/4034610589 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9a40024030 con 0x7f9a48140b00 2024-03-30T13:11:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.615+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a480ceaa0 con 0x7f9a48140b00 2024-03-30T13:11:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.615+0000 7f9a2ffff640 1 -- 172.21.15.43:0/4034610589 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f9a40057420 con 0x7f9a48140b00 2024-03-30T13:11:31.625 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:31.625 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:11:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9a34067ff0 msgr2=0x7f9a3406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9a34067ff0 0x7f9a3406a4b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9a48144460 tx=0x7f9a3c009040 comp rx=0 tx=0).stop 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a48140b00 msgr2=0x7f9a480ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a48140b00 0x7f9a480ce180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9a40007f90 tx=0x7f9a40011660 comp rx=0 tx=0).stop 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 shutdown_connections 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9a34067ff0 0x7f9a3406a4b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9a481419d0 0x7f9a481568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9a480ce6c0 0x7f9a48141490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 --2- 172.21.15.43:0/4034610589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a48140b00 0x7f9a480ce180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 >> 172.21.15.43:0/4034610589 conn(0x7f9a480b29b0 msgr2=0x7f9a480cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 shutdown_connections 2024-03-30T13:11:31.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:31.619+0000 7f9a4da3e640 1 -- 172.21.15.43:0/4034610589 wait complete. 2024-03-30T13:11:32.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T13:10:55.286025+0000', 'modified': '2024-03-30T13:11:18.390289+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4239}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4239': {'gid': 4239, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/855012184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 855012184}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 855012184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4239, 'qdb_cluster': [4239]} 2024-03-30T13:11:32.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T13:11:32.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-30T13:11:32.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51aac0640 1 Processor -- start 2024-03-30T13:11:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51aac0640 1 -- start start 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51aac0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe51413f0a0 0x7fe514160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51aac0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5141609b0 0x7fe51406d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51aac0640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe514141970 con 0x7fe51406dcc0 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51aac0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe514141c50 con 0x7fe51413f0a0 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51aac0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe514141f30 con 0x7fe5141609b0 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe513fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5141609b0 0x7fe51406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51885c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe51413f0a0 0x7fe514160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51885c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe51413f0a0 0x7fe514160470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:54518/0 (socket says 172.21.15.43:54518) 2024-03-30T13:11:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51885c640 1 -- 172.21.15.43:0/4191341577 learned_addr learned my addr 172.21.15.43:0/4191341577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:32.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe51905d640 1 --1- 172.21.15.43:0/4191341577 >> v1:172.21.15.113:6789/0 conn(0x7fe51406dcc0 0x7fe51406b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:56994/0 (socket says 172.21.15.43:56994) 2024-03-30T13:11:32.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe5137fe640 1 -- 172.21.15.43:0/4191341577 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2823331727 0 0) 0x7fe514141970 con 0x7fe51406dcc0 2024-03-30T13:11:32.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe5137fe640 1 -- 172.21.15.43:0/4191341577 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe500003530 con 0x7fe51406dcc0 2024-03-30T13:11:32.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe513fff640 1 -- 172.21.15.43:0/4191341577 >> v2:172.21.15.113:3300/0 conn(0x7fe51413f0a0 msgr2=0x7fe514160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe513fff640 1 --2- 172.21.15.43:0/4191341577 >> v2:172.21.15.113:3300/0 conn(0x7fe51413f0a0 0x7fe514160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe513fff640 1 -- 172.21.15.43:0/4191341577 >> v1:172.21.15.113:6789/0 conn(0x7fe51406dcc0 legacy=0x7fe51406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.171+0000 7fe513fff640 1 -- 172.21.15.43:0/4191341577 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe514142210 con 0x7fe5141609b0 2024-03-30T13:11:32.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe513fff640 1 --2- 172.21.15.43:0/4191341577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5141609b0 0x7fe51406d780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0097c0 tx=0x7fe4fc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a98963704fa504d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe5137fe640 1 -- 172.21.15.43:0/4191341577 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0b1070 con 0x7fe5141609b0 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe5137fe640 1 -- 172.21.15.43:0/4191341577 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0ac410 con 0x7fe5141609b0 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe5137fe640 1 -- 172.21.15.43:0/4191341577 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0ac7c0 con 0x7fe5141609b0 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 -- 172.21.15.43:0/4191341577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5141609b0 msgr2=0x7fe51406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 --2- 172.21.15.43:0/4191341577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5141609b0 0x7fe51406d780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0097c0 tx=0x7fe4fc01c4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 -- 172.21.15.43:0/4191341577 shutdown_connections 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 --2- 172.21.15.43:0/4191341577 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe5141609b0 0x7fe51406d780 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 --2- 172.21.15.43:0/4191341577 >> v2:172.21.15.113:3300/0 conn(0x7fe51413f0a0 0x7fe514160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 -- 172.21.15.43:0/4191341577 >> 172.21.15.43:0/4191341577 conn(0x7fe514072500 msgr2=0x7fe514072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 -- 172.21.15.43:0/4191341577 shutdown_connections 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 -- 172.21.15.43:0/4191341577 wait complete. 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 Processor -- start 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.175+0000 7fe51aac0640 1 -- start start 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe51406dcc0 0x7fe514158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51413f0a0 0x7fe5141577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5141609b0 0x7fe51415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe514169030 con 0x7fe51413f0a0 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe514168eb0 con 0x7fe5141609b0 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe5141691b0 con 0x7fe51406dcc0 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51413f0a0 0x7fe5141577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51413f0a0 0x7fe5141577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32892/0 (socket says 172.21.15.43:32892) 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 -- 172.21.15.43:0/2791050319 learned_addr learned my addr 172.21.15.43:0/2791050319 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51885c640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe51406dcc0 0x7fe514158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51905d640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5141609b0 0x7fe51415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 -- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe51406dcc0 msgr2=0x7fe514158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe51406dcc0 0x7fe514158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 -- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5141609b0 msgr2=0x7fe51415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5141609b0 0x7fe51415a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 -- 172.21.15.43:0/2791050319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51413df90 con 0x7fe51413f0a0 2024-03-30T13:11:32.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51905d640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5141609b0 0x7fe51415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe513fff640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51413f0a0 0x7fe5141577a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01a0d0 tx=0x7fe4fc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe5117fa640 1 -- 172.21.15.43:0/2791050319 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0b1040 con 0x7fe51413f0a0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe5117fa640 1 -- 172.21.15.43:0/2791050319 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0bc070 con 0x7fe51413f0a0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe514139cd0 con 0x7fe51413f0a0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe5117fa640 1 -- 172.21.15.43:0/2791050319 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc006e20 con 0x7fe51413f0a0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51413e510 con 0x7fe51413f0a0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.179+0000 7fe5117fa640 1 -- 172.21.15.43:0/2791050319 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe4fc0b7640 con 0x7fe51413f0a0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe5117fa640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe4f4067fc0 0x7fe4f406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe5117fa640 1 -- 172.21.15.43:0/2791050319 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe4fc120b40 con 0x7fe51413f0a0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51885c640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe4f4067fc0 0x7fe4f406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 Processor -- start 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- start start 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51885c640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe4f4067fc0 0x7fe4f406a480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009d80 tx=0x7fe50c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3cfbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfc3c0 0x55aea3caef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3caf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aea3afd950 con 0x55aea3cfc3c0 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aea3a9a990 con 0x55aea3caf4c0 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aea3af9900 con 0x55aea3cfbaa0 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfc3c0 0x55aea3caef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfc3c0 0x55aea3caef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32902/0 (socket says 172.21.15.43:32902) 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 -- 172.21.15.43:0/1772053243 learned_addr learned my addr 172.21.15.43:0/1772053243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51905d640 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3caf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51885c640 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3cfbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 -- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 msgr2=0x55aea3cfbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3cfbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 -- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 msgr2=0x55aea3caf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3caf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 -- 172.21.15.43:0/1772053243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aea3b42100 con 0x55aea3cfc3c0 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51885c640 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3cfbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe513fff640 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfc3c0 0x55aea3caef80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0f8ec0 tx=0x7fe4fc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687970e2d7e03c61 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe50a7fc640 1 -- 172.21.15.43:0/1772053243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0f93f0 con 0x55aea3cfc3c0 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe50a7fc640 1 -- 172.21.15.43:0/1772053243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0eeb10 con 0x55aea3cfc3c0 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe50a7fc640 1 -- 172.21.15.43:0/1772053243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc0f8f60 con 0x55aea3cfc3c0 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- 172.21.15.43:0/1772053243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfc3c0 msgr2=0x55aea3caef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfc3c0 0x55aea3caef80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0f8ec0 tx=0x7fe4fc009190 comp rx=0 tx=0).stop 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- 172.21.15.43:0/1772053243 shutdown_connections 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3caf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfc3c0 0x55aea3caef80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1772053243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3cfbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- 172.21.15.43:0/1772053243 >> 172.21.15.43:0/1772053243 conn(0x55aea3b95b80 msgr2=0x55aea3cda460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- 172.21.15.43:0/1772053243 shutdown_connections 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- 172.21.15.43:0/1772053243 wait complete. 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 -- start start 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.183+0000 7fe51dc42000 1 Processor -- start 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d03630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d03b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfc3c0 0x55aea3cfc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aea3a957b0 con 0x55aea3cfbaa0 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aea3abb3e0 con 0x55aea3caf4c0 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aea3afd950 con 0x55aea3cfc3c0 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 10 client.0 _ll_register_callbacks cb 0x55aea3b5e0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:32.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d03b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d03b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32918/0 (socket says 172.21.15.43:32918) 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 -- 172.21.15.43:0/2389199824 learned_addr learned my addr 172.21.15.43:0/2389199824 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51885c640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d03630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51905d640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfc3c0 0x55aea3cfc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 -- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfc3c0 msgr2=0x55aea3cfc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfc3c0 0x55aea3cfc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 -- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 msgr2=0x55aea3d03630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d03630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 -- 172.21.15.43:0/2389199824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aea3ac78c0 con 0x55aea3cfbaa0 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51885c640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe513fff640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d03b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01ca10 tx=0x7fe4fc000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11415ddcefed8c3c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 1 -- 172.21.15.43:0/2389199824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc10e060 con 0x55aea3cfbaa0 2024-03-30T13:11:32.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 1 -- 172.21.15.43:0/2389199824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe4fc0f8f60 con 0x55aea3cfbaa0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4fc0eeb10 con 0x55aea3cfbaa0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 1 -- 172.21.15.43:0/2389199824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fe4fc0f93f0 con 0x55aea3cfbaa0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 10 client.4851 fetch_fsmap learned FSMap version 9 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55aea3a85e80 con 0x55aea3cfbaa0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 1 -- 172.21.15.43:0/2389199824 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc0eeb10 con 0x55aea3cfbaa0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 1 -- 172.21.15.43:0/2389199824 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fe4fc0b4030 con 0x55aea3cfbaa0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 10 client.4851 fetch_fsmap finished waiting for FSMap version 9 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 10 client.4851 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 20 client.4851 populate_metadata read hostname 'smithi043' 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfd3c0 0x55aea3cfd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 4 client.4851 mds_command: new command op to 4239 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 --> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aea3a98bc0 con 0x55aea3cfd3c0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51885c640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfd3c0 0x55aea3cfd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51885c640 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfd3c0 0x55aea3cfd7a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ec4348f2bd38ff server_cookie=a30b6a70f8c74c19 in_seq=0 out_seq=0 2024-03-30T13:11:32.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 10 client.4851 ms_handle_connect on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 1 -- 172.21.15.43:0/2389199824 <== mds.0 v2:172.21.15.43:6834/855012184 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55aea3a98bc0 con 0x55aea3cfd3c0 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 10 client.4851 handle_command_reply: tid=0 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 client.4851 shutdown 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 msgr2=0x55aea3d03b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d03b70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01ca10 tx=0x7fe4fc000bd0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 shutdown_connections 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfd3c0 0x55aea3cfd7a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfc3c0 0x55aea3cfc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d03b70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 --2- 172.21.15.43:0/2389199824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d03630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe4f37fe640 0 client.4851 ms_handle_reset on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 >> 172.21.15.43:0/2389199824 conn(0x55aea3b95b80 msgr2=0x55aea3caa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 shutdown_connections 2024-03-30T13:11:32.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 1 -- 172.21.15.43:0/2389199824 wait complete. 2024-03-30T13:11:32.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.187+0000 7fe51dc42000 20 client.4851 trim_cache size 0 max 16384 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 Processor -- start 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- start start 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3b95b80 0x55aea3d13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d12830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d12d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe5140b5ca0 con 0x55aea3cfbaa0 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe5141401a0 con 0x55aea3caf4c0 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aea3a9a990 con 0x55aea3b95b80 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d12d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d12d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32922/0 (socket says 172.21.15.43:32922) 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 -- 172.21.15.43:0/3331779560 learned_addr learned my addr 172.21.15.43:0/3331779560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 -- 172.21.15.43:0/3331779560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3b95b80 msgr2=0x55aea3d13990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3b95b80 0x55aea3d13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe513fff640 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d12830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 -- 172.21.15.43:0/3331779560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 msgr2=0x55aea3d12830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d12830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 -- 172.21.15.43:0/3331779560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aea3ab8490 con 0x55aea3cfbaa0 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51905d640 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d12d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe504005a30 tx=0x7fe504005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684e357e5233 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe509ffb640 1 -- 172.21.15.43:0/3331779560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50401b070 con 0x55aea3cfbaa0 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe509ffb640 1 -- 172.21.15.43:0/3331779560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5040047f0 con 0x55aea3cfbaa0 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe509ffb640 1 -- 172.21.15.43:0/3331779560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504004bc0 con 0x55aea3cfbaa0 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- 172.21.15.43:0/3331779560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 msgr2=0x55aea3d12d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d12d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe504005a30 tx=0x7fe504005ef0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- 172.21.15.43:0/3331779560 shutdown_connections 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3cfbaa0 0x55aea3d12d70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3d12830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 --2- 172.21.15.43:0/3331779560 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3b95b80 0x55aea3d13990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- 172.21.15.43:0/3331779560 >> 172.21.15.43:0/3331779560 conn(0x55aea3d01d60 msgr2=0x55aea3ca92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- 172.21.15.43:0/3331779560 shutdown_connections 2024-03-30T13:11:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- 172.21.15.43:0/3331779560 wait complete. 2024-03-30T13:11:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 -- start start 2024-03-30T13:11:32.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.207+0000 7fe51dc42000 1 Processor -- start 2024-03-30T13:11:32.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3b95b80 0x55aea3cb27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3cb2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3d6b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aea3afd950 con 0x55aea3b95b80 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aea3b01f80 con 0x55aea3caf4c0 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aea3a957b0 con 0x55aea3cfbaa0 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 10 client.0 _ll_register_callbacks cb 0x7fe4fc0f0390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3b95b80 0x55aea3cb27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3b95b80 0x55aea3cb27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32932/0 (socket says 172.21.15.43:32932) 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 -- 172.21.15.43:0/1395456521 learned_addr learned my addr 172.21.15.43:0/1395456521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe513fff640 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3cb2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51905d640 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3d6b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 -- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 msgr2=0x55aea3d6b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3d6b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 -- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 msgr2=0x55aea3cb2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3cb2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 -- 172.21.15.43:0/1395456521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aea3ab6e80 con 0x55aea3b95b80 2024-03-30T13:11:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51885c640 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3b95b80 0x55aea3cb27c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0158b0 tx=0x7fe50c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1a41e85b535fe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:11:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 1 -- 172.21.15.43:0/1395456521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50c029070 con 0x55aea3b95b80 2024-03-30T13:11:32.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 1 -- 172.21.15.43:0/1395456521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe50c0163f0 con 0x55aea3b95b80 2024-03-30T13:11:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5040047f0 con 0x55aea3b95b80 2024-03-30T13:11:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 1 -- 172.21.15.43:0/1395456521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fe50c034050 con 0x55aea3b95b80 2024-03-30T13:11:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 10 client.4857 fetch_fsmap learned FSMap version 9 2024-03-30T13:11:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55aea3aa4270 con 0x55aea3b95b80 2024-03-30T13:11:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 1 -- 172.21.15.43:0/1395456521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5040047f0 con 0x55aea3b95b80 2024-03-30T13:11:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 1 -- 172.21.15.43:0/1395456521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7fe50c037020 con 0x55aea3b95b80 2024-03-30T13:11:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 10 client.4857 fetch_fsmap finished waiting for FSMap version 9 2024-03-30T13:11:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 10 client.4857 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-30T13:11:32.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 20 client.4857 populate_metadata read hostname 'smithi043' 2024-03-30T13:11:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfc3c0 0x55aea3d6b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 4 client.4857 mds_command: new command op to 4239 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T13:11:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 --> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] -- command(tid 0: {"prefix": "status"}) -- 0x55aea3b0ae90 con 0x55aea3cfc3c0 2024-03-30T13:11:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe513fff640 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfc3c0 0x55aea3d6b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe513fff640 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfc3c0 0x55aea3d6b770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d9abe4ebe3ee49 server_cookie=802817130f67b37e in_seq=0 out_seq=0 2024-03-30T13:11:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 10 client.4857 ms_handle_connect on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 1 -- 172.21.15.43:0/1395456521 <== mds.0 v2:172.21.15.43:6834/855012184 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55aea3b0ae90 con 0x55aea3cfc3c0 2024-03-30T13:11:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 10 client.4857 handle_command_reply: tid=0 2024-03-30T13:11:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 client.4857 shutdown 2024-03-30T13:11:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3b95b80 msgr2=0x55aea3cb27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3b95b80 0x55aea3cb27c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe50c0158b0 tx=0x7fe50c009810 comp rx=0 tx=0).stop 2024-03-30T13:11:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 shutdown_connections 2024-03-30T13:11:32.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:6834/855012184,v1:172.21.15.43:6835/855012184] conn(0x55aea3cfc3c0 0x55aea3d6b770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aea3cfbaa0 0x55aea3d6b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aea3caf4c0 0x55aea3cb2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 --2- 172.21.15.43:0/1395456521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aea3b95b80 0x55aea3cb27c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe4f37fe640 0 client.4857 ms_handle_reset on v2:172.21.15.43:6834/855012184 2024-03-30T13:11:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 >> 172.21.15.43:0/1395456521 conn(0x55aea3d01d60 msgr2=0x55aea3cdbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 shutdown_connections 2024-03-30T13:11:32.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 1 -- 172.21.15.43:0/1395456521 wait complete. 2024-03-30T13:11:32.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51dc42000 20 client.4857 trim_cache size 0 max 16384 2024-03-30T13:11:32.232 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:11:32.232 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "ace86ea7-ebbf-4c39-b9cf-d22b6fea8142", 2024-03-30T13:11:32.232 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4239, 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 36.267880208999998, 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 9, 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 27, 2024-03-30T13:11:32.233 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 27, 2024-03-30T13:11:32.234 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 43.108549707000002 2024-03-30T13:11:32.234 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:11:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe4f4067fc0 msgr2=0x7fe4f406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe4f4067fc0 0x7fe4f406a480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009d80 tx=0x7fe50c00b040 comp rx=0 tx=0).stop 2024-03-30T13:11:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51413f0a0 msgr2=0x7fe5141577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51413f0a0 0x7fe5141577a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc01a0d0 tx=0x7fe4fc019240 comp rx=0 tx=0).stop 2024-03-30T13:11:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 shutdown_connections 2024-03-30T13:11:32.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe4f4067fc0 0x7fe4f406a480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe5141609b0 0x7fe51415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe51413f0a0 0x7fe5141577a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 --2- 172.21.15.43:0/2791050319 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe51406dcc0 0x7fe514158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 >> 172.21.15.43:0/2791050319 conn(0x7fe514072500 msgr2=0x7fe51406c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 shutdown_connections 2024-03-30T13:11:32.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.211+0000 7fe51aac0640 1 -- 172.21.15.43:0/2791050319 wait complete. 2024-03-30T13:11:32.238 INFO:teuthology.run_tasks:Running task sequential... 2024-03-30T13:11:32.250 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-30T13:11:32.250 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-30T13:11:32.251 INFO:teuthology.task.print:Enabling mgr modules 2024-03-30T13:11:32.251 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-30T13:11:32.276 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-30T13:11:32.277 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-30T13:11:32.277 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-30T13:11:32.277 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:11:32.277 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:32.277 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-30T13:11:32.277 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:11:32.277 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:32.277 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-30T13:11:32.277 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:11:32.277 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:32.277 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-30T13:11:32.278 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-30T13:11:32.278 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:32.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:32.278 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-30T13:11:32.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:32.285 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-03-30T13:11:32.346 INFO:teuthology.orchestra.run.smithi113.stderr:Cannot find device "ceph-brx" 2024-03-30T13:11:32.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:11:32.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:32.347 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-30T13:11:32.354 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-03-30T13:11:32.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:32.354 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-03-30T13:11:32.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:32.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-03-30T13:11:32.602 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-30T13:11:32.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:11:32.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.707+0000 7f9776334640 1 Processor -- start 2024-03-30T13:11:32.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f9776334640 1 -- start start 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f9776334640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9770145e10 0x7f97700cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f9776334640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 0x7f97700b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f9776334640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97700d06c0 con 0x7f97700d0220 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f9776334640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f97700d09a0 con 0x7f9770145e10 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f9776334640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f97700d0c80 con 0x7f9770160470 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f97700d0220 0x7f977015fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32986/0 (socket says 172.21.15.43:32986) 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 -- 172.21.15.43:0/950019302 learned_addr learned my addr 172.21.15.43:0/950019302 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:32.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976ffff640 1 --2- 172.21.15.43:0/950019302 >> v2:172.21.15.113:3300/0 conn(0x7f9770145e10 0x7f97700cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97748d1640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 0x7f97700b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2536410641 0 0) 0x7f97700d06c0 con 0x7f97700d0220 2024-03-30T13:11:32.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f975c003530 con 0x7f97700d0220 2024-03-30T13:11:32.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3343050200 0 0) 0x7f975c003530 con 0x7f97700d0220 2024-03-30T13:11:32.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 msgr2=0x7f97700b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 0x7f97700b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 >> v2:172.21.15.113:3300/0 conn(0x7f9770145e10 msgr2=0x7f97700cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 --2- 172.21.15.43:0/950019302 >> v2:172.21.15.113:3300/0 conn(0x7f9770145e10 0x7f97700cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97700d0f60 con 0x7f97700d0220 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f9758001680 con 0x7f97700d0220 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 >> v1:172.21.15.43:6789/0 conn(0x7f97700d0220 legacy=0x7f977015fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c0039c0 0x7f975c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f975c004810 0x7f975c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 0x7f975c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97700d09a0 con 0x7f975c004810 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97700d0c80 con 0x7f975c0039c0 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f975c00b9d0 con 0x7f9770160470 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f975c004810 0x7f975c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97748d1640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 0x7f975c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976ffff640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c0039c0 0x7f975c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 -- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 msgr2=0x7f975c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 0x7f975c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 -- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c0039c0 msgr2=0x7f975c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:32.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c0039c0 0x7f975c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97677fe640 1 -- 172.21.15.43:0/950019302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c003530 con 0x7f975c004810 2024-03-30T13:11:32.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f976ffff640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c0039c0 0x7f975c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:32.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.711+0000 7f97748d1640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770160470 0x7f975c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97677fe640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f975c004810 0x7f975c004c90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f975800c210 tx=0x7f975801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae6a5bfed7dd8ce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758007d60 con 0x7f975c004810 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9758005ae0 con 0x7f975c004810 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f976f7fe640 1 -- 172.21.15.43:0/950019302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758007070 con 0x7f975c004810 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- 172.21.15.43:0/950019302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f975c004810 msgr2=0x7f975c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 --2- 172.21.15.43:0/950019302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f975c004810 0x7f975c004c90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f975800c210 tx=0x7f975801f3f0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 -- 172.21.15.43:0/950019302 reap_dead start 2024-03-30T13:11:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- 172.21.15.43:0/950019302 shutdown_connections 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- 172.21.15.43:0/950019302 >> 172.21.15.43:0/950019302 conn(0x7f97700ceb90 msgr2=0x7f97700cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- 172.21.15.43:0/950019302 shutdown_connections 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- 172.21.15.43:0/950019302 wait complete. 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 Processor -- start 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- start start 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c004810 0x7f97700cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770154ca0 0x7f9770155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770155660 0x7f9770156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9770160c10 con 0x7f9770155660 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9770160a90 con 0x7f975c004810 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f9776334640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f97700b1810 con 0x7f9770154ca0 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770155660 0x7f9770156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770155660 0x7f9770156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32954/0 (socket says 172.21.15.43:32954) 2024-03-30T13:11:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 -- 172.21.15.43:0/431077417 learned_addr learned my addr 172.21.15.43:0/431077417 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f976ffff640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c004810 0x7f97700cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97677fe640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770154ca0 0x7f9770155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 -- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770154ca0 msgr2=0x7f9770155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770154ca0 0x7f9770155120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 -- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c004810 msgr2=0x7f97700cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c004810 0x7f97700cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f97748d1640 1 -- 172.21.15.43:0/431077417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9770144c10 con 0x7f9770155660 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.715+0000 7f976ffff640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c004810 0x7f97700cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f97748d1640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770155660 0x7f9770156c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9770068e90 tx=0x7f976800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976d7fa640 1 -- 172.21.15.43:0/431077417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976801c070 con 0x7f9770155660 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976d7fa640 1 -- 172.21.15.43:0/431077417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9768017400 con 0x7f9770155660 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9770155ae0 con 0x7f9770155660 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976d7fa640 1 -- 172.21.15.43:0/431077417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680177d0 con 0x7f9770155660 2024-03-30T13:11:32.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9770159c80 con 0x7f9770155660 2024-03-30T13:11:32.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9770068890 con 0x7f9770155660 2024-03-30T13:11:32.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976d7fa640 1 -- 172.21.15.43:0/431077417 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f976801f070 con 0x7f9770155660 2024-03-30T13:11:32.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976d7fa640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9750067fc0 0x7f975006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:32.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976d7fa640 1 -- 172.21.15.43:0/431077417 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9768086910 con 0x7f9770155660 2024-03-30T13:11:32.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976ffff640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9750067fc0 0x7f975006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:32.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.719+0000 7f976ffff640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9750067fc0 0x7f975006a480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9760009ac0 tx=0x7f9760008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:32.723+0000 7f976d7fa640 1 -- 172.21.15.43:0/431077417 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9768012070 con 0x7f9770155660 2024-03-30T13:11:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f97700b5940 con 0x7f9770155660 2024-03-30T13:11:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f976d7fa640 1 -- 172.21.15.43:0/431077417 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f9768056d80 con 0x7f9770155660 2024-03-30T13:11:33.065 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9750067fc0 msgr2=0x7f975006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9750067fc0 0x7f975006a480 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9760009ac0 tx=0x7f9760008040 comp rx=0 tx=0).stop 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770155660 msgr2=0x7f9770156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770155660 0x7f9770156c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9770068e90 tx=0x7f976800f4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 shutdown_connections 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9750067fc0 0x7f975006a480 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9770155660 0x7f9770156c80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9770154ca0 0x7f9770155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 --2- 172.21.15.43:0/431077417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f975c004810 0x7f97700cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 >> 172.21.15.43:0/431077417 conn(0x7f97700ceb90 msgr2=0x7f9770144500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 shutdown_connections 2024-03-30T13:11:33.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33.059+0000 7f9776334640 1 -- 172.21.15.43:0/431077417 wait complete. 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:11:33.083 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:11:33.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:33.084 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-03-30T13:11:33.090 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:33.091 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-03-30T13:11:33.092 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-03-30T13:11:33.092 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:11:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-03-30T13:11:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-30T13:11:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42461sec preferred_lft 42461sec 2024-03-30T13:11:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-03-30T13:11:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:33.094 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-03-30T13:11:33.094 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T13:11:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:bf:2f:c7:16 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T13:11:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:33.095 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T13:11:33.095 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:33.096 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-03-30T13:11:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-03-30T13:11:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-03-30T13:11:33.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T13:11:33.096 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:33.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:33.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:33.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:33.205 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:11:33.256 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-03-30T13:11:33.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:33.257 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-03-30T13:11:33.305 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-03-30T13:11:33.305 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:11:33.306 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-30T13:11:33.306 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T13:11:33.306 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-30T13:11:33.306 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:11:33.306 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:33.306 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:33.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T13:11:33.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T13:11:33.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T13:11:33.307 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:33.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:33.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:33.424 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-30T13:11:33.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:33.473 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-03-30T13:11:33.521 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:33.521 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:33.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:33.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-30T13:11:33.521 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:33.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:33.636 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-30T13:11:33.637 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:33.637 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:33.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-30T13:11:33.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T13:11:33.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-30T13:11:33.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-30T13:11:33.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-30T13:11:33.638 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:33.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:33.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:33.728 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:33.728 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:33.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-03-30T13:11:33.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T13:11:33.729 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:33.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:33.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:33.830 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T13:11:33.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:33.830 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:33.881 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:11:33.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:33.881 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:33.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:33.984 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:34.037 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:34.038 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:34.039 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:34.039 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:34.039 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:34.039 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:34.040 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:34.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:34.041 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:34.089 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T13:11:34.089 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T13:11:34.134 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:34.145 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:34.194 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:34.195 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:34.196 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:34.196 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:34.196 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:34.196 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:34.196 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:34.196 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:34.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:34.197 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:34.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-03-30T13:11:34.195+0000 7f6f83f54580 -1 init, newargv = 0x5600e8790990 newargc=15 2024-03-30T13:11:34.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[35401]: starting ceph client 2024-03-30T13:11:34.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[35401]: starting fuse 2024-03-30T13:11:35.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:35.324 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:35.374 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:35.374 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:35.376 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:35.377 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:35.377 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:35.377 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:35.377 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:35.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:35.379 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:35.426 INFO:teuthology.orchestra.run.smithi043.stdout:45 2024-03-30T13:11:35.427 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:35.427 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-03-30T13:11:35.428 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-03-30T13:11:35.552 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.35401.asok 2024-03-30T13:11:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:35.564 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35401.asok 2024-03-30T13:11:35.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:35.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35401.asok status 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "35401", 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-03-30T13:11:35.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4869, 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4869 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2943270380 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:35.721 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2943270380 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4869 192.168.0.1:0/2943270380", 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2943270380", 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 9, 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 27, 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-03-30T13:11:35.722 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:11:35.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:11:35.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83f574640 1 Processor -- start 2024-03-30T13:11:35.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83f574640 1 -- start start 2024-03-30T13:11:35.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83f574640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff838147e10 0x7ff838071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83f574640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff838072070 con 0x7ff838162b90 2024-03-30T13:11:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83f574640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83806dc30 con 0x7ff83806c670 2024-03-30T13:11:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83f574640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff83806e280 con 0x7ff838147e10 2024-03-30T13:11:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83cb0f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff838162b90 0x7ff83806e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32996/0 (socket says 172.21.15.43:32996) 2024-03-30T13:11:35.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83cb0f640 1 -- 172.21.15.43:0/686828269 learned_addr learned my addr 172.21.15.43:0/686828269 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:35.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff83d310640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff838147e10 0x7ff838071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3630723145 0 0) 0x7ff838072070 con 0x7ff838162b90 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff824003530 con 0x7ff838162b90 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2791434059 0 0) 0x7ff824003530 con 0x7ff838162b90 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff838147e10 msgr2=0x7ff838071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff838147e10 0x7ff838071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 >> v1:172.21.15.113:6789/0 conn(0x7ff83806c670 legacy=0x7ff8380b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83806df10 con 0x7ff838162b90 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ff82c001680 con 0x7ff838162b90 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 >> v1:172.21.15.43:6789/0 conn(0x7ff838162b90 legacy=0x7ff83806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8240039f0 0x7ff824003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004810 0x7ff824004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.851+0000 7ff82bfff640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff838147e10 0x7ff824005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff83806e280 con 0x7ff8240039f0 2024-03-30T13:11:35.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff82400bb00 con 0x7ff824004810 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff82400bd00 con 0x7ff838147e10 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83d310640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8240039f0 0x7ff824003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83cb0f640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004810 0x7ff824004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83d310640 1 -- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff838147e10 msgr2=0x7ff824005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83d310640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff838147e10 0x7ff824005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83d310640 1 -- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004810 msgr2=0x7ff824004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83d310640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004810 0x7ff824004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83d310640 1 -- 172.21.15.43:0/686828269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff824003530 con 0x7ff8240039f0 2024-03-30T13:11:35.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83cb0f640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff824004810 0x7ff824004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:35.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83d310640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8240039f0 0x7ff824003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff834009ae0 tx=0x7ff83401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46725c6a625a437f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:35.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8340096b0 con 0x7ff8240039f0 2024-03-30T13:11:35.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8340a6510 con 0x7ff8240039f0 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff82bfff640 1 -- 172.21.15.43:0/686828269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8340a6900 con 0x7ff8240039f0 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83f574640 1 -- 172.21.15.43:0/686828269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8240039f0 msgr2=0x7ff824003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83f574640 1 --2- 172.21.15.43:0/686828269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8240039f0 0x7ff824003e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff834009ae0 tx=0x7ff83401cab0 comp rx=0 tx=0).stop 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83db11640 1 -- 172.21.15.43:0/686828269 reap_dead start 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83f574640 1 -- 172.21.15.43:0/686828269 shutdown_connections 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83f574640 1 -- 172.21.15.43:0/686828269 >> 172.21.15.43:0/686828269 conn(0x7ff8380cfee0 msgr2=0x7ff8380d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83f574640 1 -- 172.21.15.43:0/686828269 shutdown_connections 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.855+0000 7ff83f574640 1 -- 172.21.15.43:0/686828269 wait complete. 2024-03-30T13:11:35.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 Processor -- start 2024-03-30T13:11:35.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 -- start start 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8240039f0 0x7ff8381f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8381fa350 0x7ff838142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8381fa7b0 0x7ff8381457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff83816a250 con 0x7ff8381fa7b0 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff83816a0d0 con 0x7ff8240039f0 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff83816a3d0 con 0x7ff8381fa350 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8381fa7b0 0x7ff8381457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8381fa7b0 0x7ff8381457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32974/0 (socket says 172.21.15.43:32974) 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 -- 172.21.15.43:0/2483182851 learned_addr learned my addr 172.21.15.43:0/2483182851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83d310640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8240039f0 0x7ff8381f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83cb0f640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8381fa350 0x7ff838142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:35.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 -- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8381fa350 msgr2=0x7ff838142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8381fa350 0x7ff838142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 -- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8240039f0 msgr2=0x7ff8381f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8240039f0 0x7ff8381f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 -- 172.21.15.43:0/2483182851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8381f7bd0 con 0x7ff8381fa7b0 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83cb0f640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8381fa350 0x7ff838142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83db11640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8381fa7b0 0x7ff8381457a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8380cfc90 tx=0x7ff83000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff829ffb640 1 -- 172.21.15.43:0/2483182851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830003980 con 0x7ff8381fa7b0 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff829ffb640 1 -- 172.21.15.43:0/2483182851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff830003b40 con 0x7ff8381fa7b0 2024-03-30T13:11:35.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff829ffb640 1 -- 172.21.15.43:0/2483182851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830004020 con 0x7ff8381fa7b0 2024-03-30T13:11:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff838143800 con 0x7ff8381fa7b0 2024-03-30T13:11:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.859+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff838143dc0 con 0x7ff8381fa7b0 2024-03-30T13:11:35.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.863+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff838068890 con 0x7ff8381fa7b0 2024-03-30T13:11:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.863+0000 7ff829ffb640 1 -- 172.21.15.43:0/2483182851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff8300041c0 con 0x7ff8381fa7b0 2024-03-30T13:11:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.863+0000 7ff829ffb640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff81c067fc0 0x7ff81c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.863+0000 7ff829ffb640 1 -- 172.21.15.43:0/2483182851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff830085d90 con 0x7ff8381fa7b0 2024-03-30T13:11:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.863+0000 7ff83d310640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff81c067fc0 0x7ff81c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:35.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.863+0000 7ff83d310640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff81c067fc0 0x7ff81c06a480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8340af000 tx=0x7ff8340046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:35.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:35.867+0000 7ff829ffb640 1 -- 172.21.15.43:0/2483182851 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff8300560d0 con 0x7ff8381fa7b0 2024-03-30T13:11:36.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff83813ea10 con 0x7ff8381fa7b0 2024-03-30T13:11:36.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff829ffb640 1 -- 172.21.15.43:0/2483182851 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7ff8300559d0 con 0x7ff8381fa7b0 2024-03-30T13:11:36.082 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:11:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff81c067fc0 msgr2=0x7ff81c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff81c067fc0 0x7ff81c06a480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff8340af000 tx=0x7ff8340046d0 comp rx=0 tx=0).stop 2024-03-30T13:11:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8381fa7b0 msgr2=0x7ff8381457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8381fa7b0 0x7ff8381457a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8380cfc90 tx=0x7ff83000c960 comp rx=0 tx=0).stop 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 shutdown_connections 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff81c067fc0 0x7ff81c06a480 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8381fa7b0 0x7ff8381457a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff8381fa350 0x7ff838142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 --2- 172.21.15.43:0/2483182851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff8240039f0 0x7ff8381f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 >> 172.21.15.43:0/2483182851 conn(0x7ff8380cfee0 msgr2=0x7ff83813ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 shutdown_connections 2024-03-30T13:11:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36.075+0000 7ff83f574640 1 -- 172.21.15.43:0/2483182851 wait complete. 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:11:36.102 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-30T13:11:36.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:36.103 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-03-30T13:11:36.108 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:36.108 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:36.109 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-03-30T13:11:36.110 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42458sec preferred_lft 42458sec 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-03-30T13:11:36.111 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:bf:2f:c7:16 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.112 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T13:11:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.113 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:11:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T13:11:36.113 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:36.113 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:11:36.162 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-03-30T13:11:36.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:36.164 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-03-30T13:11:36.213 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-03-30T13:11:36.213 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:11:36.214 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-30T13:11:36.214 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T13:11:36.214 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-30T13:11:36.214 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:11:36.214 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:36.214 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:36.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T13:11:36.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T13:11:36.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T13:11:36.214 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:36.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:36.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:36.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:36.345 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-30T13:11:36.351 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T13:11:36.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:36.352 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-03-30T13:11:36.397 INFO:teuthology.orchestra.run.smithi043.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-30T13:11:36.398 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:36.398 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:36.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:36.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-30T13:11:36.398 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:36.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:36.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:36.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-30T13:11:36.567 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:36.568 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.569 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 4a:70:25:50:3d:6d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:11:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:11:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::4870:25ff:fe50:3d6d/64 scope link 2024-03-30T13:11:36.569 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:36.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 4a:70:25:50:3d:6d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:11:36.618 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::4870:25ff:fe50:3d6d/64 scope link 2024-03-30T13:11:36.619 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:36.620 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-30T13:11:36.620 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:36.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:36.758 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:36.758 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:11:36.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.1 up 2024-03-30T13:11:36.758 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.1 master ceph-brx 2024-03-30T13:11:36.758 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:11:36.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:36.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:36.819 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-30T13:11:36.820 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:36.820 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:36.864 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-30T13:11:36.865 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:36.865 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:36.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:36.963 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:37.014 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:37.015 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:37.016 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:37.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:37.017 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:37.061 INFO:teuthology.orchestra.run.smithi043.stdout:45 2024-03-30T13:11:37.061 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-03-30T13:11:37.061 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-30T13:11:37.106 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:37.119 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:37.166 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:37.167 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:37.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:37.168 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:37.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:2024-03-30T13:11:37.175+0000 7fd9abdbe580 -1 init, newargv = 0x5649d7573670 newargc=15 2024-03-30T13:11:37.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[35546]: starting ceph client 2024-03-30T13:11:37.186 INFO:teuthology.orchestra.run.smithi043.stdout:45 2024-03-30T13:11:37.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[35546]: starting fuse 2024-03-30T13:11:38.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:11:38.303 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:38.350 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:11:38.350 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:11:38.350 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:11:38.350 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:11:38.350 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:11:38.351 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:38.352 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:38.353 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:11:38.353 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:38.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:38.354 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:11:38.401 INFO:teuthology.orchestra.run.smithi043.stdout:45 2024-03-30T13:11:38.401 INFO:teuthology.orchestra.run.smithi043.stdout:46 2024-03-30T13:11:38.401 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-03-30T13:11:38.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:11:38.401 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-03-30T13:11:38.401 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-03-30T13:11:38.401 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-03-30T13:11:38.401 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-03-30T13:11:38.401 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-03-30T13:11:38.402 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-03-30T13:11:38.403 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-03-30T13:11:38.403 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:11:38.403 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:11:38.403 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.2")) 2024-03-30T13:11:38.403 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-03-30T13:11:38.527 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.2.35546.asok 2024-03-30T13:11:38.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:38.539 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35546.asok 2024-03-30T13:11:38.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:38.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35546.asok status 2024-03-30T13:11:38.694 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:11:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-03-30T13:11:38.694 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "2", 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "35546", 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4881, 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4881 2024-03-30T13:11:38.695 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1006211374 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1006211374 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4881 192.168.0.2:0/1006211374", 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.2:0/1006211374", 2024-03-30T13:11:38.696 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-03-30T13:11:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 9, 2024-03-30T13:11:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 29, 2024-03-30T13:11:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:11:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-03-30T13:11:38.697 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-03-30T13:11:38.697 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:11:38.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:11:38.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f049576f640 1 Processor -- start 2024-03-30T13:11:38.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f049576f640 1 -- start start 2024-03-30T13:11:38.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f049576f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04901628b0 0x7f0490166410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:38.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f049576f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f049006e160 0x7f0490166950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f049576f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049006cfa0 con 0x7f0490162c90 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f049576f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f049006f770 con 0x7f049006e160 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f049576f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f049006f1b0 con 0x7f04901628b0 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f048f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0490162c90 0x7f04900b3680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33006/0 (socket says 172.21.15.43:33006) 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f048f7fe640 1 -- 172.21.15.43:0/792314197 learned_addr learned my addr 172.21.15.43:0/792314197 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f04867fc640 1 --2- 172.21.15.43:0/792314197 >> v2:172.21.15.43:3300/0 conn(0x7f049006e160 0x7f0490166950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f048effd640 1 --2- 172.21.15.43:0/792314197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04901628b0 0x7f0490166410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f048e7fc640 1 -- 172.21.15.43:0/792314197 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 982361539 0 0) 0x7f049006cfa0 con 0x7f0490162c90 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f048e7fc640 1 -- 172.21.15.43:0/792314197 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0470003530 con 0x7f0490162c90 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f04867fc640 1 -- 172.21.15.43:0/792314197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04901628b0 msgr2=0x7f0490166410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f04867fc640 1 --2- 172.21.15.43:0/792314197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04901628b0 0x7f0490166410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f04867fc640 1 -- 172.21.15.43:0/792314197 >> v1:172.21.15.43:6789/0 conn(0x7f0490162c90 legacy=0x7f04900b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:38.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.823+0000 7f04867fc640 1 -- 172.21.15.43:0/792314197 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049006f490 con 0x7f049006e160 2024-03-30T13:11:38.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f04867fc640 1 --2- 172.21.15.43:0/792314197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490166950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04800097c0 tx=0x7f048001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e51847dda9a60c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:38.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f048e7fc640 1 -- 172.21.15.43:0/792314197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800b0070 con 0x7f049006e160 2024-03-30T13:11:38.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f048e7fc640 1 -- 172.21.15.43:0/792314197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04800ab600 con 0x7f049006e160 2024-03-30T13:11:38.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f048e7fc640 1 -- 172.21.15.43:0/792314197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04800ab9d0 con 0x7f049006e160 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 -- 172.21.15.43:0/792314197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 msgr2=0x7f0490166950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 --2- 172.21.15.43:0/792314197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490166950 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04800097c0 tx=0x7f048001c350 comp rx=0 tx=0).stop 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 -- 172.21.15.43:0/792314197 shutdown_connections 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 --2- 172.21.15.43:0/792314197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490166950 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 --2- 172.21.15.43:0/792314197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04901628b0 0x7f0490166410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 -- 172.21.15.43:0/792314197 >> 172.21.15.43:0/792314197 conn(0x7f04900b2980 msgr2=0x7f04900d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 -- 172.21.15.43:0/792314197 shutdown_connections 2024-03-30T13:11:38.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 -- 172.21.15.43:0/792314197 wait complete. 2024-03-30T13:11:38.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 Processor -- start 2024-03-30T13:11:38.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.827+0000 7f049576f640 1 -- start start 2024-03-30T13:11:38.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:38.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04901628b0 0x7f0490160d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0490162c90 0x7f04901612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f049016f930 con 0x7f049006e160 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f049016f7b0 con 0x7f04901628b0 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f049016fab0 con 0x7f0490162c90 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490154b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32986/0 (socket says 172.21.15.43:32986) 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 -- 172.21.15.43:0/3275792003 learned_addr learned my addr 172.21.15.43:0/3275792003 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f04867fc640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04901628b0 0x7f0490160d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048f7fe640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0490162c90 0x7f04901612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 -- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0490162c90 msgr2=0x7f04901612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:38.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0490162c90 0x7f04901612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 -- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04901628b0 msgr2=0x7f0490160d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04901628b0 0x7f0490160d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 -- 172.21.15.43:0/3275792003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0490156ae0 con 0x7f049006e160 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f04867fc640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04901628b0 0x7f0490160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f048effd640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490154b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f047800d620 tx=0x7f047800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f0487fff640 1 -- 172.21.15.43:0/3275792003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047800b620 con 0x7f049006e160 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f0487fff640 1 -- 172.21.15.43:0/3275792003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0478019070 con 0x7f049006e160 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0490155e30 con 0x7f049006e160 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f0487fff640 1 -- 172.21.15.43:0/3275792003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047800b9e0 con 0x7f049006e160 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04901563f0 con 0x7f049006e160 2024-03-30T13:11:38.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0490068890 con 0x7f049006e160 2024-03-30T13:11:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f0487fff640 1 -- 172.21.15.43:0/3275792003 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f047800bb80 con 0x7f049006e160 2024-03-30T13:11:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f0487fff640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0474067fc0 0x7f047406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f0487fff640 1 -- 172.21.15.43:0/3275792003 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f04780876b0 con 0x7f049006e160 2024-03-30T13:11:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.831+0000 7f04867fc640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0474067fc0 0x7f047406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:38.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.835+0000 7f04867fc640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0474067fc0 0x7f047406a480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f04800ad000 tx=0x7f0480008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:38.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:38.835+0000 7f0487fff640 1 -- 172.21.15.43:0/3275792003 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0490156ae0 con 0x7f049006e160 2024-03-30T13:11:39.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f04900d1340 con 0x7f049006e160 2024-03-30T13:11:39.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f0487fff640 1 -- 172.21.15.43:0/3275792003 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f0478057c60 con 0x7f049006e160 2024-03-30T13:11:39.042 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:11:39.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0474067fc0 msgr2=0x7f047406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:39.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0474067fc0 0x7f047406a480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f04800ad000 tx=0x7f0480008040 comp rx=0 tx=0).stop 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 msgr2=0x7f0490154b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490154b40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f047800d620 tx=0x7f047800dae0 comp rx=0 tx=0).stop 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 shutdown_connections 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0474067fc0 0x7f047406a480 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0490162c90 0x7f04901612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04901628b0 0x7f0490160d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 --2- 172.21.15.43:0/3275792003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f049006e160 0x7f0490154b40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.035+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 >> 172.21.15.43:0/3275792003 conn(0x7f04900b2980 msgr2=0x7f04900b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.039+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 shutdown_connections 2024-03-30T13:11:39.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:39.039+0000 7f049576f640 1 -- 172.21.15.43:0/3275792003 wait complete. 2024-03-30T13:11:39.062 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:11:39.062 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:11:39.062 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-03-30T13:11:39.062 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T13:11:39.063 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:39.063 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:11:39.063 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:11:39.063 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:11:39.063 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:11:39.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:39.063 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-03-30T13:11:39.070 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:39.070 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:39.070 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:39.070 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:39.070 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:39.070 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:11:39.071 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42469sec preferred_lft 42469sec 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 02:42:46:0d:39:59 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T13:11:39.072 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:39.072 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-30T13:11:39.072 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:39.072 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:39.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add name ceph-brx type bridge 2024-03-30T13:11:39.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr flush dev ceph-brx 2024-03-30T13:11:39.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set ceph-brx up 2024-03-30T13:11:39.073 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-30T13:11:39.073 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:39.161 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:39.196 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:39.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:39.204 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:11:39.213 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-03-30T13:11:39.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:39.215 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:11:39.274 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:11:39.275 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:39.275 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:39.275 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:11:39.275 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:11:39.275 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:11:39.275 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:39.359 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:39.386 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:39.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:39.393 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-30T13:11:39.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:39.442 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-03-30T13:11:39.493 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:39.493 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:39.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:39.494 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-30T13:11:39.494 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:39.579 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:39.596 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:39.603 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-30T13:11:39.604 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:39.687 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:39.741 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:39.748 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:39.748 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:39.749 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-03-30T13:11:39.749 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.0 master ceph-brx 2024-03-30T13:11:39.749 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:39.831 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:39.849 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:39.856 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-30T13:11:39.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:39.857 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:39.904 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:11:39.905 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:39.905 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:39.952 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:11:40.010 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:11:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:11:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:11:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:11:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:11:40.059 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:40.060 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:11:40.061 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:11:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:11:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:11:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:11:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:11:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:11:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:40.062 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:40.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:40.063 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:11:40.105 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T13:11:40.105 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-30T13:11:40.150 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:11:40.165 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:11:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:11:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:11:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:11:40.213 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:11:40.214 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:40.215 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:40.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:40.216 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:11:40.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:2024-03-30T13:11:40.223+0000 7fb12773b580 -1 init, newargv = 0x562fff4a7710 newargc=15 2024-03-30T13:11:40.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[33019]: starting ceph client 2024-03-30T13:11:40.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[33019]: starting fuse 2024-03-30T13:11:41.337 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:11:41.354 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:11:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:11:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:11:41.411 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:11:41.412 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:41.413 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:41.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:41.414 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:11:41.457 INFO:teuthology.orchestra.run.smithi113.stdout:45 2024-03-30T13:11:41.457 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-30T13:11:41.457 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> import glob 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> import re 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> import subprocess 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> def _find_admin_socket(client_name): 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> files = glob.glob(asok_path) 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> # Given a non-glob path, it better be there 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> if "*" not in asok_path: 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> assert(len(files) == 1) 2024-03-30T13:11:41.458 DEBUG:teuthology.orchestra.run.smithi113:> return files[0] 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> for f in files: 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> contents = proc_f.read() 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> if mountpoint in contents: 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> return f 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> print(_find_admin_socket("client.1")) 2024-03-30T13:11:41.459 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-03-30T13:11:41.582 INFO:teuthology.orchestra.run.smithi113.stdout:/var/run/ceph/ceph-client.1.33019.asok 2024-03-30T13:11:41.586 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:41.594 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33019.asok 2024-03-30T13:11:41.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:41.594 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33019.asok status 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "metadata": { 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "entity_id": "1", 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": "smithi113", 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "pid": "33019", 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "root": "/" 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_count": 0, 2024-03-30T13:11:41.748 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_pinned_count": 0, 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4699, 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "inst": { 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "name": { 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "client", 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "num": 4699 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 278497937 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2024-03-30T13:11:41.749 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 278497937 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "inst_str": "client.4699 192.168.0.1:0/278497937", 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "addr_str": "192.168.0.1:0/278497937", 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "inode_count": 1, 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_epoch": 9, 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch": 29, 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "blocklisted": false, 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs" 2024-03-30T13:11:41.750 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-03-30T13:11:41.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:11:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7163c86640 1 Processor -- start 2024-03-30T13:11:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7163c86640 1 -- start start 2024-03-30T13:11:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7163c86640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f715c06f390 0x7f715c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7163c86640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c0cfd20 0x7f715c0d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7163c86640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715c15c610 con 0x7f715c15c230 2024-03-30T13:11:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7163c86640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f715c0b4fa0 con 0x7f715c06f390 2024-03-30T13:11:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7163c86640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f715c0cf4f0 con 0x7f715c0cfd20 2024-03-30T13:11:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7161a22640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f715c15c230 0x7f715c15fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60528/0 (socket says 172.21.15.43:60528) 2024-03-30T13:11:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7161a22640 1 -- 172.21.15.43:0/3994246662 learned_addr learned my addr 172.21.15.43:0/3994246662 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7162223640 1 --2- 172.21.15.43:0/3994246662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c0cfd20 0x7f715c0d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7161221640 1 --2- 172.21.15.43:0/3994246662 >> v2:172.21.15.113:3300/0 conn(0x7f715c06f390 0x7f715c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:41.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7160a20640 1 -- 172.21.15.43:0/3994246662 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4064805148 0 0) 0x7f715c15c610 con 0x7f715c15c230 2024-03-30T13:11:41.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7160a20640 1 -- 172.21.15.43:0/3994246662 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7148003530 con 0x7f715c15c230 2024-03-30T13:11:41.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7162223640 1 -- 172.21.15.43:0/3994246662 >> v2:172.21.15.113:3300/0 conn(0x7f715c06f390 msgr2=0x7f715c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:41.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7162223640 1 --2- 172.21.15.43:0/3994246662 >> v2:172.21.15.113:3300/0 conn(0x7f715c06f390 0x7f715c1604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:41.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7162223640 1 -- 172.21.15.43:0/3994246662 >> v1:172.21.15.43:6789/0 conn(0x7f715c15c230 legacy=0x7f715c15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:41.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.887+0000 7f7162223640 1 -- 172.21.15.43:0/3994246662 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c0cc9f0 con 0x7f715c0cfd20 2024-03-30T13:11:41.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7162223640 1 --2- 172.21.15.43:0/3994246662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c0cfd20 0x7f715c0d0130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7154009d90 tx=0x7f715401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f3df41e5bdb495 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7160a20640 1 -- 172.21.15.43:0/3994246662 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540b2070 con 0x7f715c0cfd20 2024-03-30T13:11:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7160a20640 1 -- 172.21.15.43:0/3994246662 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71540a56b0 con 0x7f715c0cfd20 2024-03-30T13:11:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7160a20640 1 -- 172.21.15.43:0/3994246662 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540a5a60 con 0x7f715c0cfd20 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 -- 172.21.15.43:0/3994246662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c0cfd20 msgr2=0x7f715c0d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 --2- 172.21.15.43:0/3994246662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c0cfd20 0x7f715c0d0130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7154009d90 tx=0x7f715401c8e0 comp rx=0 tx=0).stop 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 -- 172.21.15.43:0/3994246662 shutdown_connections 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 --2- 172.21.15.43:0/3994246662 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c0cfd20 0x7f715c0d0130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 --2- 172.21.15.43:0/3994246662 >> v2:172.21.15.113:3300/0 conn(0x7f715c06f390 0x7f715c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 -- 172.21.15.43:0/3994246662 >> 172.21.15.43:0/3994246662 conn(0x7f715c0b3c80 msgr2=0x7f715c0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 -- 172.21.15.43:0/3994246662 shutdown_connections 2024-03-30T13:11:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.891+0000 7f7163c86640 1 -- 172.21.15.43:0/3994246662 wait complete. 2024-03-30T13:11:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 Processor -- start 2024-03-30T13:11:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 -- start start 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06f390 0x7f715c0c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f715c0cfd20 0x7f715c0c5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c15c230 0x7f715c0c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f715c0d1b20 con 0x7f715c0cfd20 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f715c0cc7a0 con 0x7f715c06f390 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f715c1609f0 con 0x7f715c15c230 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f715c0cfd20 0x7f715c0c5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f715c0cfd20 0x7f715c0c5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46178/0 (socket says 172.21.15.43:46178) 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 -- 172.21.15.43:0/1883903284 learned_addr learned my addr 172.21.15.43:0/1883903284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161a22640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06f390 0x7f715c0c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7162223640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c15c230 0x7f715c0c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 -- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c15c230 msgr2=0x7f715c0c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c15c230 0x7f715c0c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 -- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06f390 msgr2=0x7f715c0c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06f390 0x7f715c0c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 -- 172.21.15.43:0/1883903284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c15c860 con 0x7f715c0cfd20 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7161221640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f715c0cfd20 0x7f715c0c5c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e930 tx=0x7f714c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f71527fc640 1 -- 172.21.15.43:0/1883903284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c017070 con 0x7f715c0cfd20 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f71527fc640 1 -- 172.21.15.43:0/1883903284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714c00ca00 con 0x7f715c0cfd20 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c1f2d40 con 0x7f715c0cfd20 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f71527fc640 1 -- 172.21.15.43:0/1883903284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c00cdb0 con 0x7f715c0cfd20 2024-03-30T13:11:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c15d0c0 con 0x7f715c0cfd20 2024-03-30T13:11:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c068890 con 0x7f715c0cfd20 2024-03-30T13:11:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.895+0000 7f71527fc640 1 -- 172.21.15.43:0/1883903284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f714c01a070 con 0x7f715c0cfd20 2024-03-30T13:11:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.899+0000 7f71527fc640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7140067fc0 0x7f714006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.899+0000 7f71527fc640 1 -- 172.21.15.43:0/1883903284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f714c086930 con 0x7f715c0cfd20 2024-03-30T13:11:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.899+0000 7f7161a22640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7140067fc0 0x7f714006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.899+0000 7f7161a22640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7140067fc0 0x7f714006a480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7158005890 tx=0x7f7158009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:41.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:41.903+0000 7f71527fc640 1 -- 172.21.15.43:0/1883903284 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f714c056db0 con 0x7f715c0cfd20 2024-03-30T13:11:42.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.107+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f715c070230 con 0x7f715c0cfd20 2024-03-30T13:11:42.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.107+0000 7f71527fc640 1 -- 172.21.15.43:0/1883903284 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f714c0566b0 con 0x7f715c0cfd20 2024-03-30T13:11:42.115 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7140067fc0 msgr2=0x7f714006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7140067fc0 0x7f714006a480 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7158005890 tx=0x7f7158009040 comp rx=0 tx=0).stop 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f715c0cfd20 msgr2=0x7f715c0c5c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f715c0cfd20 0x7f715c0c5c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f714c00e930 tx=0x7f714c00edf0 comp rx=0 tx=0).stop 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 shutdown_connections 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7140067fc0 0x7f714006a480 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f715c15c230 0x7f715c0c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f715c0cfd20 0x7f715c0c5c80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 --2- 172.21.15.43:0/1883903284 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f715c06f390 0x7f715c0c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:42.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 >> 172.21.15.43:0/1883903284 conn(0x7f715c0b3c80 msgr2=0x7f715c15a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:42.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 shutdown_connections 2024-03-30T13:11:42.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:42.111+0000 7f7163c86640 1 -- 172.21.15.43:0/1883903284 wait complete. 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:11:42.135 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-30T13:11:42.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:42.135 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-03-30T13:11:42.142 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:42.142 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:42.143 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42466sec preferred_lft 42466sec 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 02:42:46:0d:39:59 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:11:42.144 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T13:11:42.145 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:42.145 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:11:42.198 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-03-30T13:11:42.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:42.200 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-03-30T13:11:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-03-30T13:11:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:11:42.253 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:11:42.254 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:11:42.254 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:42.255 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:42.255 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:11:42.255 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:11:42.255 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:11:42.255 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:42.342 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:42.369 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:42.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:42.377 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-30T13:11:42.382 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T13:11:42.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:42.383 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list-id 2024-03-30T13:11:42.428 INFO:teuthology.orchestra.run.smithi113.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-30T13:11:42.429 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:42.429 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:42.429 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:42.429 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-30T13:11:42.429 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:42.518 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:42.539 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:42.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:42.547 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:42.560 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether a6:5b:41:ce:ae:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:11:42.561 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:11:42.561 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.561 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a45b:41ff:fece:aeaf/64 scope link 2024-03-30T13:11:42.561 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:42.563 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether a6:5b:41:ce:ae:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-30T13:11:42.616 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-30T13:11:42.617 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.617 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::a45b:41ff:fece:aeaf/64 scope link 2024-03-30T13:11:42.617 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:11:42.617 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-30T13:11:42.617 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:42.617 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:42.617 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-30T13:11:42.617 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-30T13:11:42.617 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-30T13:11:42.618 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-30T13:11:42.618 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-30T13:11:42.618 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:42.706 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:42.750 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:42.757 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:11:42.757 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:11:42.757 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.1 up 2024-03-30T13:11:42.757 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set dev brx.1 master ceph-brx 2024-03-30T13:11:42.757 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:11:42.796 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:11:42.812 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:42.819 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-30T13:11:42.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:42.819 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:42.868 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-30T13:11:42.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:11:42.869 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:42.916 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:11:42.974 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:11:43.023 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:11:43.024 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:11:43.025 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:11:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:43.026 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:43.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:43.027 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:11:43.077 INFO:teuthology.orchestra.run.smithi113.stdout:45 2024-03-30T13:11:43.077 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-03-30T13:11:43.077 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-30T13:11:43.122 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:11:43.136 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:11:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:11:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:11:43.185 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:11:43.186 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:43.187 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:43.188 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:43.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:43.189 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:11:43.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:2024-03-30T13:11:43.199+0000 7f019595a580 -1 init, newargv = 0x559560dba690 newargc=15 2024-03-30T13:11:43.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:ceph-fuse[33138]: starting ceph client 2024-03-30T13:11:43.210 INFO:teuthology.orchestra.run.smithi113.stdout:45 2024-03-30T13:11:43.212 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi113.stderr:ceph-fuse[33138]: starting fuse 2024-03-30T13:11:44.312 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:11:44.327 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:11:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:11:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:11:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:11:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:11:44.374 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:11:44.375 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:11:44.376 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:44.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.377 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:11:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:45 2024-03-30T13:11:44.429 INFO:teuthology.orchestra.run.smithi113.stdout:46 2024-03-30T13:11:44.430 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> import glob 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> import re 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> import subprocess 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> def _find_admin_socket(client_name): 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> files = glob.glob(asok_path) 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:44.430 DEBUG:teuthology.orchestra.run.smithi113:> # Given a non-glob path, it better be there 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> if "*" not in asok_path: 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> assert(len(files) == 1) 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> return files[0] 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> for f in files: 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> contents = proc_f.read() 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> if mountpoint in contents: 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> return f 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> print(_find_admin_socket("client.3")) 2024-03-30T13:11:44.431 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-03-30T13:11:44.554 INFO:teuthology.orchestra.run.smithi113.stdout:/var/run/ceph/ceph-client.3.33138.asok 2024-03-30T13:11:44.558 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:11:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:11:44.566 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33138.asok 2024-03-30T13:11:44.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.567 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33138.asok status 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout: "metadata": { 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout: "entity_id": "3", 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": "smithi113", 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-30T13:11:44.721 INFO:teuthology.orchestra.run.smithi113.stdout: "pid": "33138", 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "root": "/" 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_count": 0, 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_pinned_count": 0, 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 4714, 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "inst": { 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "name": { 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "client", 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "num": 4714 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:44.722 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.2:0", 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 85922431 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2024-03-30T13:11:44.723 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.2:0", 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 85922431 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: "inst_str": "client.4714 192.168.0.2:0/85922431", 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: "addr_str": "192.168.0.2:0/85922431", 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: "inode_count": 1, 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_epoch": 9, 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch": 29, 2024-03-30T13:11:44.724 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:11:44.725 INFO:teuthology.orchestra.run.smithi113.stdout: "blocklisted": false, 2024-03-30T13:11:44.725 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs" 2024-03-30T13:11:44.725 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-03-30T13:11:44.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.734 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-03-30T13:11:44.734 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:44.742 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-03-30T13:11:44.743 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:44.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:44.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.798 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-03-30T13:11:44.798 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:44.851 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-03-30T13:11:44.851 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:44.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-30T13:11:44.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.906 DEBUG:teuthology.orchestra.run.smithi113:> stat --file-system '--printf=%T 2024-03-30T13:11:44.906 DEBUG:teuthology.orchestra.run.smithi113:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:44.913 INFO:teuthology.orchestra.run.smithi113.stdout:fuseblk 2024-03-30T13:11:44.914 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:44.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.914 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:11:44.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:44.970 DEBUG:teuthology.orchestra.run.smithi113:> stat --file-system '--printf=%T 2024-03-30T13:11:44.970 DEBUG:teuthology.orchestra.run.smithi113:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:45.021 INFO:teuthology.orchestra.run.smithi113.stdout:fuseblk 2024-03-30T13:11:45.021 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:45.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:45.021 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-30T13:11:45.078 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-30T13:11:45.442 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:45.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe6703c640 1 Processor -- start 2024-03-30T13:11:45.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe6703c640 1 -- start start 2024-03-30T13:11:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe6703c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efe600b5b20 0x7efe6006d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe6703c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6015edd0 con 0x7efe60070490 2024-03-30T13:11:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe6703c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe6015f0b0 con 0x7efe6006dcc0 2024-03-30T13:11:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe6703c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efe6015f390 con 0x7efe600b5b20 2024-03-30T13:11:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe64dd8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efe60070490 0x7efe6015c130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60530/0 (socket says 172.21.15.43:60530) 2024-03-30T13:11:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe64dd8640 1 -- 172.21.15.43:0/1639471021 learned_addr learned my addr 172.21.15.43:0/1639471021 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:45.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5ffff640 1 --2- 172.21.15.43:0/1639471021 >> v2:172.21.15.113:3300/0 conn(0x7efe600b5b20 0x7efe6006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894552590 0 0) 0x7efe6015edd0 con 0x7efe60070490 2024-03-30T13:11:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe50005740 con 0x7efe60070490 2024-03-30T13:11:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 151189874 0 0) 0x7efe50005740 con 0x7efe60070490 2024-03-30T13:11:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 >> v2:172.21.15.113:3300/0 conn(0x7efe600b5b20 msgr2=0x7efe6006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 --2- 172.21.15.43:0/1639471021 >> v2:172.21.15.113:3300/0 conn(0x7efe600b5b20 0x7efe6006d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 >> v1:172.21.15.113:6789/0 conn(0x7efe6006dcc0 legacy=0x7efe6006e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6015f670 con 0x7efe60070490 2024-03-30T13:11:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7efe540016a0 con 0x7efe60070490 2024-03-30T13:11:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 >> v1:172.21.15.43:6789/0 conn(0x7efe60070490 legacy=0x7efe6015c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe50005bd0 0x7efe50005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50006a20 0x7efe50006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe50007500 0x7efe5000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe6015f390 con 0x7efe50005bd0 2024-03-30T13:11:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.563+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe5000e3d0 con 0x7efe50006a20 2024-03-30T13:11:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efe5000e5d0 con 0x7efe50007500 2024-03-30T13:11:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe64dd8640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe50005bd0 0x7efe50005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe64dd8640 1 -- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe50007500 msgr2=0x7efe5000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe655d9640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe50007500 0x7efe5000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe5ffff640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50006a20 0x7efe50006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe64dd8640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe50007500 0x7efe5000dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe64dd8640 1 -- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50006a20 msgr2=0x7efe50006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe64dd8640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50006a20 0x7efe50006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe64dd8640 1 -- 172.21.15.43:0/1639471021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe50005740 con 0x7efe50005bd0 2024-03-30T13:11:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe655d9640 1 -- 172.21.15.43:0/1639471021 reap_dead start 2024-03-30T13:11:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe5ffff640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50006a20 0x7efe50006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe64dd8640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe50005bd0 0x7efe50005ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe60073a20 tx=0x7efe5401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73fd0eba05dafbb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54007d60 con 0x7efe50005bd0 2024-03-30T13:11:45.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe540a9070 con 0x7efe50005bd0 2024-03-30T13:11:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe5f7fe640 1 -- 172.21.15.43:0/1639471021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54005d40 con 0x7efe50005bd0 2024-03-30T13:11:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe6703c640 1 -- 172.21.15.43:0/1639471021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe50005bd0 msgr2=0x7efe50005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe6703c640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe50005bd0 0x7efe50005ff0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe60073a20 tx=0x7efe5401ccc0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe6703c640 1 -- 172.21.15.43:0/1639471021 shutdown_connections 2024-03-30T13:11:45.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe6703c640 1 --2- 172.21.15.43:0/1639471021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe50005bd0 0x7efe50005ff0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe6703c640 1 -- 172.21.15.43:0/1639471021 >> 172.21.15.43:0/1639471021 conn(0x7efe60072500 msgr2=0x7efe60072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe6703c640 1 -- 172.21.15.43:0/1639471021 shutdown_connections 2024-03-30T13:11:45.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.567+0000 7efe6703c640 1 -- 172.21.15.43:0/1639471021 wait complete. 2024-03-30T13:11:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 Processor -- start 2024-03-30T13:11:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 -- start start 2024-03-30T13:11:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50005bd0 0x7efe601468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe600ad8e0 0x7efe600add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe601563c0 0x7efe60156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efe600b4820 con 0x7efe600ad8e0 2024-03-30T13:11:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efe600b46a0 con 0x7efe50005bd0 2024-03-30T13:11:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efe600b49a0 con 0x7efe601563c0 2024-03-30T13:11:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe600ad8e0 0x7efe600add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe600ad8e0 0x7efe600add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46194/0 (socket says 172.21.15.43:46194) 2024-03-30T13:11:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 -- 172.21.15.43:0/3737623448 learned_addr learned my addr 172.21.15.43:0/3737623448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe64dd8640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50005bd0 0x7efe601468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe655d9640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe601563c0 0x7efe60156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 -- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe601563c0 msgr2=0x7efe60156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe601563c0 0x7efe60156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 -- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50005bd0 msgr2=0x7efe601468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50005bd0 0x7efe601468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 -- 172.21.15.43:0/3737623448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe60146df0 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5ffff640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe600ad8e0 0x7efe600add60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe4c010790 tx=0x7efe4c010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5d7fa640 1 -- 172.21.15.43:0/3737623448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00f040 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5d7fa640 1 -- 172.21.15.43:0/3737623448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe4c00eb00 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe60155020 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5d7fa640 1 -- 172.21.15.43:0/3737623448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c010ea0 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe600ae730 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe60068890 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.571+0000 7efe5d7fa640 1 -- 172.21.15.43:0/3737623448 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efe4c01c400 con 0x7efe600ad8e0 2024-03-30T13:11:45.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.575+0000 7efe5d7fa640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efe44067ff0 0x7efe4406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.575+0000 7efe5d7fa640 1 -- 172.21.15.43:0/3737623448 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7efe4c086c80 con 0x7efe600ad8e0 2024-03-30T13:11:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.575+0000 7efe64dd8640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efe44067ff0 0x7efe4406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.575+0000 7efe64dd8640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efe44067ff0 0x7efe4406a4b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efe540ab000 tx=0x7efe540b0040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:45.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.579+0000 7efe5d7fa640 1 -- 172.21.15.43:0/3737623448 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efe4c018030 con 0x7efe600ad8e0 2024-03-30T13:11:45.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.739+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe600b53f0 con 0x7efe600ad8e0 2024-03-30T13:11:45.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.739+0000 7efe5d7fa640 1 -- 172.21.15.43:0/3737623448 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7efe4c0570c0 con 0x7efe600ad8e0 2024-03-30T13:11:45.748 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:45.748 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:45.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efe44067ff0 msgr2=0x7efe4406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efe44067ff0 0x7efe4406a4b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efe540ab000 tx=0x7efe540b0040 comp rx=0 tx=0).stop 2024-03-30T13:11:45.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe600ad8e0 msgr2=0x7efe600add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe600ad8e0 0x7efe600add60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe4c010790 tx=0x7efe4c010c50 comp rx=0 tx=0).stop 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 shutdown_connections 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efe44067ff0 0x7efe4406a4b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efe601563c0 0x7efe60156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe600ad8e0 0x7efe600add60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 --2- 172.21.15.43:0/3737623448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efe50005bd0 0x7efe601468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 >> 172.21.15.43:0/3737623448 conn(0x7efe60072500 msgr2=0x7efe601577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 shutdown_connections 2024-03-30T13:11:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.743+0000 7efe6703c640 1 -- 172.21.15.43:0/3737623448 wait complete. 2024-03-30T13:11:45.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:45.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10dd0aa640 1 Processor -- start 2024-03-30T13:11:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10dd0aa640 1 -- start start 2024-03-30T13:11:45.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10dd0aa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 0x7f10d8160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10dd0aa640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d8160c80 con 0x7f10d815c160 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10dd0aa640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d80cd680 con 0x7f10d80ccaa0 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10dd0aa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f10d81607d0 con 0x7f10d815fe20 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f10d815c160 0x7f10d806b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60536/0 (socket says 172.21.15.43:60536) 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 -- 172.21.15.43:0/1026406074 learned_addr learned my addr 172.21.15.43:0/1026406074 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d659c640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 0x7f10d8160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645005440 0 0) 0x7f10d8160c80 con 0x7f10d815c160 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10c0003530 con 0x7f10d815c160 2024-03-30T13:11:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1877721922 0 0) 0x7f10c0003530 con 0x7f10d815c160 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 msgr2=0x7f10d8160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 0x7f10d8160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 >> v1:172.21.15.113:6789/0 conn(0x7f10d80ccaa0 legacy=0x7f10d80cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80cd960 con 0x7f10d815c160 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f10d0001a00 con 0x7f10d815c160 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 >> v1:172.21.15.43:6789/0 conn(0x7f10d815c160 legacy=0x7f10d806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c00043e0 0x7f10c00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 0x7f10c0005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10d81607d0 con 0x7f10c00039f0 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f10c000ba30 con 0x7f10c00043e0 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f10c000bc30 con 0x7f10d815fe20 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d659c640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c00043e0 0x7f10c00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d759e640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 0x7f10c0005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 -- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 msgr2=0x7f10c0005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:45.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815fe20 0x7f10c0005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 -- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c00043e0 msgr2=0x7f10c00048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:45.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10c00043e0 0x7f10c00048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.899+0000 7f10d6d9d640 1 -- 172.21.15.43:0/1026406074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80cd960 con 0x7f10c00039f0 2024-03-30T13:11:45.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10d6d9d640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10c0003e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f10d000a940 tx=0x7f10d001d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10639a6c34d039ae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d000a540 con 0x7f10c00039f0 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d00072d0 con 0x7f10c00039f0 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10d5d9b640 1 -- 172.21.15.43:0/1026406074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d00076a0 con 0x7f10c00039f0 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 -- 172.21.15.43:0/1026406074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 msgr2=0x7f10c0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 --2- 172.21.15.43:0/1026406074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10c0003e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f10d000a940 tx=0x7f10d001d430 comp rx=0 tx=0).stop 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10d759e640 1 -- 172.21.15.43:0/1026406074 reap_dead start 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 -- 172.21.15.43:0/1026406074 shutdown_connections 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 -- 172.21.15.43:0/1026406074 >> 172.21.15.43:0/1026406074 conn(0x7f10d80d0c90 msgr2=0x7f10d80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 -- 172.21.15.43:0/1026406074 shutdown_connections 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 -- 172.21.15.43:0/1026406074 wait complete. 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 Processor -- start 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.903+0000 7f10dd0aa640 1 -- start start 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10d815aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815b480 0x7f10d8161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d81618a0 0x7f10d8159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f10d815d6d0 con 0x7f10c00039f0 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f10d815d550 con 0x7f10d81618a0 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80d0390 con 0x7f10d815b480 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10d815aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10d815aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46204/0 (socket says 172.21.15.43:46204) 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 -- 172.21.15.43:0/3729102077 learned_addr learned my addr 172.21.15.43:0/3729102077 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:45.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d759e640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d81618a0 0x7f10d8159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d659c640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815b480 0x7f10d8161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 -- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815b480 msgr2=0x7f10d8161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815b480 0x7f10d8161360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 -- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d81618a0 msgr2=0x7f10d8159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d81618a0 0x7f10d8159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 -- 172.21.15.43:0/3729102077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80c5930 con 0x7f10c00039f0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d659c640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815b480 0x7f10d8161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10d6d9d640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10d815aa60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f10d0001d00 tx=0x7f10c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10bf7fe640 1 -- 172.21.15.43:0/3729102077 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0004b70 con 0x7f10c00039f0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10bf7fe640 1 -- 172.21.15.43:0/3729102077 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d0004d10 con 0x7f10c00039f0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d815b900 con 0x7f10c00039f0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10bf7fe640 1 -- 172.21.15.43:0/3729102077 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0001a00 con 0x7f10c00039f0 2024-03-30T13:11:45.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d80c5eb0 con 0x7f10c00039f0 2024-03-30T13:11:45.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d8068900 con 0x7f10c00039f0 2024-03-30T13:11:45.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10bf7fe640 1 -- 172.21.15.43:0/3729102077 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f10d00bd060 con 0x7f10c00039f0 2024-03-30T13:11:45.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10bf7fe640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f10c4067fc0 0x7f10c406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:45.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.907+0000 7f10bf7fe640 1 -- 172.21.15.43:0/3729102077 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f10d0120700 con 0x7f10c00039f0 2024-03-30T13:11:45.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.911+0000 7f10d659c640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f10c4067fc0 0x7f10c406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:45.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.911+0000 7f10d659c640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f10c4067fc0 0x7f10c406a480 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10d80b2080 tx=0x7f10cc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:45.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:45.911+0000 7f10bf7fe640 1 -- 172.21.15.43:0/3729102077 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f10d0017170 con 0x7f10c00039f0 2024-03-30T13:11:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.075+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10d806e040 con 0x7f10c00039f0 2024-03-30T13:11:46.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.075+0000 7f10bf7fe640 1 -- 172.21.15.43:0/3729102077 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f10d00b4380 con 0x7f10c00039f0 2024-03-30T13:11:46.084 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f10c4067fc0 msgr2=0x7f10c406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f10c4067fc0 0x7f10c406a480 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f10d80b2080 tx=0x7f10cc008040 comp rx=0 tx=0).stop 2024-03-30T13:11:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 msgr2=0x7f10d815aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10d815aa60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f10d0001d00 tx=0x7f10c0005840 comp rx=0 tx=0).stop 2024-03-30T13:11:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 shutdown_connections 2024-03-30T13:11:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f10c4067fc0 0x7f10c406a480 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f10d81618a0 0x7f10d8159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f10d815b480 0x7f10d8161360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 --2- 172.21.15.43:0/3729102077 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10c00039f0 0x7f10d815aa60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 >> 172.21.15.43:0/3729102077 conn(0x7f10d80d0c90 msgr2=0x7f10d815ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 shutdown_connections 2024-03-30T13:11:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.079+0000 7f10dd0aa640 1 -- 172.21.15.43:0/3729102077 wait complete. 2024-03-30T13:11:46.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:46.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c2a84640 1 Processor -- start 2024-03-30T13:11:46.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c2a84640 1 -- start start 2024-03-30T13:11:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c2a84640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f00bc06c790 0x7f00bc06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c2a84640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f00bc06f9e0 0x7f00bc06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c2a84640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00bc15e9e0 con 0x7f00bc161fd0 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c2a84640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f00bc070370 con 0x7f00bc06f9e0 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c2a84640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f00bc15f8b0 con 0x7f00bc06c790 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00bbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f00bc06f9e0 0x7f00bc06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c1021640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f00bc161fd0 0x7f00bc1623e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60552/0 (socket says 172.21.15.43:60552) 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c1021640 1 -- 172.21.15.43:0/3548727062 learned_addr learned my addr 172.21.15.43:0/3548727062 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00c0820640 1 --2- 172.21.15.43:0/3548727062 >> v2:172.21.15.113:3300/0 conn(0x7f00bc06c790 0x7f00bc06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00bbfff640 1 -- 172.21.15.43:0/3548727062 >> v2:172.21.15.113:3300/0 conn(0x7f00bc06c790 msgr2=0x7f00bc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00bbfff640 1 --2- 172.21.15.43:0/3548727062 >> v2:172.21.15.113:3300/0 conn(0x7f00bc06c790 0x7f00bc06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00bbfff640 1 -- 172.21.15.43:0/3548727062 >> v1:172.21.15.43:6789/0 conn(0x7f00bc161fd0 legacy=0x7f00bc1623e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.215+0000 7f00bbfff640 1 -- 172.21.15.43:0/3548727062 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc15eca0 con 0x7f00bc06f9e0 2024-03-30T13:11:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00bbfff640 1 --2- 172.21.15.43:0/3548727062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06f9e0 0x7f00bc06fe30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f00bc06f720 tx=0x7f00ac01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ba03f6d5cdec02 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00bb7fe640 1 -- 172.21.15.43:0/3548727062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0ab650 con 0x7f00bc06f9e0 2024-03-30T13:11:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00bb7fe640 1 -- 172.21.15.43:0/3548727062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac0b0070 con 0x7f00bc06f9e0 2024-03-30T13:11:46.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00bb7fe640 1 -- 172.21.15.43:0/3548727062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0aba10 con 0x7f00bc06f9e0 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 -- 172.21.15.43:0/3548727062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06f9e0 msgr2=0x7f00bc06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 --2- 172.21.15.43:0/3548727062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06f9e0 0x7f00bc06fe30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f00bc06f720 tx=0x7f00ac01c450 comp rx=0 tx=0).stop 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 -- 172.21.15.43:0/3548727062 shutdown_connections 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 --2- 172.21.15.43:0/3548727062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06f9e0 0x7f00bc06fe30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 --2- 172.21.15.43:0/3548727062 >> v2:172.21.15.113:3300/0 conn(0x7f00bc06c790 0x7f00bc06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 -- 172.21.15.43:0/3548727062 >> 172.21.15.43:0/3548727062 conn(0x7f00bc0d1070 msgr2=0x7f00bc0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 -- 172.21.15.43:0/3548727062 shutdown_connections 2024-03-30T13:11:46.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 -- 172.21.15.43:0/3548727062 wait complete. 2024-03-30T13:11:46.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 Processor -- start 2024-03-30T13:11:46.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.219+0000 7f00c2a84640 1 -- start start 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06c790 0x7f00bc14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00bc06f9e0 0x7f00bc0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00bc161fd0 0x7f00bc0ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc06cb70 con 0x7f00bc06c790 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc070b20 con 0x7f00bc06f9e0 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc06ccf0 con 0x7f00bc161fd0 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06c790 0x7f00bc14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06c790 0x7f00bc14bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46232/0 (socket says 172.21.15.43:46232) 2024-03-30T13:11:46.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 -- 172.21.15.43:0/2772625050 learned_addr learned my addr 172.21.15.43:0/2772625050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00bbfff640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00bc06f9e0 0x7f00bc0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c1021640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00bc161fd0 0x7f00bc0ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 -- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00bc161fd0 msgr2=0x7f00bc0ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00bc161fd0 0x7f00bc0ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 -- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00bc06f9e0 msgr2=0x7f00bc0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00bc06f9e0 0x7f00bc0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 -- 172.21.15.43:0/2772625050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc14d040 con 0x7f00bc06c790 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00bbfff640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00bc06f9e0 0x7f00bc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c1021640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00bc161fd0 0x7f00bc0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c0820640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06c790 0x7f00bc14bd40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00a400d830 tx=0x7f00a400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00b97fa640 1 -- 172.21.15.43:0/2772625050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a400b6c0 con 0x7f00bc06c790 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00b97fa640 1 -- 172.21.15.43:0/2772625050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a4019070 con 0x7f00bc06c790 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc14d2e0 con 0x7f00bc06c790 2024-03-30T13:11:46.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00b97fa640 1 -- 172.21.15.43:0/2772625050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a400ba60 con 0x7f00bc06c790 2024-03-30T13:11:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc14d8a0 con 0x7f00bc06c790 2024-03-30T13:11:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc068900 con 0x7f00bc06c790 2024-03-30T13:11:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00b97fa640 1 -- 172.21.15.43:0/2772625050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f00a401e460 con 0x7f00bc06c790 2024-03-30T13:11:46.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00b97fa640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f009c067fc0 0x7f009c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.223+0000 7f00b97fa640 1 -- 172.21.15.43:0/2772625050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f00a4087950 con 0x7f00bc06c790 2024-03-30T13:11:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.227+0000 7f00bbfff640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f009c067fc0 0x7f009c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.227+0000 7f00bbfff640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f009c067fc0 0x7f009c06a480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0af000 tx=0x7f00ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.227+0000 7f00b97fa640 1 -- 172.21.15.43:0/2772625050 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f00a4057f00 con 0x7f00bc06c790 2024-03-30T13:11:46.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.383+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f00bc14c280 con 0x7f00bc06c790 2024-03-30T13:11:46.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.383+0000 7f00b97fa640 1 -- 172.21.15.43:0/2772625050 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f00a4057800 con 0x7f00bc06c790 2024-03-30T13:11:46.392 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:46.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f009c067fc0 msgr2=0x7f009c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f009c067fc0 0x7f009c06a480 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0af000 tx=0x7f00ac008040 comp rx=0 tx=0).stop 2024-03-30T13:11:46.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06c790 msgr2=0x7f00bc14bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06c790 0x7f00bc14bd40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00a400d830 tx=0x7f00a400dcf0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 shutdown_connections 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f009c067fc0 0x7f009c06a480 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00bc161fd0 0x7f00bc0ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00bc06f9e0 0x7f00bc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 --2- 172.21.15.43:0/2772625050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00bc06c790 0x7f00bc14bd40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 >> 172.21.15.43:0/2772625050 conn(0x7f00bc0d1070 msgr2=0x7f00bc0b2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 shutdown_connections 2024-03-30T13:11:46.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.387+0000 7f00c2a84640 1 -- 172.21.15.43:0/2772625050 wait complete. 2024-03-30T13:11:46.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:46.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 Processor -- start 2024-03-30T13:11:46.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 -- start start 2024-03-30T13:11:46.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6768144b30 0x7f67680cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67680cd2e0 0x7f67680b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67680b44b0 0x7f6768073000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f676806f810 con 0x7f67680cd2e0 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f67680d0180 con 0x7f67680b44b0 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f67704ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6768070090 con 0x7f6768144b30 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67680cd2e0 0x7f67680b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67680cd2e0 0x7f67680b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46236/0 (socket says 172.21.15.43:46236) 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 -- 172.21.15.43:0/3598090840 learned_addr learned my addr 172.21.15.43:0/3598090840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676ea6b640 1 --2- 172.21.15.43:0/3598090840 >> v2:172.21.15.113:3300/0 conn(0x7f67680b44b0 0x7f6768073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676e26a640 1 --2- 172.21.15.43:0/3598090840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6768144b30 0x7f67680cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 -- 172.21.15.43:0/3598090840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6768144b30 msgr2=0x7f67680cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 --2- 172.21.15.43:0/3598090840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6768144b30 0x7f67680cf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 -- 172.21.15.43:0/3598090840 >> v2:172.21.15.113:3300/0 conn(0x7f67680b44b0 msgr2=0x7f6768073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 --2- 172.21.15.43:0/3598090840 >> v2:172.21.15.113:3300/0 conn(0x7f67680b44b0 0x7f6768073000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 -- 172.21.15.43:0/3598090840 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67680cfea0 con 0x7f67680cd2e0 2024-03-30T13:11:46.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676e26a640 1 --2- 172.21.15.43:0/3598090840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6768144b30 0x7f67680cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:46.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676da69640 1 --2- 172.21.15.43:0/3598090840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cd2e0 0x7f67680b3f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67680b0b90 tx=0x7f675801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c489e3335f221fe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.543+0000 7f676d268640 1 -- 172.21.15.43:0/3598090840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580b0070 con 0x7f67680cd2e0 2024-03-30T13:11:46.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f676d268640 1 -- 172.21.15.43:0/3598090840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67580ab600 con 0x7f67680cd2e0 2024-03-30T13:11:46.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f676d268640 1 -- 172.21.15.43:0/3598090840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67580ab9d0 con 0x7f67680cd2e0 2024-03-30T13:11:46.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 -- 172.21.15.43:0/3598090840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cd2e0 msgr2=0x7f67680b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 --2- 172.21.15.43:0/3598090840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cd2e0 0x7f67680b3f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f67680b0b90 tx=0x7f675801c4f0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 -- 172.21.15.43:0/3598090840 shutdown_connections 2024-03-30T13:11:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 --2- 172.21.15.43:0/3598090840 >> v2:172.21.15.113:3300/0 conn(0x7f67680b44b0 0x7f6768073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 --2- 172.21.15.43:0/3598090840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67680cd2e0 0x7f67680b3f70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 --2- 172.21.15.43:0/3598090840 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6768144b30 0x7f67680cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 -- 172.21.15.43:0/3598090840 >> 172.21.15.43:0/3598090840 conn(0x7f67680724e0 msgr2=0x7f67680728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 -- 172.21.15.43:0/3598090840 shutdown_connections 2024-03-30T13:11:46.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 -- 172.21.15.43:0/3598090840 wait complete. 2024-03-30T13:11:46.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 Processor -- start 2024-03-30T13:11:46.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.547+0000 7f67704ce640 1 -- start start 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67680b44b0 0x7f67681f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67680cd2e0 0x7f676806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6768144b30 0x7f676806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67680cfc80 con 0x7f6768144b30 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67680cfb00 con 0x7f67680cd2e0 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f67680cb630 con 0x7f67680b44b0 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6768144b30 0x7f676806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6768144b30 0x7f676806d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46238/0 (socket says 172.21.15.43:46238) 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 -- 172.21.15.43:0/2434256916 learned_addr learned my addr 172.21.15.43:0/2434256916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 -- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67680b44b0 msgr2=0x7f67681f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67680b44b0 0x7f67681f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 -- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67680cd2e0 msgr2=0x7f676806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67680cd2e0 0x7f676806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 -- 172.21.15.43:0/2434256916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67680c7300 con 0x7f6768144b30 2024-03-30T13:11:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676ea6b640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6768144b30 0x7f676806d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6764007e70 tx=0x7f676400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f675effd640 1 -- 172.21.15.43:0/2434256916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764014620 con 0x7f6768144b30 2024-03-30T13:11:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f675effd640 1 -- 172.21.15.43:0/2434256916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6764019070 con 0x7f6768144b30 2024-03-30T13:11:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67680c75e0 con 0x7f6768144b30 2024-03-30T13:11:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f675effd640 1 -- 172.21.15.43:0/2434256916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764014a20 con 0x7f6768144b30 2024-03-30T13:11:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676813e0a0 con 0x7f6768144b30 2024-03-30T13:11:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6768068900 con 0x7f6768144b30 2024-03-30T13:11:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f675effd640 1 -- 172.21.15.43:0/2434256916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6764014bc0 con 0x7f6768144b30 2024-03-30T13:11:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f675effd640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f674c067ff0 0x7f674c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f675effd640 1 -- 172.21.15.43:0/2434256916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f6764087160 con 0x7f6768144b30 2024-03-30T13:11:46.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.551+0000 7f676e26a640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f674c067ff0 0x7f674c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.555+0000 7f676e26a640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f674c067ff0 0x7f674c06a4b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6760006b20 tx=0x7f6760008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.555+0000 7f675effd640 1 -- 172.21.15.43:0/2434256916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6764011260 con 0x7f6768144b30 2024-03-30T13:11:46.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6768067ea0 con 0x7f6768144b30 2024-03-30T13:11:46.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f675effd640 1 -- 172.21.15.43:0/2434256916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f67640575a0 con 0x7f6768144b30 2024-03-30T13:11:46.730 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:46.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f674c067ff0 msgr2=0x7f674c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f674c067ff0 0x7f674c06a4b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6760006b20 tx=0x7f6760008040 comp rx=0 tx=0).stop 2024-03-30T13:11:46.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6768144b30 msgr2=0x7f676806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6768144b30 0x7f676806d300 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6764007e70 tx=0x7f676400e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 shutdown_connections 2024-03-30T13:11:46.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f674c067ff0 0x7f674c06a4b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6768144b30 0x7f676806d300 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67680cd2e0 0x7f676806cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 --2- 172.21.15.43:0/2434256916 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67680b44b0 0x7f67681f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 >> 172.21.15.43:0/2434256916 conn(0x7f67680724e0 msgr2=0x7f676815dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:46.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 shutdown_connections 2024-03-30T13:11:46.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.723+0000 7f67704ce640 1 -- 172.21.15.43:0/2434256916 wait complete. 2024-03-30T13:11:46.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:46.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef8113b640 1 Processor -- start 2024-03-30T13:11:46.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef8113b640 1 -- start start 2024-03-30T13:11:46.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef8113b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 0x7fef7c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef8113b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c160c80 con 0x7fef7c15fe20 2024-03-30T13:11:46.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef8113b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c0cd680 con 0x7fef7c0cd1b0 2024-03-30T13:11:46.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef8113b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c0cd960 con 0x7fef7c15c160 2024-03-30T13:11:46.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fef7c15fe20 0x7fef7c0ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60566/0 (socket says 172.21.15.43:60566) 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 -- 172.21.15.43:0/3122215370 learned_addr learned my addr 172.21.15.43:0/3122215370 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7ad9d640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 0x7fef7c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1725431643 0 0) 0x7fef7c160c80 con 0x7fef7c15fe20 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef64003530 con 0x7fef7c15fe20 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3276524043 0 0) 0x7fef64003530 con 0x7fef7c15fe20 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 msgr2=0x7fef7c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 0x7fef7c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 >> v1:172.21.15.113:6789/0 conn(0x7fef7c0cd1b0 legacy=0x7fef7c0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c0cdc40 con 0x7fef7c15fe20 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fef70001680 con 0x7fef7c15fe20 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 >> v1:172.21.15.43:6789/0 conn(0x7fef7c15fe20 legacy=0x7fef7c0ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 0x7fef640039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64004d50 0x7fef640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0cd960 con 0x7fef64003f70 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fef6400bab0 con 0x7fef64004d50 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fef6400bcb0 con 0x7fef7c15c160 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7ad9d640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 0x7fef640039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7b59e640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64004d50 0x7fef640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 -- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 msgr2=0x7fef640039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c15c160 0x7fef640039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 -- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64004d50 msgr2=0x7fef640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef64004d50 0x7fef640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.879+0000 7fef7a59c640 1 -- 172.21.15.43:0/3122215370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef64003530 con 0x7fef64003f70 2024-03-30T13:11:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef7a59c640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef64004810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c240 tx=0x7fef7001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feeaf11d694fa2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70007d20 con 0x7fef64003f70 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef70005ae0 con 0x7fef64003f70 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef79d9b640 1 -- 172.21.15.43:0/3122215370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70007070 con 0x7fef64003f70 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef8113b640 1 -- 172.21.15.43:0/3122215370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 msgr2=0x7fef64004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef8113b640 1 --2- 172.21.15.43:0/3122215370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef64004810 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c240 tx=0x7fef7001f3f0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef7b59e640 1 -- 172.21.15.43:0/3122215370 reap_dead start 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef8113b640 1 -- 172.21.15.43:0/3122215370 shutdown_connections 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef8113b640 1 -- 172.21.15.43:0/3122215370 >> 172.21.15.43:0/3122215370 conn(0x7fef7c0d0c90 msgr2=0x7fef7c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef8113b640 1 -- 172.21.15.43:0/3122215370 shutdown_connections 2024-03-30T13:11:46.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef8113b640 1 -- 172.21.15.43:0/3122215370 wait complete. 2024-03-30T13:11:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.883+0000 7fef8113b640 1 Processor -- start 2024-03-30T13:11:46.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 -- start start 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef7c1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef7c1f2ba0 0x7fef7c0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c0c7dc0 0x7fef7c0c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0cf420 con 0x7fef64003f70 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0cf2a0 con 0x7fef7c1f2ba0 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c15d550 con 0x7fef7c0c7dc0 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef7c1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef7c1f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46256/0 (socket says 172.21.15.43:46256) 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 -- 172.21.15.43:0/2886902886 learned_addr learned my addr 172.21.15.43:0/2886902886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7a59c640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef7c1f2ba0 0x7fef7c0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7b59e640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c0c7dc0 0x7fef7c0c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 -- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c0c7dc0 msgr2=0x7fef7c0c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c0c7dc0 0x7fef7c0c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 -- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef7c1f2ba0 msgr2=0x7fef7c0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef7c1f2ba0 0x7fef7c0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 -- 172.21.15.43:0/2886902886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c0c67e0 con 0x7fef64003f70 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7a59c640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef7c1f2ba0 0x7fef7c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7b59e640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c0c7dc0 0x7fef7c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef7ad9d640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef7c1f2660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef6c011610 tx=0x7fef6c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef637fe640 1 -- 172.21.15.43:0/2886902886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00f8f0 con 0x7fef64003f70 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef637fe640 1 -- 172.21.15.43:0/2886902886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6c00fa90 con 0x7fef64003f70 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c0c6ac0 con 0x7fef64003f70 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef637fe640 1 -- 172.21.15.43:0/2886902886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c00fe60 con 0x7fef64003f70 2024-03-30T13:11:46.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c0c7080 con 0x7fef64003f70 2024-03-30T13:11:46.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c068900 con 0x7fef64003f70 2024-03-30T13:11:46.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.887+0000 7fef637fe640 1 -- 172.21.15.43:0/2886902886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fef6c015460 con 0x7fef64003f70 2024-03-30T13:11:46.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.891+0000 7fef637fe640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef68067fc0 0x7fef6806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:46.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.891+0000 7fef637fe640 1 -- 172.21.15.43:0/2886902886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fef6c087220 con 0x7fef64003f70 2024-03-30T13:11:46.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.891+0000 7fef7a59c640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef68067fc0 0x7fef6806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:46.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.891+0000 7fef7a59c640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef68067fc0 0x7fef6806a480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef700af000 tx=0x7fef700b2040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:46.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:46.895+0000 7fef637fe640 1 -- 172.21.15.43:0/2886902886 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fef6c019030 con 0x7fef64003f70 2024-03-30T13:11:47.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.219+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fef7c06e040 con 0x7fef64003f70 2024-03-30T13:11:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.219+0000 7fef637fe640 1 -- 172.21.15.43:0/2886902886 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fef6c0570a0 con 0x7fef64003f70 2024-03-30T13:11:47.227 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:47.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef68067fc0 msgr2=0x7fef6806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef68067fc0 0x7fef6806a480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fef700af000 tx=0x7fef700b2040 comp rx=0 tx=0).stop 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 msgr2=0x7fef7c1f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef7c1f2660 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef6c011610 tx=0x7fef6c011ad0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 shutdown_connections 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef68067fc0 0x7fef6806a480 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef7c0c7dc0 0x7fef7c0c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef7c1f2ba0 0x7fef7c0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 --2- 172.21.15.43:0/2886902886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef64003f70 0x7fef7c1f2660 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 >> 172.21.15.43:0/2886902886 conn(0x7fef7c0d0c90 msgr2=0x7fef7c06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 shutdown_connections 2024-03-30T13:11:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.223+0000 7fef8113b640 1 -- 172.21.15.43:0/2886902886 wait complete. 2024-03-30T13:11:47.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:47.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.351+0000 7ff02d7cc640 1 Processor -- start 2024-03-30T13:11:47.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.351+0000 7ff02d7cc640 1 -- start start 2024-03-30T13:11:47.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff02d7cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806c670 0x7ff0280cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff02d7cc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff02806ca50 0x7ff0280ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff02d7cc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0280ccef0 con 0x7ff0280b41f0 2024-03-30T13:11:47.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff02d7cc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff0280b34c0 con 0x7ff02806ca50 2024-03-30T13:11:47.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff02d7cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff0280b3710 con 0x7ff02806c670 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff0277fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff02806ca50 0x7ff0280ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff026ffd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806c670 0x7ff0280cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff0267fc640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7ff0280b41f0 0x7ff02806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:47438/0 (socket says 172.21.15.43:47438) 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff0267fc640 1 -- 172.21.15.43:0/1166447728 learned_addr learned my addr 172.21.15.43:0/1166447728 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff0277fe640 1 -- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806c670 msgr2=0x7ff0280cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff0277fe640 1 --2- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806c670 0x7ff0280cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff0277fe640 1 -- 172.21.15.43:0/1166447728 >> v1:172.21.15.113:6789/0 conn(0x7ff0280b41f0 legacy=0x7ff02806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff0277fe640 1 -- 172.21.15.43:0/1166447728 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0280b39f0 con 0x7ff02806ca50 2024-03-30T13:11:47.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.355+0000 7ff026ffd640 1 --2- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806c670 0x7ff0280cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff0277fe640 1 --2- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff02806ca50 0x7ff0280ce5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff020009d90 tx=0x7ff02001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf599b098a67b12 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff025ffb640 1 -- 172.21.15.43:0/1166447728 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200b2070 con 0x7ff02806ca50 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff025ffb640 1 -- 172.21.15.43:0/1166447728 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0200a5940 con 0x7ff02806ca50 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff025ffb640 1 -- 172.21.15.43:0/1166447728 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0200a5d10 con 0x7ff02806ca50 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff02806ca50 msgr2=0x7ff0280ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff02806ca50 0x7ff0280ce5b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff020009d90 tx=0x7ff02001c970 comp rx=0 tx=0).stop 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1166447728 shutdown_connections 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff02806ca50 0x7ff0280ce5b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1166447728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806c670 0x7ff0280cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1166447728 >> 172.21.15.43:0/1166447728 conn(0x7ff0280cfee0 msgr2=0x7ff0280d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1166447728 shutdown_connections 2024-03-30T13:11:47.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1166447728 wait complete. 2024-03-30T13:11:47.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 Processor -- start 2024-03-30T13:11:47.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.359+0000 7ff02d7cc640 1 -- start start 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff02806c670 0x7ff02813fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806ca50 0x7ff028071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0280b41f0 0x7ff0280722b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff028169070 con 0x7ff02806c670 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff028168ef0 con 0x7ff0280b41f0 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff0281691f0 con 0x7ff02806ca50 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff02806c670 0x7ff02813fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff02806c670 0x7ff02813fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46270/0 (socket says 172.21.15.43:46270) 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 -- 172.21.15.43:0/1729739131 learned_addr learned my addr 172.21.15.43:0/1729739131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff0277fe640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0280b41f0 0x7ff0280722b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff0267fc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806ca50 0x7ff028071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 -- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806ca50 msgr2=0x7ff028071d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806ca50 0x7ff028071d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 -- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0280b41f0 msgr2=0x7ff0280722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0280b41f0 0x7ff0280722b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 -- 172.21.15.43:0/1729739131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff028165300 con 0x7ff02806c670 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff0277fe640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0280b41f0 0x7ff0280722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff026ffd640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff02806c670 0x7ff02813fc10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e980 tx=0x7ff01800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff01f7fe640 1 -- 172.21.15.43:0/1729739131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff018017070 con 0x7ff02806c670 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff01f7fe640 1 -- 172.21.15.43:0/1729739131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff01800ca00 con 0x7ff02806c670 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0281655e0 con 0x7ff02806c670 2024-03-30T13:11:47.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff01f7fe640 1 -- 172.21.15.43:0/1729739131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff01800cdd0 con 0x7ff02806c670 2024-03-30T13:11:47.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff028140150 con 0x7ff02806c670 2024-03-30T13:11:47.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff028068890 con 0x7ff02806c670 2024-03-30T13:11:47.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff01f7fe640 1 -- 172.21.15.43:0/1729739131 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff01801a070 con 0x7ff02806c670 2024-03-30T13:11:47.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff01f7fe640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff008067fc0 0x7ff00806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff01f7fe640 1 -- 172.21.15.43:0/1729739131 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7ff018086af0 con 0x7ff02806c670 2024-03-30T13:11:47.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.363+0000 7ff0267fc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff008067fc0 0x7ff00806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.367+0000 7ff0267fc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff008067fc0 0x7ff00806a480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff02813b7b0 tx=0x7ff010009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:47.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.375+0000 7ff01f7fe640 1 -- 172.21.15.43:0/1729739131 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff018056fc0 con 0x7ff02806c670 2024-03-30T13:11:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.531+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0280b32a0 con 0x7ff02806c670 2024-03-30T13:11:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff01f7fe640 1 -- 172.21.15.43:0/1729739131 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7ff0180568c0 con 0x7ff02806c670 2024-03-30T13:11:47.542 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:47.542 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff008067fc0 msgr2=0x7ff00806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff008067fc0 0x7ff00806a480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff02813b7b0 tx=0x7ff010009040 comp rx=0 tx=0).stop 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff02806c670 msgr2=0x7ff02813fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff02806c670 0x7ff02813fc10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff01800e980 tx=0x7ff01800ee40 comp rx=0 tx=0).stop 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 shutdown_connections 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff008067fc0 0x7ff00806a480 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff0280b41f0 0x7ff0280722b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff02806ca50 0x7ff028071d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 --2- 172.21.15.43:0/1729739131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff02806c670 0x7ff02813fc10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 >> 172.21.15.43:0/1729739131 conn(0x7ff0280cfee0 msgr2=0x7ff02806dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 shutdown_connections 2024-03-30T13:11:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.535+0000 7ff02d7cc640 1 -- 172.21.15.43:0/1729739131 wait complete. 2024-03-30T13:11:47.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:47.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 Processor -- start 2024-03-30T13:11:47.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 -- start start 2024-03-30T13:11:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006c790 0x7f9dd006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9dd006f9e0 0x7f9dd006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9dd0161fd0 0x7f9dd0162440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd015f8b0 con 0x7f9dd006f9e0 2024-03-30T13:11:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd0070340 con 0x7f9dd0161fd0 2024-03-30T13:11:47.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd7695640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd015e9e0 con 0x7f9dd006c790 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9dd006f9e0 0x7f9dd006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9dd006f9e0 0x7f9dd006fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46276/0 (socket says 172.21.15.43:46276) 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 -- 172.21.15.43:0/1877512581 learned_addr learned my addr 172.21.15.43:0/1877512581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd5c32640 1 --2- 172.21.15.43:0/1877512581 >> v2:172.21.15.113:3300/0 conn(0x7f9dd0161fd0 0x7f9dd0162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd5431640 1 --2- 172.21.15.43:0/1877512581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006c790 0x7f9dd006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 -- 172.21.15.43:0/1877512581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006c790 msgr2=0x7f9dd006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 --2- 172.21.15.43:0/1877512581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006c790 0x7f9dd006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 -- 172.21.15.43:0/1877512581 >> v2:172.21.15.113:3300/0 conn(0x7f9dd0161fd0 msgr2=0x7f9dd0162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 --2- 172.21.15.43:0/1877512581 >> v2:172.21.15.113:3300/0 conn(0x7f9dd0161fd0 0x7f9dd0162440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd4c30640 1 -- 172.21.15.43:0/1877512581 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd015eca0 con 0x7f9dd006f9e0 2024-03-30T13:11:47.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.683+0000 7f9dd5c32640 1 --2- 172.21.15.43:0/1877512581 >> v2:172.21.15.113:3300/0 conn(0x7f9dd0161fd0 0x7f9dd0162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd4c30640 1 --2- 172.21.15.43:0/1877512581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006f9e0 0x7f9dd006fe00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00509c0 tx=0x7f9dc001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9620a69f9ab62e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dc7fff640 1 -- 172.21.15.43:0/1877512581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00aa610 con 0x7f9dd006f9e0 2024-03-30T13:11:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dc7fff640 1 -- 172.21.15.43:0/1877512581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc00af070 con 0x7f9dd006f9e0 2024-03-30T13:11:47.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dc7fff640 1 -- 172.21.15.43:0/1877512581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00aa9d0 con 0x7f9dd006f9e0 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- 172.21.15.43:0/1877512581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006f9e0 msgr2=0x7f9dd006fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 --2- 172.21.15.43:0/1877512581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006f9e0 0x7f9dd006fe00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00509c0 tx=0x7f9dc001c4f0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- 172.21.15.43:0/1877512581 shutdown_connections 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 --2- 172.21.15.43:0/1877512581 >> v2:172.21.15.113:3300/0 conn(0x7f9dd0161fd0 0x7f9dd0162440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 --2- 172.21.15.43:0/1877512581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006f9e0 0x7f9dd006fe00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 --2- 172.21.15.43:0/1877512581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006c790 0x7f9dd006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- 172.21.15.43:0/1877512581 >> 172.21.15.43:0/1877512581 conn(0x7f9dd00d1070 msgr2=0x7f9dd00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- 172.21.15.43:0/1877512581 shutdown_connections 2024-03-30T13:11:47.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- 172.21.15.43:0/1877512581 wait complete. 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 Processor -- start 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- start start 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006c790 0x7f9dd0153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006f9e0 0x7f9dd0147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dd0161fd0 0x7f9dd00ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0070b20 con 0x7f9dd006c790 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd0162d90 con 0x7f9dd0161fd0 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.687+0000 7f9dd7695640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd006cb70 con 0x7f9dd006f9e0 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006c790 0x7f9dd0153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006c790 0x7f9dd0153ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46290/0 (socket says 172.21.15.43:46290) 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 -- 172.21.15.43:0/912503285 learned_addr learned my addr 172.21.15.43:0/912503285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5c32640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dd0161fd0 0x7f9dd00ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd4c30640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006f9e0 0x7f9dd0147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 -- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006f9e0 msgr2=0x7f9dd0147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006f9e0 0x7f9dd0147420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 -- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dd0161fd0 msgr2=0x7f9dd00ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dd0161fd0 0x7f9dd00ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 -- 172.21.15.43:0/912503285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd00ce4b0 con 0x7f9dd006c790 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd5431640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006c790 0x7f9dd0153ff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc007e60 tx=0x7f9dcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/912503285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc014620 con 0x7f9dd006c790 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/912503285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc019070 con 0x7f9dd006c790 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd0148080 con 0x7f9dd006c790 2024-03-30T13:11:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/912503285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0149e0 con 0x7f9dd006c790 2024-03-30T13:11:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd0148640 con 0x7f9dd006c790 2024-03-30T13:11:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd0068900 con 0x7f9dd006c790 2024-03-30T13:11:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/912503285 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9dcc014b80 con 0x7f9dd006c790 2024-03-30T13:11:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dc5ffb640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9db4067fc0 0x7f9db406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/912503285 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f9dcc086fd0 con 0x7f9dd006c790 2024-03-30T13:11:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.691+0000 7f9dd4c30640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9db4067fc0 0x7f9db406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:47.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.695+0000 7f9dd4c30640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9db4067fc0 0x7f9db406a480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00ae000 tx=0x7f9dc00a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.695+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/912503285 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9dcc0574a0 con 0x7f9dd006c790 2024-03-30T13:11:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.863+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9dd0169140 con 0x7f9dd006c790 2024-03-30T13:11:47.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.863+0000 7f9dc5ffb640 1 -- 172.21.15.43:0/912503285 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f9dcc056da0 con 0x7f9dd006c790 2024-03-30T13:11:47.872 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:47.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9db4067fc0 msgr2=0x7f9db406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9db4067fc0 0x7f9db406a480 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00ae000 tx=0x7f9dc00a7000 comp rx=0 tx=0).stop 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006c790 msgr2=0x7f9dd0153ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006c790 0x7f9dd0153ff0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc007e60 tx=0x7f9dcc00e4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 shutdown_connections 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9db4067fc0 0x7f9db406a480 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9dd0161fd0 0x7f9dd00ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9dd006f9e0 0x7f9dd0147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 --2- 172.21.15.43:0/912503285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dd006c790 0x7f9dd0153ff0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 >> 172.21.15.43:0/912503285 conn(0x7f9dd00d1070 msgr2=0x7f9dd01612d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 shutdown_connections 2024-03-30T13:11:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.867+0000 7f9dd7695640 1 -- 172.21.15.43:0/912503285 wait complete. 2024-03-30T13:11:47.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:48.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 Processor -- start 2024-03-30T13:11:48.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 -- start start 2024-03-30T13:11:48.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5fec15f880 0x7f5fec0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec070640 0x7f5fec06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5fec06eec0 0x7f5fec0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5fec070aa0 con 0x7f5fec15f880 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5fec0ce4d0 con 0x7f5fec06eec0 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5ff0b3c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5fec0cc7f0 con 0x7f5fec070640 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5fec15f880 0x7f5fec0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5fec15f880 0x7f5fec0b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46296/0 (socket says 172.21.15.43:46296) 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 -- 172.21.15.43:0/2324877005 learned_addr learned my addr 172.21.15.43:0/2324877005 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fe9d9b640 1 --2- 172.21.15.43:0/2324877005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec070640 0x7f5fec06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fead9d640 1 --2- 172.21.15.43:0/2324877005 >> v2:172.21.15.113:3300/0 conn(0x7f5fec06eec0 0x7f5fec0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 -- 172.21.15.43:0/2324877005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec070640 msgr2=0x7f5fec06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 --2- 172.21.15.43:0/2324877005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec070640 0x7f5fec06e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 -- 172.21.15.43:0/2324877005 >> v2:172.21.15.113:3300/0 conn(0x7f5fec06eec0 msgr2=0x7f5fec0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 --2- 172.21.15.43:0/2324877005 >> v2:172.21.15.113:3300/0 conn(0x7f5fec06eec0 0x7f5fec0ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:47.999+0000 7f5fea59c640 1 -- 172.21.15.43:0/2324877005 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec0ccad0 con 0x7f5fec15f880 2024-03-30T13:11:48.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5fea59c640 1 --2- 172.21.15.43:0/2324877005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec15f880 0x7f5fec0b4ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0cc6e0 tx=0x7f5fdc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9781d3e93afa24f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5fe959a640 1 -- 172.21.15.43:0/2324877005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0b0070 con 0x7f5fec15f880 2024-03-30T13:11:48.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5fe959a640 1 -- 172.21.15.43:0/2324877005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fdc0a4e50 con 0x7f5fec15f880 2024-03-30T13:11:48.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5fe959a640 1 -- 172.21.15.43:0/2324877005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc0ab820 con 0x7f5fec15f880 2024-03-30T13:11:48.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2324877005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec15f880 msgr2=0x7f5fec0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2324877005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec15f880 0x7f5fec0b4ff0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0cc6e0 tx=0x7f5fdc01c400 comp rx=0 tx=0).stop 2024-03-30T13:11:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2324877005 shutdown_connections 2024-03-30T13:11:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2324877005 >> v2:172.21.15.113:3300/0 conn(0x7f5fec06eec0 0x7f5fec0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2324877005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec070640 0x7f5fec06e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2324877005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec15f880 0x7f5fec0b4ff0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2324877005 >> 172.21.15.43:0/2324877005 conn(0x7f5fec0b3cf0 msgr2=0x7f5fec0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2324877005 shutdown_connections 2024-03-30T13:11:48.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2324877005 wait complete. 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.003+0000 7f5ff0b3c640 1 Processor -- start 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 -- start start 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec06eec0 0x7f5fec153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec070640 0x7f5fec06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fec15f880 0x7f5fec152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1609d0 con 0x7f5fec070640 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec160850 con 0x7f5fec15f880 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5fec0d0000 con 0x7f5fec06eec0 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec070640 0x7f5fec06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec070640 0x7f5fec06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46308/0 (socket says 172.21.15.43:46308) 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 -- 172.21.15.43:0/2967835935 learned_addr learned my addr 172.21.15.43:0/2967835935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fea59c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec06eec0 0x7f5fec153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fead9d640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fec15f880 0x7f5fec152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 -- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec06eec0 msgr2=0x7f5fec153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec06eec0 0x7f5fec153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 -- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fec15f880 msgr2=0x7f5fec152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fec15f880 0x7f5fec152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 -- 172.21.15.43:0/2967835935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec06e760 con 0x7f5fec070640 2024-03-30T13:11:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fe9d9b640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec070640 0x7f5fec06db20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c7f0 tx=0x7f5fe000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fdaffd640 1 -- 172.21.15.43:0/2967835935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0015070 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fdaffd640 1 -- 172.21.15.43:0/2967835935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe0007820 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec06e1a0 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fdaffd640 1 -- 172.21.15.43:0/2967835935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0007c50 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec165380 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fec068900 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fdaffd640 1 -- 172.21.15.43:0/2967835935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5fe0018070 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fdaffd640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fcc067fc0 0x7f5fcc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.007+0000 7f5fdaffd640 1 -- 172.21.15.43:0/2967835935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f5fe00868d0 con 0x7f5fec070640 2024-03-30T13:11:48.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.011+0000 7f5fea59c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fcc067fc0 0x7f5fcc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.011+0000 7f5fea59c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fcc067fc0 0x7f5fcc06a480 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ad000 tx=0x7f5fdc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.015+0000 7f5fdaffd640 1 -- 172.21.15.43:0/2967835935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5fe0056da0 con 0x7f5fec070640 2024-03-30T13:11:48.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5fec0b7ff0 con 0x7f5fec070640 2024-03-30T13:11:48.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5fdaffd640 1 -- 172.21.15.43:0/2967835935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f5fe00566a0 con 0x7f5fec070640 2024-03-30T13:11:48.186 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fcc067fc0 msgr2=0x7f5fcc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fcc067fc0 0x7f5fcc06a480 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc0ad000 tx=0x7f5fdc008040 comp rx=0 tx=0).stop 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec070640 msgr2=0x7f5fec06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec070640 0x7f5fec06db20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5fe000c7f0 tx=0x7f5fe000ccb0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 shutdown_connections 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fcc067fc0 0x7f5fcc06a480 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fec15f880 0x7f5fec152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fec070640 0x7f5fec06db20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 --2- 172.21.15.43:0/2967835935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5fec06eec0 0x7f5fec153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.179+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 >> 172.21.15.43:0/2967835935 conn(0x7f5fec0b3cf0 msgr2=0x7f5fec0b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.183+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 shutdown_connections 2024-03-30T13:11:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.183+0000 7f5ff0b3c640 1 -- 172.21.15.43:0/2967835935 wait complete. 2024-03-30T13:11:48.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:48.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 Processor -- start 2024-03-30T13:11:48.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 -- start start 2024-03-30T13:11:48.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f09e41450f0 0x7f09e415ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09e4160440 0x7f09e40cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f09e40d0b50 con 0x7f09e4160440 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f09e40d0e30 con 0x7f09e41450f0 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09eaed6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f09e406b700 con 0x7f09e40d02d0 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09e4160440 0x7f09e40cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09e4160440 0x7f09e40cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46320/0 (socket says 172.21.15.43:46320) 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 -- 172.21.15.43:0/1423254279 learned_addr learned my addr 172.21.15.43:0/1423254279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e8c72640 1 --2- 172.21.15.43:0/1423254279 >> v2:172.21.15.113:3300/0 conn(0x7f09e41450f0 0x7f09e415ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e9473640 1 --2- 172.21.15.43:0/1423254279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 -- 172.21.15.43:0/1423254279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 msgr2=0x7f09e406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 --2- 172.21.15.43:0/1423254279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 -- 172.21.15.43:0/1423254279 >> v2:172.21.15.113:3300/0 conn(0x7f09e41450f0 msgr2=0x7f09e415ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:48.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 --2- 172.21.15.43:0/1423254279 >> v2:172.21.15.113:3300/0 conn(0x7f09e41450f0 0x7f09e415ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e3fff640 1 -- 172.21.15.43:0/1423254279 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e406b9e0 con 0x7f09e4160440 2024-03-30T13:11:48.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.319+0000 7f09e9473640 1 --2- 172.21.15.43:0/1423254279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:48.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09e3fff640 1 --2- 172.21.15.43:0/1423254279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e4160440 0x7f09e40cfd90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009870 tx=0x7f09d801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e19111e0703ba62 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09e37fe640 1 -- 172.21.15.43:0/1423254279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80b0070 con 0x7f09e4160440 2024-03-30T13:11:48.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09e37fe640 1 -- 172.21.15.43:0/1423254279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e41456e0 con 0x7f09e4160440 2024-03-30T13:11:48.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09e37fe640 1 -- 172.21.15.43:0/1423254279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80ab820 con 0x7f09e4160440 2024-03-30T13:11:48.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 -- 172.21.15.43:0/1423254279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e4160440 msgr2=0x7f09e40cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 --2- 172.21.15.43:0/1423254279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e4160440 0x7f09e40cfd90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009870 tx=0x7f09d801c260 comp rx=0 tx=0).stop 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 -- 172.21.15.43:0/1423254279 shutdown_connections 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 --2- 172.21.15.43:0/1423254279 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 --2- 172.21.15.43:0/1423254279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e4160440 0x7f09e40cfd90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 --2- 172.21.15.43:0/1423254279 >> v2:172.21.15.113:3300/0 conn(0x7f09e41450f0 0x7f09e415ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 -- 172.21.15.43:0/1423254279 >> 172.21.15.43:0/1423254279 conn(0x7f09e40d1070 msgr2=0x7f09e40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 -- 172.21.15.43:0/1423254279 shutdown_connections 2024-03-30T13:11:48.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.323+0000 7f09eaed6640 1 -- 172.21.15.43:0/1423254279 wait complete. 2024-03-30T13:11:48.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 Processor -- start 2024-03-30T13:11:48.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 -- start start 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e41450f0 0x7f09e415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e4160440 0x7f09e4161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f09e406c1e0 con 0x7f09e41450f0 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f09e40d08f0 con 0x7f09e4160440 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f09e406c360 con 0x7f09e40d02d0 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e41450f0 0x7f09e415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e41450f0 0x7f09e415e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46330/0 (socket says 172.21.15.43:46330) 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 -- 172.21.15.43:0/630607278 learned_addr learned my addr 172.21.15.43:0/630607278 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e8c72640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e9473640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e4160440 0x7f09e4161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 -- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 msgr2=0x7f09e415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e415fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 -- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e4160440 msgr2=0x7f09e4161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e4160440 0x7f09e4161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 -- 172.21.15.43:0/630607278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e415edb0 con 0x7f09e41450f0 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e3fff640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e41450f0 0x7f09e415e6b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f09d801cb00 tx=0x7f09d8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e17fa640 1 -- 172.21.15.43:0/630607278 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80ba020 con 0x7f09e41450f0 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e415f090 con 0x7f09e41450f0 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e415f650 con 0x7f09e41450f0 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e17fa640 1 -- 172.21.15.43:0/630607278 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d80b6420 con 0x7f09e41450f0 2024-03-30T13:11:48.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e17fa640 1 -- 172.21.15.43:0/630607278 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80b0040 con 0x7f09e41450f0 2024-03-30T13:11:48.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c40052a0 con 0x7f09e41450f0 2024-03-30T13:11:48.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.327+0000 7f09e17fa640 1 -- 172.21.15.43:0/630607278 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f09d80bd020 con 0x7f09e41450f0 2024-03-30T13:11:48.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.331+0000 7f09e17fa640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f09d0067fc0 0x7f09d006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.331+0000 7f09e17fa640 1 -- 172.21.15.43:0/630607278 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f09d8120a50 con 0x7f09e41450f0 2024-03-30T13:11:48.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.331+0000 7f09e8c72640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f09d0067fc0 0x7f09d006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.331+0000 7f09e8c72640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f09d0067fc0 0x7f09d006a480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09e413c600 tx=0x7f09d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.335+0000 7f09e17fa640 1 -- 172.21.15.43:0/630607278 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f09d80c4030 con 0x7f09e41450f0 2024-03-30T13:11:48.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.495+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f09c4005860 con 0x7f09e41450f0 2024-03-30T13:11:48.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.495+0000 7f09e17fa640 1 -- 172.21.15.43:0/630607278 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f09d8016110 con 0x7f09e41450f0 2024-03-30T13:11:48.504 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:48.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f09d0067fc0 msgr2=0x7f09d006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f09d0067fc0 0x7f09d006a480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f09e413c600 tx=0x7f09d4008040 comp rx=0 tx=0).stop 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e41450f0 msgr2=0x7f09e415e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e41450f0 0x7f09e415e6b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f09d801cb00 tx=0x7f09d8019000 comp rx=0 tx=0).stop 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 shutdown_connections 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f09d0067fc0 0x7f09d006a480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f09e4160440 0x7f09e4161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09e41450f0 0x7f09e415e6b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 --2- 172.21.15.43:0/630607278 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f09e40d02d0 0x7f09e415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 >> 172.21.15.43:0/630607278 conn(0x7f09e40d1070 msgr2=0x7f09e40b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 shutdown_connections 2024-03-30T13:11:48.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.499+0000 7f09eaed6640 1 -- 172.21.15.43:0/630607278 wait complete. 2024-03-30T13:11:48.521 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:48.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.627+0000 7fbf33a50640 1 Processor -- start 2024-03-30T13:11:48.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.627+0000 7fbf33a50640 1 -- start start 2024-03-30T13:11:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf33a50640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbf2c06f9e0 0x7fbf2c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf33a50640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf2c0d0bc0 0x7fbf2c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf33a50640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf2c15d190 con 0x7fbf2c06cfe0 2024-03-30T13:11:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf33a50640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c06cb50 con 0x7fbf2c0d0bc0 2024-03-30T13:11:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf33a50640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbf2c06bf40 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf2c0d0bc0 0x7fbf2c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf2c0d0bc0 0x7fbf2c15c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46346/0 (socket says 172.21.15.43:46346) 2024-03-30T13:11:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 -- 172.21.15.43:0/3673204052 learned_addr learned my addr 172.21.15.43:0/3673204052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf317ec640 1 --2- 172.21.15.43:0/3673204052 >> v2:172.21.15.113:3300/0 conn(0x7fbf2c06f9e0 0x7fbf2c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 -- 172.21.15.43:0/3673204052 >> v2:172.21.15.113:3300/0 conn(0x7fbf2c06f9e0 msgr2=0x7fbf2c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 --2- 172.21.15.43:0/3673204052 >> v2:172.21.15.113:3300/0 conn(0x7fbf2c06f9e0 0x7fbf2c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 -- 172.21.15.43:0/3673204052 >> v1:172.21.15.113:6789/0 conn(0x7fbf2c06cfe0 legacy=0x7fbf2c06fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 -- 172.21.15.43:0/3673204052 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c06c220 con 0x7fbf2c0d0bc0 2024-03-30T13:11:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf317ec640 1 --2- 172.21.15.43:0/3673204052 >> v2:172.21.15.113:3300/0 conn(0x7fbf2c06f9e0 0x7fbf2c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.631+0000 7fbf30feb640 1 --2- 172.21.15.43:0/3673204052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c15c860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0d2170 tx=0x7fbf1c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab46688889dacdd5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf23fff640 1 -- 172.21.15.43:0/3673204052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0b1070 con 0x7fbf2c0d0bc0 2024-03-30T13:11:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf23fff640 1 -- 172.21.15.43:0/3673204052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0ac620 con 0x7fbf2c0d0bc0 2024-03-30T13:11:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf23fff640 1 -- 172.21.15.43:0/3673204052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0ac9f0 con 0x7fbf2c0d0bc0 2024-03-30T13:11:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- 172.21.15.43:0/3673204052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c0d0bc0 msgr2=0x7fbf2c15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 --2- 172.21.15.43:0/3673204052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c15c860 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0d2170 tx=0x7fbf1c01c4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- 172.21.15.43:0/3673204052 shutdown_connections 2024-03-30T13:11:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 --2- 172.21.15.43:0/3673204052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c15c860 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 --2- 172.21.15.43:0/3673204052 >> v2:172.21.15.113:3300/0 conn(0x7fbf2c06f9e0 0x7fbf2c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- 172.21.15.43:0/3673204052 >> 172.21.15.43:0/3673204052 conn(0x7fbf2c0d1070 msgr2=0x7fbf2c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- 172.21.15.43:0/3673204052 shutdown_connections 2024-03-30T13:11:48.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- 172.21.15.43:0/3673204052 wait complete. 2024-03-30T13:11:48.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 Processor -- start 2024-03-30T13:11:48.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- start start 2024-03-30T13:11:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf2c06cfe0 0x7fbf2c1516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06f9e0 0x7fbf2c0aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c15cf20 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c15cda0 con 0x7fbf2c0d0bc0 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf33a50640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c1696b0 con 0x7fbf2c06cfe0 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06f9e0 0x7fbf2c0aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06f9e0 0x7fbf2c0aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46358/0 (socket says 172.21.15.43:46358) 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 -- 172.21.15.43:0/4054240918 learned_addr learned my addr 172.21.15.43:0/4054240918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf317ec640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf2c06cfe0 0x7fbf2c1516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf31fed640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 -- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf2c06cfe0 msgr2=0x7fbf2c1516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf2c06cfe0 0x7fbf2c1516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 -- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf2c0d0bc0 msgr2=0x7fbf2c0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c0af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf30feb640 1 -- 172.21.15.43:0/4054240918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c1fa8c0 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.635+0000 7fbf31fed640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf30feb640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06f9e0 0x7fbf2c0aebd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0af040 tx=0x7fbf1c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf21ffb640 1 -- 172.21.15.43:0/4054240918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0bb020 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf21ffb640 1 -- 172.21.15.43:0/4054240918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf1c0b7620 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c1faba0 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf21ffb640 1 -- 172.21.15.43:0/4054240918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0b1040 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c152ff0 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c068900 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf21ffb640 1 -- 172.21.15.43:0/4054240918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbf1c0be020 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf21ffb640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbf10067fc0 0x7fbf1006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf21ffb640 1 -- 172.21.15.43:0/4054240918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fbf1c121500 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf317ec640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbf10067fc0 0x7fbf1006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.639+0000 7fbf317ec640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbf10067fc0 0x7fbf1006a480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0af720 tx=0x7fbf28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.643+0000 7fbf21ffb640 1 -- 172.21.15.43:0/4054240918 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbf1c0c5030 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.803+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbf2c074df0 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.803+0000 7fbf21ffb640 1 -- 172.21.15.43:0/4054240918 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fbf1c0160d0 con 0x7fbf2c06f9e0 2024-03-30T13:11:48.812 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbf10067fc0 msgr2=0x7fbf1006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbf10067fc0 0x7fbf1006a480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0af720 tx=0x7fbf28008040 comp rx=0 tx=0).stop 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06f9e0 msgr2=0x7fbf2c0aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06f9e0 0x7fbf2c0aebd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0af040 tx=0x7fbf1c019250 comp rx=0 tx=0).stop 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 shutdown_connections 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbf10067fc0 0x7fbf1006a480 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbf2c0d0bc0 0x7fbf2c0af110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf2c06f9e0 0x7fbf2c0aebd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 --2- 172.21.15.43:0/4054240918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbf2c06cfe0 0x7fbf2c1516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 >> 172.21.15.43:0/4054240918 conn(0x7fbf2c0d1070 msgr2=0x7fbf2c06fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:48.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 shutdown_connections 2024-03-30T13:11:48.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.807+0000 7fbf33a50640 1 -- 172.21.15.43:0/4054240918 wait complete. 2024-03-30T13:11:48.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:48.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00df738640 1 Processor -- start 2024-03-30T13:11:48.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00df738640 1 -- start start 2024-03-30T13:11:48.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00df738640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f00d813e820 0x7f00d806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00df738640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d8141200 con 0x7f00d815ec40 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00df738640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00d806fd10 con 0x7f00d815f730 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00df738640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f00d806fff0 con 0x7f00d813e820 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00dccd3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f00d813e820 0x7f00d806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00dd4d4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f00d815ec40 0x7f00d815f020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60582/0 (socket says 172.21.15.43:60582) 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00dd4d4640 1 -- 172.21.15.43:0/892174606 learned_addr learned my addr 172.21.15.43:0/892174606 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00dccd3640 1 -- 172.21.15.43:0/892174606 >> v1:172.21.15.113:6789/0 conn(0x7f00d815f730 legacy=0x7f00d806f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00dccd3640 1 -- 172.21.15.43:0/892174606 >> v1:172.21.15.43:6789/0 conn(0x7f00d815ec40 legacy=0x7f00d815f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.931+0000 7f00dccd3640 1 -- 172.21.15.43:0/892174606 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d806c770 con 0x7f00d813e820 2024-03-30T13:11:48.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00dccd3640 1 --2- 172.21.15.43:0/892174606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d813e820 0x7f00d806ef90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f00c80097c0 tx=0x7f00c801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e2c393cba4597a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00cffff640 1 -- 172.21.15.43:0/892174606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c80b1070 con 0x7f00d813e820 2024-03-30T13:11:48.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00cffff640 1 -- 172.21.15.43:0/892174606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c80ac600 con 0x7f00d813e820 2024-03-30T13:11:48.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00cffff640 1 -- 172.21.15.43:0/892174606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c80ac9d0 con 0x7f00d813e820 2024-03-30T13:11:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 -- 172.21.15.43:0/892174606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d813e820 msgr2=0x7f00d806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 --2- 172.21.15.43:0/892174606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d813e820 0x7f00d806ef90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f00c80097c0 tx=0x7f00c801c3a0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 -- 172.21.15.43:0/892174606 shutdown_connections 2024-03-30T13:11:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 --2- 172.21.15.43:0/892174606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d813e820 0x7f00d806ef90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 -- 172.21.15.43:0/892174606 >> 172.21.15.43:0/892174606 conn(0x7f00d80b2c70 msgr2=0x7f00d80b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 -- 172.21.15.43:0/892174606 shutdown_connections 2024-03-30T13:11:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 -- 172.21.15.43:0/892174606 wait complete. 2024-03-30T13:11:48.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 Processor -- start 2024-03-30T13:11:48.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.935+0000 7f00df738640 1 -- start start 2024-03-30T13:11:48.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00d813e820 0x7f00d81587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d815ec40 0x7f00d813c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00d815f730 0x7f00d81f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f00d8168970 con 0x7f00d815ec40 2024-03-30T13:11:48.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f00d81687f0 con 0x7f00d815f730 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f00d8168af0 con 0x7f00d813e820 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d815ec40 0x7f00d813c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d815ec40 0x7f00d813c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46376/0 (socket says 172.21.15.43:46376) 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 -- 172.21.15.43:0/91015331 learned_addr learned my addr 172.21.15.43:0/91015331 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00ddcd5640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00d815f730 0x7f00d81f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dd4d4640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00d813e820 0x7f00d81587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 -- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00d813e820 msgr2=0x7f00d81587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00d813e820 0x7f00d81587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 -- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00d815f730 msgr2=0x7f00d81f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00d815f730 0x7f00d81f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 -- 172.21.15.43:0/91015331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d813c9d0 con 0x7f00d815ec40 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dd4d4640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00d813e820 0x7f00d81587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00ddcd5640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00d815f730 0x7f00d81f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:48.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dccd3640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d815ec40 0x7f00d813c490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00c8018fb0 tx=0x7f00c801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00cdffb640 1 -- 172.21.15.43:0/91015331 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c80b1040 con 0x7f00d815ec40 2024-03-30T13:11:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00cdffb640 1 -- 172.21.15.43:0/91015331 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c80bc070 con 0x7f00d815ec40 2024-03-30T13:11:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d81f56a0 con 0x7f00d815ec40 2024-03-30T13:11:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00cdffb640 1 -- 172.21.15.43:0/91015331 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c80b7890 con 0x7f00d815ec40 2024-03-30T13:11:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d81f5c60 con 0x7f00d815ec40 2024-03-30T13:11:48.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d8068890 con 0x7f00d815ec40 2024-03-30T13:11:48.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00cdffb640 1 -- 172.21.15.43:0/91015331 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f00c80ac4a0 con 0x7f00d815ec40 2024-03-30T13:11:48.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00cdffb640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f00bc067fc0 0x7f00bc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:48.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00cdffb640 1 -- 172.21.15.43:0/91015331 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f00c8121790 con 0x7f00d815ec40 2024-03-30T13:11:48.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.939+0000 7f00dd4d4640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f00bc067fc0 0x7f00bc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:48.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.943+0000 7f00dd4d4640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f00bc067fc0 0x7f00bc06a480 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00d815e930 tx=0x7f00d4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:48.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:48.943+0000 7f00cdffb640 1 -- 172.21.15.43:0/91015331 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f00d813c9d0 con 0x7f00d815ec40 2024-03-30T13:11:49.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.099+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00d815c390 con 0x7f00d815ec40 2024-03-30T13:11:49.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.099+0000 7f00cdffb640 1 -- 172.21.15.43:0/91015331 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7f00c8016050 con 0x7f00d815ec40 2024-03-30T13:11:49.108 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:49.108 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:49.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f00bc067fc0 msgr2=0x7f00bc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f00bc067fc0 0x7f00bc06a480 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00d815e930 tx=0x7f00d4009040 comp rx=0 tx=0).stop 2024-03-30T13:11:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d815ec40 msgr2=0x7f00d813c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d815ec40 0x7f00d813c490 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f00c8018fb0 tx=0x7f00c801abe0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 shutdown_connections 2024-03-30T13:11:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f00bc067fc0 0x7f00bc06a480 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f00d815f730 0x7f00d81f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00d815ec40 0x7f00d813c490 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 --2- 172.21.15.43:0/91015331 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f00d813e820 0x7f00d81587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 >> 172.21.15.43:0/91015331 conn(0x7f00d80b2c70 msgr2=0x7f00d80b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 shutdown_connections 2024-03-30T13:11:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.103+0000 7f00df738640 1 -- 172.21.15.43:0/91015331 wait complete. 2024-03-30T13:11:49.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:49.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:49.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.235+0000 7f97c8eec640 1 Processor -- start 2024-03-30T13:11:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.235+0000 7f97c8eec640 1 -- start start 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c8eec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f97c415c160 0x7f97c406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c8eec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 0x7f97c4160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c8eec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c4160c80 con 0x7f97c40ccaa0 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c8eec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f97c41607d0 con 0x7f97c415c160 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c8eec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f97c40cd680 con 0x7f97c415fe20 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f97c40ccaa0 0x7f97c40cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60596/0 (socket says 172.21.15.43:60596) 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c2d9d640 1 -- 172.21.15.43:0/1338686239 learned_addr learned my addr 172.21.15.43:0/1338686239 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510434216 0 0) 0x7f97c4160c80 con 0x7f97c40ccaa0 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97b0003530 con 0x7f97c40ccaa0 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 --2- 172.21.15.43:0/1338686239 >> v2:172.21.15.113:3300/0 conn(0x7f97c415c160 0x7f97c406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97b9d9b640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 0x7f97c4160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 120555451 0 0) 0x7f97b0003530 con 0x7f97c40ccaa0 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 msgr2=0x7f97c4160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 0x7f97c4160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 >> v2:172.21.15.113:3300/0 conn(0x7f97c415c160 msgr2=0x7f97c406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 --2- 172.21.15.43:0/1338686239 >> v2:172.21.15.113:3300/0 conn(0x7f97c415c160 0x7f97c406b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c40cd960 con 0x7f97c40ccaa0 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f97bc001a00 con 0x7f97c40ccaa0 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 >> v1:172.21.15.43:6789/0 conn(0x7f97c40ccaa0 legacy=0x7f97c40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b00039c0 0x7f97b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 0x7f97b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b0004d50 0x7f97b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97c41607d0 con 0x7f97b00039c0 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97c40cd680 con 0x7f97b0004d50 2024-03-30T13:11:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f97b000bab0 con 0x7f97c415fe20 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 --2- 172.21.15.43:0/1338686239 >> v2:172.21.15.113:3300/0 conn(0x7f97c415c160 0x7f97c406b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b00039c0 0x7f97b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c2d9d640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b0004d50 0x7f97b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97b9d9b640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 0x7f97b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 -- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 msgr2=0x7f97b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97c415fe20 0x7f97b0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 -- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b0004d50 msgr2=0x7f97b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97b0004d50 0x7f97b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.239+0000 7f97c259c640 1 -- 172.21.15.43:0/1338686239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b0003530 con 0x7f97b00039c0 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c259c640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b00039c0 0x7f97b0003de0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97b40098b0 tx=0x7f97b401c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8132d2c45ac427e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b400f040 con 0x7f97b00039c0 2024-03-30T13:11:49.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b4004220 con 0x7f97b00039c0 2024-03-30T13:11:49.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c1d9b640 1 -- 172.21.15.43:0/1338686239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4003620 con 0x7f97b00039c0 2024-03-30T13:11:49.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- 172.21.15.43:0/1338686239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b00039c0 msgr2=0x7f97b0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 --2- 172.21.15.43:0/1338686239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97b00039c0 0x7f97b0003de0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f97b40098b0 tx=0x7f97b401c5a0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c2d9d640 1 -- 172.21.15.43:0/1338686239 reap_dead start 2024-03-30T13:11:49.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- 172.21.15.43:0/1338686239 shutdown_connections 2024-03-30T13:11:49.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- 172.21.15.43:0/1338686239 >> 172.21.15.43:0/1338686239 conn(0x7f97c40d0c90 msgr2=0x7f97c40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:49.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- 172.21.15.43:0/1338686239 shutdown_connections 2024-03-30T13:11:49.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- 172.21.15.43:0/1338686239 wait complete. 2024-03-30T13:11:49.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 Processor -- start 2024-03-30T13:11:49.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- start start 2024-03-30T13:11:49.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97b00039c0 0x7f97c4142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c41f89d0 0x7f97c41fb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c4151940 0x7f97c415d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f97c415d550 con 0x7f97c4151940 2024-03-30T13:11:49.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f97c415d6d0 con 0x7f97c41f89d0 2024-03-30T13:11:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c8eec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f97c40d0390 con 0x7f97b00039c0 2024-03-30T13:11:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c4151940 0x7f97c415d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c4151940 0x7f97c415d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46380/0 (socket says 172.21.15.43:46380) 2024-03-30T13:11:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.243+0000 7f97c2d9d640 1 -- 172.21.15.43:0/3539944271 learned_addr learned my addr 172.21.15.43:0/3539944271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97b9d9b640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c41f89d0 0x7f97c41fb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c259c640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97b00039c0 0x7f97c4142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c2d9d640 1 -- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97b00039c0 msgr2=0x7f97c4142330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c2d9d640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97b00039c0 0x7f97c4142330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c2d9d640 1 -- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c41f89d0 msgr2=0x7f97c41fb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c2d9d640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c41f89d0 0x7f97c41fb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c2d9d640 1 -- 172.21.15.43:0/3539944271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c41fbc70 con 0x7f97c4151940 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c259c640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97b00039c0 0x7f97c4142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c2d9d640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c4151940 0x7f97c415d8b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f97bc002240 tx=0x7f97bc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97bb7fe640 1 -- 172.21.15.43:0/3539944271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00eb90 con 0x7f97c4151940 2024-03-30T13:11:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97bb7fe640 1 -- 172.21.15.43:0/3539944271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc018070 con 0x7f97c4151940 2024-03-30T13:11:49.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c41fbf50 con 0x7f97c4151940 2024-03-30T13:11:49.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97bb7fe640 1 -- 172.21.15.43:0/3539944271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc013420 con 0x7f97c4151940 2024-03-30T13:11:49.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c41fb840 con 0x7f97c4151940 2024-03-30T13:11:49.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c4068900 con 0x7f97c4151940 2024-03-30T13:11:49.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97bb7fe640 1 -- 172.21.15.43:0/3539944271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f97bc0135c0 con 0x7f97c4151940 2024-03-30T13:11:49.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97bb7fe640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f97a4067ff0 0x7f97a406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97bb7fe640 1 -- 172.21.15.43:0/3539944271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f97bc086a90 con 0x7f97c4151940 2024-03-30T13:11:49.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c259c640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f97a4067ff0 0x7f97a406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.247+0000 7f97c259c640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f97a4067ff0 0x7f97a406a4b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97b40ae000 tx=0x7f97b40aa040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.251+0000 7f97bb7fe640 1 -- 172.21.15.43:0/3539944271 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f97bc056f60 con 0x7f97c4151940 2024-03-30T13:11:49.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f97c406f1c0 con 0x7f97c4151940 2024-03-30T13:11:49.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97bb7fe640 1 -- 172.21.15.43:0/3539944271 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f97bc056860 con 0x7f97c4151940 2024-03-30T13:11:49.418 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f97a4067ff0 msgr2=0x7f97a406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f97a4067ff0 0x7f97a406a4b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97b40ae000 tx=0x7f97b40aa040 comp rx=0 tx=0).stop 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c4151940 msgr2=0x7f97c415d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c4151940 0x7f97c415d8b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f97bc002240 tx=0x7f97bc00e530 comp rx=0 tx=0).stop 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 shutdown_connections 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f97a4067ff0 0x7f97a406a4b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97c4151940 0x7f97c415d8b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f97c41f89d0 0x7f97c41fb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 --2- 172.21.15.43:0/3539944271 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f97b00039c0 0x7f97c4142330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 >> 172.21.15.43:0/3539944271 conn(0x7f97c40d0c90 msgr2=0x7f97c4161570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:49.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 shutdown_connections 2024-03-30T13:11:49.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.411+0000 7f97c8eec640 1 -- 172.21.15.43:0/3539944271 wait complete. 2024-03-30T13:11:49.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:49.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c8df0e640 1 Processor -- start 2024-03-30T13:11:49.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c8df0e640 1 -- start start 2024-03-30T13:11:49.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c8df0e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c880b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c8df0e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c880b4520 0x7f9c8806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c8df0e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c880b4ba0 con 0x7f9c8806ff20 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c8df0e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9c88070c30 con 0x7f9c880b4520 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c8df0e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9c880b4960 con 0x7f9c8815afc0 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c86ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c880b4520 0x7f9c8806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c87fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9c8806ff20 0x7f9c8806c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60608/0 (socket says 172.21.15.43:60608) 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c87fff640 1 -- 172.21.15.43:0/2268273601 learned_addr learned my addr 172.21.15.43:0/2268273601 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c877fe640 1 --2- 172.21.15.43:0/2268273601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c880b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c86ffd640 1 -- 172.21.15.43:0/2268273601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 msgr2=0x7f9c880b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c86ffd640 1 --2- 172.21.15.43:0/2268273601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c880b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c86ffd640 1 -- 172.21.15.43:0/2268273601 >> v1:172.21.15.43:6789/0 conn(0x7f9c8806ff20 legacy=0x7f9c8806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c86ffd640 1 -- 172.21.15.43:0/2268273601 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8815ec60 con 0x7f9c880b4520 2024-03-30T13:11:49.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c877fe640 1 --2- 172.21.15.43:0/2268273601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c880b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:49.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.555+0000 7f9c86ffd640 1 --2- 172.21.15.43:0/2268273601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c8806f9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c700097c0 tx=0x7f9c7001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d526f6ba8d8d8f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c867fc640 1 -- 172.21.15.43:0/2268273601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700b0070 con 0x7f9c880b4520 2024-03-30T13:11:49.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c867fc640 1 -- 172.21.15.43:0/2268273601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c700ab600 con 0x7f9c880b4520 2024-03-30T13:11:49.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c867fc640 1 -- 172.21.15.43:0/2268273601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700ab9d0 con 0x7f9c880b4520 2024-03-30T13:11:49.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- 172.21.15.43:0/2268273601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 msgr2=0x7f9c8806f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/2268273601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c8806f9e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c700097c0 tx=0x7f9c7001c4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- 172.21.15.43:0/2268273601 shutdown_connections 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/2268273601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c8806f9e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/2268273601 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c880b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- 172.21.15.43:0/2268273601 >> 172.21.15.43:0/2268273601 conn(0x7f9c880d1070 msgr2=0x7f9c880d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- 172.21.15.43:0/2268273601 shutdown_connections 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- 172.21.15.43:0/2268273601 wait complete. 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 Processor -- start 2024-03-30T13:11:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- start start 2024-03-30T13:11:49.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8806ff20 0x7f9c881440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c88157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c88159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8815fa20 con 0x7f9c880b4520 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c880cf7c0 con 0x7f9c8806ff20 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9c8814e1f0 con 0x7f9c8815afc0 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c88157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c88157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46402/0 (socket says 172.21.15.43:46402) 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 -- 172.21.15.43:0/3385424893 learned_addr learned my addr 172.21.15.43:0/3385424893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c877fe640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8806ff20 0x7f9c881440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 -- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 msgr2=0x7f9c88159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c87fff640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c88159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c88159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 -- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8806ff20 msgr2=0x7f9c881440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8806ff20 0x7f9c881440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 -- 172.21.15.43:0/3385424893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8815d4f0 con 0x7f9c880b4520 2024-03-30T13:11:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c877fe640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8806ff20 0x7f9c881440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c86ffd640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c88157da0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c700ae040 tx=0x7f9c70019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c7ffff640 1 -- 172.21.15.43:0/3385424893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700b0040 con 0x7f9c880b4520 2024-03-30T13:11:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c7ffff640 1 -- 172.21.15.43:0/3385424893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c700b4410 con 0x7f9c880b4520 2024-03-30T13:11:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c7ffff640 1 -- 172.21.15.43:0/3385424893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700b47e0 con 0x7f9c880b4520 2024-03-30T13:11:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8815d7d0 con 0x7f9c880b4520 2024-03-30T13:11:49.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.559+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8814de80 con 0x7f9c880b4520 2024-03-30T13:11:49.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.563+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c88068900 con 0x7f9c880b4520 2024-03-30T13:11:49.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.563+0000 7f9c7ffff640 1 -- 172.21.15.43:0/3385424893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9c700bb030 con 0x7f9c880b4520 2024-03-30T13:11:49.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.563+0000 7f9c7ffff640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067fc0 0x7f9c6806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.563+0000 7f9c7ffff640 1 -- 172.21.15.43:0/3385424893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f9c7011fc60 con 0x7f9c880b4520 2024-03-30T13:11:49.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.563+0000 7f9c877fe640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067fc0 0x7f9c6806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.563+0000 7f9c877fe640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067fc0 0x7f9c6806a480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c78007f40 tx=0x7f9c78007490 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.567+0000 7f9c7ffff640 1 -- 172.21.15.43:0/3385424893 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9c700c2060 con 0x7f9c880b4520 2024-03-30T13:11:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.783+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c880706c0 con 0x7f9c880b4520 2024-03-30T13:11:49.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.783+0000 7f9c7ffff640 1 -- 172.21.15.43:0/3385424893 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f9c70016090 con 0x7f9c880b4520 2024-03-30T13:11:49.791 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.783+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067fc0 msgr2=0x7f9c6806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.783+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067fc0 0x7f9c6806a480 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c78007f40 tx=0x7f9c78007490 comp rx=0 tx=0).stop 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 msgr2=0x7f9c88157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c88157da0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9c700ae040 tx=0x7f9c70019240 comp rx=0 tx=0).stop 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 shutdown_connections 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067fc0 0x7f9c6806a480 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c8815afc0 0x7f9c88159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c880b4520 0x7f9c88157da0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 --2- 172.21.15.43:0/3385424893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c8806ff20 0x7f9c881440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 >> 172.21.15.43:0/3385424893 conn(0x7f9c880d1070 msgr2=0x7f9c88073030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:49.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 shutdown_connections 2024-03-30T13:11:49.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.787+0000 7f9c8df0e640 1 -- 172.21.15.43:0/3385424893 wait complete. 2024-03-30T13:11:49.814 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:49.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.923+0000 7f0f24f69640 1 Processor -- start 2024-03-30T13:11:49.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.923+0000 7f0f24f69640 1 -- start start 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f24f69640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f2015f880 0x7f0f200b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f24f69640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f20070640 0x7f0f2006e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f24f69640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f20070aa0 con 0x7f0f2006eec0 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f24f69640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0f200ce4d0 con 0x7f0f2015f880 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f24f69640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0f200cc7f0 con 0x7f0f20070640 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1dd9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f20070640 0x7f0f2006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1e59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f2015f880 0x7f0f200b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1ed9d640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f0f2006eec0 0x7f0f200ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:47472/0 (socket says 172.21.15.43:47472) 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1ed9d640 1 -- 172.21.15.43:0/769743804 learned_addr learned my addr 172.21.15.43:0/769743804 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1d59a640 1 -- 172.21.15.43:0/769743804 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1738072681 0 0) 0x7f0f20070aa0 con 0x7f0f2006eec0 2024-03-30T13:11:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1d59a640 1 -- 172.21.15.43:0/769743804 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f08003530 con 0x7f0f2006eec0 2024-03-30T13:11:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1dd9b640 1 -- 172.21.15.43:0/769743804 >> v2:172.21.15.113:3300/0 conn(0x7f0f2015f880 msgr2=0x7f0f200b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1dd9b640 1 --2- 172.21.15.43:0/769743804 >> v2:172.21.15.113:3300/0 conn(0x7f0f2015f880 0x7f0f200b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1dd9b640 1 -- 172.21.15.43:0/769743804 >> v1:172.21.15.113:6789/0 conn(0x7f0f2006eec0 legacy=0x7f0f200ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1dd9b640 1 -- 172.21.15.43:0/769743804 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f200ccad0 con 0x7f0f20070640 2024-03-30T13:11:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.927+0000 7f0f1e59c640 1 --2- 172.21.15.43:0/769743804 >> v2:172.21.15.113:3300/0 conn(0x7f0f2015f880 0x7f0f200b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:11:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 --2- 172.21.15.43:0/769743804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f20070640 0x7f0f2006e980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f140097c0 tx=0x7f0f1401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=710a451dee5810de server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1d59a640 1 -- 172.21.15.43:0/769743804 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140b0070 con 0x7f0f20070640 2024-03-30T13:11:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1d59a640 1 -- 172.21.15.43:0/769743804 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f140ab600 con 0x7f0f20070640 2024-03-30T13:11:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1d59a640 1 -- 172.21.15.43:0/769743804 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140ab9b0 con 0x7f0f20070640 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- 172.21.15.43:0/769743804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f20070640 msgr2=0x7f0f2006e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 --2- 172.21.15.43:0/769743804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f20070640 0x7f0f2006e980 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f140097c0 tx=0x7f0f1401c380 comp rx=0 tx=0).stop 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- 172.21.15.43:0/769743804 shutdown_connections 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 --2- 172.21.15.43:0/769743804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f20070640 0x7f0f2006e980 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 --2- 172.21.15.43:0/769743804 >> v2:172.21.15.113:3300/0 conn(0x7f0f2015f880 0x7f0f200b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- 172.21.15.43:0/769743804 >> 172.21.15.43:0/769743804 conn(0x7f0f200b3cf0 msgr2=0x7f0f200b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- 172.21.15.43:0/769743804 shutdown_connections 2024-03-30T13:11:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- 172.21.15.43:0/769743804 wait complete. 2024-03-30T13:11:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 Processor -- start 2024-03-30T13:11:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- start start 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2006eec0 0x7f0f200cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20070640 0x7f0f2015b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f2015f880 0x7f0f201f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f200cf650 con 0x7f0f20070640 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f200cf4d0 con 0x7f0f2006eec0 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f24f69640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0f20160850 con 0x7f0f2015f880 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20070640 0x7f0f2015b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20070640 0x7f0f2015b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46408/0 (socket says 172.21.15.43:46408) 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 -- 172.21.15.43:0/442771013 learned_addr learned my addr 172.21.15.43:0/442771013 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1e59c640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2006eec0 0x7f0f200cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1ed9d640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f2015f880 0x7f0f201f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 -- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f2015f880 msgr2=0x7f0f201f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f2015f880 0x7f0f201f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 -- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2006eec0 msgr2=0x7f0f200cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2006eec0 0x7f0f200cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 -- 172.21.15.43:0/442771013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f200cb9f0 con 0x7f0f20070640 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1ed9d640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f2015f880 0x7f0f201f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f1dd9b640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20070640 0x7f0f2015b780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0f14019fb0 tx=0x7f0f14019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.931+0000 7f0f0effd640 1 -- 172.21.15.43:0/442771013 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140b0040 con 0x7f0f20070640 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f0effd640 1 -- 172.21.15.43:0/442771013 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f140ab7b0 con 0x7f0f20070640 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f200cbcd0 con 0x7f0f20070640 2024-03-30T13:11:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f0effd640 1 -- 172.21.15.43:0/442771013 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f140abb60 con 0x7f0f20070640 2024-03-30T13:11:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f200cbfb0 con 0x7f0f20070640 2024-03-30T13:11:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f20068900 con 0x7f0f20070640 2024-03-30T13:11:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f0effd640 1 -- 172.21.15.43:0/442771013 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0f140abd00 con 0x7f0f20070640 2024-03-30T13:11:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f0effd640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f00067fc0 0x7f0f0006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:49.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f0effd640 1 -- 172.21.15.43:0/442771013 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0f14120cf0 con 0x7f0f20070640 2024-03-30T13:11:49.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f1e59c640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f00067fc0 0x7f0f0006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:49.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.935+0000 7f0f1e59c640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f00067fc0 0x7f0f0006a480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009580 tx=0x7f0f10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:49.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:49.947+0000 7f0f0effd640 1 -- 172.21.15.43:0/442771013 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0f14016090 con 0x7f0f20070640 2024-03-30T13:11:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0f200d1790 con 0x7f0f20070640 2024-03-30T13:11:50.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f0effd640 1 -- 172.21.15.43:0/442771013 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f0f140f1210 con 0x7f0f20070640 2024-03-30T13:11:50.110 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f00067fc0 msgr2=0x7f0f0006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f00067fc0 0x7f0f0006a480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0f10009580 tx=0x7f0f10008040 comp rx=0 tx=0).stop 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20070640 msgr2=0x7f0f2015b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20070640 0x7f0f2015b780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0f14019fb0 tx=0x7f0f14019120 comp rx=0 tx=0).stop 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 shutdown_connections 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f00067fc0 0x7f0f0006a480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f2015f880 0x7f0f201f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f20070640 0x7f0f2015b780 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 --2- 172.21.15.43:0/442771013 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f2006eec0 0x7f0f200cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 >> 172.21.15.43:0/442771013 conn(0x7f0f200b3cf0 msgr2=0x7f0f2015c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 shutdown_connections 2024-03-30T13:11:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.103+0000 7f0f24f69640 1 -- 172.21.15.43:0/442771013 wait complete. 2024-03-30T13:11:50.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.263+0000 7fe6cc65c640 1 Processor -- start 2024-03-30T13:11:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.263+0000 7fe6cc65c640 1 -- start start 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cc65c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe6c415e2f0 0x7fe6c40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cc65c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c415e6d0 0x7fe6c40ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cc65c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6c40ad8e0 0x7fe6c40ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cc65c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe6c40cd060 con 0x7fe6c40ad8e0 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cc65c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe6c406b660 con 0x7fe6c415e2f0 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cc65c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c40b58c0 con 0x7fe6c415e6d0 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6c40ad8e0 0x7fe6c40ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6c40ad8e0 0x7fe6c40ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46420/0 (socket says 172.21.15.43:46420) 2024-03-30T13:11:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 -- 172.21.15.43:0/11210505 learned_addr learned my addr 172.21.15.43:0/11210505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6ca3f8640 1 --2- 172.21.15.43:0/11210505 >> v2:172.21.15.113:3300/0 conn(0x7fe6c415e2f0 0x7fe6c40b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6c9bf7640 1 --2- 172.21.15.43:0/11210505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c415e6d0 0x7fe6c40ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 -- 172.21.15.43:0/11210505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c415e6d0 msgr2=0x7fe6c40ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 --2- 172.21.15.43:0/11210505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c415e6d0 0x7fe6c40ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 -- 172.21.15.43:0/11210505 >> v2:172.21.15.113:3300/0 conn(0x7fe6c415e2f0 msgr2=0x7fe6c40b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 --2- 172.21.15.43:0/11210505 >> v2:172.21.15.113:3300/0 conn(0x7fe6c415e2f0 0x7fe6c40b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 -- 172.21.15.43:0/11210505 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c40b5ba0 con 0x7fe6c40ad8e0 2024-03-30T13:11:50.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6c9bf7640 1 --2- 172.21.15.43:0/11210505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c415e6d0 0x7fe6c40ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6cabf9640 1 --2- 172.21.15.43:0/11210505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c40ad8e0 0x7fe6c40ae790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40b5ad0 tx=0x7fe6bc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f691a243cf5d74 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6c93f6640 1 -- 172.21.15.43:0/11210505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0a5720 con 0x7fe6c40ad8e0 2024-03-30T13:11:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6c93f6640 1 -- 172.21.15.43:0/11210505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0a58c0 con 0x7fe6c40ad8e0 2024-03-30T13:11:50.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.267+0000 7fe6c93f6640 1 -- 172.21.15.43:0/11210505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0a5c90 con 0x7fe6c40ad8e0 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- 172.21.15.43:0/11210505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c40ad8e0 msgr2=0x7fe6c40ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/11210505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c40ad8e0 0x7fe6c40ae790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40b5ad0 tx=0x7fe6bc01c8a0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- 172.21.15.43:0/11210505 shutdown_connections 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/11210505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c40ad8e0 0x7fe6c40ae790 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/11210505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c415e6d0 0x7fe6c40ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/11210505 >> v2:172.21.15.113:3300/0 conn(0x7fe6c415e2f0 0x7fe6c40b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- 172.21.15.43:0/11210505 >> 172.21.15.43:0/11210505 conn(0x7fe6c40d0c90 msgr2=0x7fe6c40d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- 172.21.15.43:0/11210505 shutdown_connections 2024-03-30T13:11:50.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- 172.21.15.43:0/11210505 wait complete. 2024-03-30T13:11:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 Processor -- start 2024-03-30T13:11:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- start start 2024-03-30T13:11:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c40ad8e0 0x7fe6c414e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6c415e2f0 0x7fe6c415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c415e6d0 0x7fe6c415d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c41628f0 con 0x7fe6c415e6d0 2024-03-30T13:11:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c4162a70 con 0x7fe6c415e2f0 2024-03-30T13:11:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c4162bf0 con 0x7fe6c40ad8e0 2024-03-30T13:11:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c415e6d0 0x7fe6c415d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c415e6d0 0x7fe6c415d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46424/0 (socket says 172.21.15.43:46424) 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 -- 172.21.15.43:0/1570419117 learned_addr learned my addr 172.21.15.43:0/1570419117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6c9bf7640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6c415e2f0 0x7fe6c415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6ca3f8640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c40ad8e0 0x7fe6c414e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 -- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c40ad8e0 msgr2=0x7fe6c414e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c40ad8e0 0x7fe6c414e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 -- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6c415e2f0 msgr2=0x7fe6c415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6c415e2f0 0x7fe6c415d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 -- 172.21.15.43:0/1570419117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c4159360 con 0x7fe6c415e6d0 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6c9bf7640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6c415e2f0 0x7fe6c415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cabf9640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c415e6d0 0x7fe6c415d660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0af040 tx=0x7fe6bc019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6baffd640 1 -- 172.21.15.43:0/1570419117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0bc020 con 0x7fe6c415e6d0 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6baffd640 1 -- 172.21.15.43:0/1570419117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6bc0b8670 con 0x7fe6c415e6d0 2024-03-30T13:11:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c4159640 con 0x7fe6c415e6d0 2024-03-30T13:11:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6baffd640 1 -- 172.21.15.43:0/1570419117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0b8a60 con 0x7fe6c415e6d0 2024-03-30T13:11:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.271+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c4152720 con 0x7fe6c415e6d0 2024-03-30T13:11:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.275+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6c4152510 con 0x7fe6c415e6d0 2024-03-30T13:11:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.275+0000 7fe6baffd640 1 -- 172.21.15.43:0/1570419117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe6bc0bf020 con 0x7fe6c415e6d0 2024-03-30T13:11:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.275+0000 7fe6baffd640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6a8067fc0 0x7fe6a806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.275+0000 7fe6baffd640 1 -- 172.21.15.43:0/1570419117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fe6bc1219a0 con 0x7fe6c415e6d0 2024-03-30T13:11:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.275+0000 7fe6ca3f8640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6a8067fc0 0x7fe6a806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.275+0000 7fe6ca3f8640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6a8067fc0 0x7fe6a806a480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40aec90 tx=0x7fe6c0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.283+0000 7fe6baffd640 1 -- 172.21.15.43:0/1570419117 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe6bc0c6030 con 0x7fe6c415e6d0 2024-03-30T13:11:50.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.439+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe6c4068900 con 0x7fe6c415e6d0 2024-03-30T13:11:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6baffd640 1 -- 172.21.15.43:0/1570419117 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fe6bc0165e0 con 0x7fe6c415e6d0 2024-03-30T13:11:50.449 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6a8067fc0 msgr2=0x7fe6a806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6a8067fc0 0x7fe6a806a480 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6c40aec90 tx=0x7fe6c0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c415e6d0 msgr2=0x7fe6c415d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c415e6d0 0x7fe6c415d660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0af040 tx=0x7fe6bc019640 comp rx=0 tx=0).stop 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 shutdown_connections 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6a8067fc0 0x7fe6a806a480 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6c415e6d0 0x7fe6c415d660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6c415e2f0 0x7fe6c415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 --2- 172.21.15.43:0/1570419117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6c40ad8e0 0x7fe6c414e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 >> 172.21.15.43:0/1570419117 conn(0x7fe6c40d0c90 msgr2=0x7fe6c4155300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 shutdown_connections 2024-03-30T13:11:50.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.443+0000 7fe6cc65c640 1 -- 172.21.15.43:0/1570419117 wait complete. 2024-03-30T13:11:50.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.587+0000 7f04373dc640 1 Processor -- start 2024-03-30T13:11:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.587+0000 7f04373dc640 1 -- start start 2024-03-30T13:11:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f04373dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f04300afc20 0x7f04300cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f04373dc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04300b5760 con 0x7f04300b50b0 2024-03-30T13:11:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f04373dc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f043006e660 con 0x7f043015b0f0 2024-03-30T13:11:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f04373dc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f043006e940 con 0x7f04300afc20 2024-03-30T13:11:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f04300b50b0 0x7f04300ce370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60616/0 (socket says 172.21.15.43:60616) 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 -- 172.21.15.43:0/3310499671 learned_addr learned my addr 172.21.15.43:0/3310499671 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0435979640 1 --2- 172.21.15.43:0/3310499671 >> v2:172.21.15.113:3300/0 conn(0x7f04300afc20 0x7f04300cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 756748817 0 0) 0x7f04300b5760 con 0x7f04300b50b0 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f041c005740 con 0x7f04300b50b0 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 135138193 0 0) 0x7f041c005740 con 0x7f04300b50b0 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 >> v2:172.21.15.113:3300/0 conn(0x7f04300afc20 msgr2=0x7f04300cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 --2- 172.21.15.43:0/3310499671 >> v2:172.21.15.113:3300/0 conn(0x7f04300afc20 0x7f04300cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 >> v1:172.21.15.113:6789/0 conn(0x7f043015b0f0 legacy=0x7f04300b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043006ec20 con 0x7f04300b50b0 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f0420001680 con 0x7f04300b50b0 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 >> v1:172.21.15.43:6789/0 conn(0x7f04300b50b0 legacy=0x7f04300ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c005bd0 0x7f041c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c006a20 0x7f041c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c0073e0 0x7f041c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f043006e940 con 0x7f041c006a20 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f041c00e280 con 0x7f041c0073e0 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f041c00e480 con 0x7f041c005bd0 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c006a20 0x7f041c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0435178640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c005bd0 0x7f041c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0435979640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c0073e0 0x7f041c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 -- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c005bd0 msgr2=0x7f041c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c005bd0 0x7f041c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 -- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c0073e0 msgr2=0x7f041c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c0073e0 0x7f041c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0434977640 1 -- 172.21.15.43:0/3310499671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c005740 con 0x7f041c006a20 2024-03-30T13:11:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.591+0000 7f0435979640 1 -- 172.21.15.43:0/3310499671 reap_dead start 2024-03-30T13:11:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c006a20 0x7f041c006ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f042000c1c0 tx=0x7f042001f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3824f74d253ce1cb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420007d00 con 0x7f041c006a20 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0420005b00 con 0x7f041c006a20 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0427fff640 1 -- 172.21.15.43:0/3310499671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420007070 con 0x7f041c006a20 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- 172.21.15.43:0/3310499671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c006a20 msgr2=0x7f041c006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c006a20 0x7f041c006ea0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f042000c1c0 tx=0x7f042001f390 comp rx=0 tx=0).stop 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- 172.21.15.43:0/3310499671 shutdown_connections 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 --2- 172.21.15.43:0/3310499671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c006a20 0x7f041c006ea0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- 172.21.15.43:0/3310499671 >> 172.21.15.43:0/3310499671 conn(0x7f04300b2980 msgr2=0x7f04300d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- 172.21.15.43:0/3310499671 shutdown_connections 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- 172.21.15.43:0/3310499671 wait complete. 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 Processor -- start 2024-03-30T13:11:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- start start 2024-03-30T13:11:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c006a20 0x7f0430153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0430156510 0x7f04300aeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04300af040 0x7f04301f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f043006f4c0 con 0x7f0430156510 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f043006f1e0 con 0x7f041c006a20 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f04373dc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f043015b4d0 con 0x7f04300af040 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0430156510 0x7f04300aeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0435178640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c006a20 0x7f0430153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0430156510 0x7f04300aeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46448/0 (socket says 172.21.15.43:46448) 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 -- 172.21.15.43:0/522416727 learned_addr learned my addr 172.21.15.43:0/522416727 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0435979640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04300af040 0x7f04301f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 -- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04300af040 msgr2=0x7f04301f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04300af040 0x7f04301f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 -- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c006a20 msgr2=0x7f0430153520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c006a20 0x7f0430153520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0434977640 1 -- 172.21.15.43:0/522416727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04300ada60 con 0x7f0430156510 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.595+0000 7f0435979640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04300af040 0x7f04301f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0434977640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0430156510 0x7f04300aeb00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f042001f1c0 tx=0x7f041c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0425ffb640 1 -- 172.21.15.43:0/522416727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420007d00 con 0x7f0430156510 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0425ffb640 1 -- 172.21.15.43:0/522416727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0420000ee0 con 0x7f0430156510 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0425ffb640 1 -- 172.21.15.43:0/522416727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420001480 con 0x7f0430156510 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04300add40 con 0x7f0430156510 2024-03-30T13:11:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04300ae300 con 0x7f0430156510 2024-03-30T13:11:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0430068890 con 0x7f0430156510 2024-03-30T13:11:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0425ffb640 1 -- 172.21.15.43:0/522416727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f04200047f0 con 0x7f0430156510 2024-03-30T13:11:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0425ffb640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0414067fc0 0x7f041406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0425ffb640 1 -- 172.21.15.43:0/522416727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0420123dd0 con 0x7f0430156510 2024-03-30T13:11:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.599+0000 7f0435178640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0414067fc0 0x7f041406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.603+0000 7f0435178640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0414067fc0 0x7f041406a480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f042c001660 tx=0x7f042c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.603+0000 7f0425ffb640 1 -- 172.21.15.43:0/522416727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f04200190a0 con 0x7f0430156510 2024-03-30T13:11:50.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.767+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04300ceed0 con 0x7f0430156510 2024-03-30T13:11:50.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.767+0000 7f0425ffb640 1 -- 172.21.15.43:0/522416727 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7f04200f4200 con 0x7f0430156510 2024-03-30T13:11:50.776 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:50.776 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0414067fc0 msgr2=0x7f041406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0414067fc0 0x7f041406a480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f042c001660 tx=0x7f042c009040 comp rx=0 tx=0).stop 2024-03-30T13:11:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0430156510 msgr2=0x7f04300aeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0430156510 0x7f04300aeb00 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f042001f1c0 tx=0x7f041c007a50 comp rx=0 tx=0).stop 2024-03-30T13:11:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 shutdown_connections 2024-03-30T13:11:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0414067fc0 0x7f041406a480 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04300af040 0x7f04301f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0430156510 0x7f04300aeb00 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 --2- 172.21.15.43:0/522416727 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c006a20 0x7f0430153520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 >> 172.21.15.43:0/522416727 conn(0x7f04300b2980 msgr2=0x7f04300b49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 shutdown_connections 2024-03-30T13:11:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.771+0000 7f04373dc640 1 -- 172.21.15.43:0/522416727 wait complete. 2024-03-30T13:11:50.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:50.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:50.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d180cd640 1 Processor -- start 2024-03-30T13:11:50.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d180cd640 1 -- start start 2024-03-30T13:11:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d180cd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 0x7f0d1006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d180cd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1015e9e0 con 0x7f0d100b42c0 2024-03-30T13:11:50.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d180cd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1015eca0 con 0x7f0d1006f9e0 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d180cd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1015f8b0 con 0x7f0d1006c790 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0d100b42c0 0x7f0d100b4730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60618/0 (socket says 172.21.15.43:60618) 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 -- 172.21.15.43:0/2485257052 learned_addr learned my addr 172.21.15.43:0/2485257052 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d15e69640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 0x7f0d1006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3784250088 0 0) 0x7f0d1015e9e0 con 0x7f0d100b42c0 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cfc003530 con 0x7f0d100b42c0 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3810901952 0 0) 0x7f0cfc003530 con 0x7f0d100b42c0 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 msgr2=0x7f0d1006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 0x7f0d1006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 >> v1:172.21.15.113:6789/0 conn(0x7f0d1006f9e0 legacy=0x7f0d1006fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1015ef80 con 0x7f0d100b42c0 2024-03-30T13:11:50.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f0d08001a00 con 0x7f0d100b42c0 2024-03-30T13:11:50.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 >> v1:172.21.15.43:6789/0 conn(0x7f0d100b42c0 legacy=0x7f0d100b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cfc0039f0 0x7f0cfc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 0x7f0cfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc004d50 0x7f0cfc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1015f8b0 con 0x7f0cfc004d50 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc00ba80 con 0x7f0cfc0039f0 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc00bc00 con 0x7f0d1006c790 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc004d50 0x7f0cfc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d15e69640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cfc0039f0 0x7f0cfc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d15668640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 0x7f0cfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 -- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 msgr2=0x7f0cfc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d1006c790 0x7f0cfc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 -- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cfc0039f0 msgr2=0x7f0cfc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cfc0039f0 0x7f0cfc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.915+0000 7f0d1666a640 1 -- 172.21.15.43:0/2485257052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc003530 con 0x7f0cfc004d50 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d1666a640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc004d50 0x7f0cfc0051d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800a880 tx=0x7f0d0801fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50dff7567c501c0f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0800a4a0 con 0x7f0cfc004d50 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d080aa730 con 0x7f0cfc004d50 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d14e67640 1 -- 172.21.15.43:0/2485257052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d080aab20 con 0x7f0cfc004d50 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d180cd640 1 -- 172.21.15.43:0/2485257052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc004d50 msgr2=0x7f0cfc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d180cd640 1 --2- 172.21.15.43:0/2485257052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cfc004d50 0x7f0cfc0051d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800a880 tx=0x7f0d0801fae0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d1666a640 1 -- 172.21.15.43:0/2485257052 reap_dead start 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d180cd640 1 -- 172.21.15.43:0/2485257052 shutdown_connections 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d180cd640 1 -- 172.21.15.43:0/2485257052 >> 172.21.15.43:0/2485257052 conn(0x7f0d100d1070 msgr2=0x7f0d100d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:50.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d180cd640 1 -- 172.21.15.43:0/2485257052 shutdown_connections 2024-03-30T13:11:50.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.919+0000 7f0d180cd640 1 -- 172.21.15.43:0/2485257052 wait complete. 2024-03-30T13:11:50.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 Processor -- start 2024-03-30T13:11:50.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 -- start start 2024-03-30T13:11:50.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d101f1310 0x7f0d100cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d101f4980 0x7f0d10161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d10161b90 0x7f0d10147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10070b20 con 0x7f0d101f4980 2024-03-30T13:11:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1015f540 con 0x7f0d101f1310 2024-03-30T13:11:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1006cb70 con 0x7f0d10161b90 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d101f4980 0x7f0d10161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d101f4980 0x7f0d10161650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46456/0 (socket says 172.21.15.43:46456) 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 -- 172.21.15.43:0/911077020 learned_addr learned my addr 172.21.15.43:0/911077020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15e69640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d101f1310 0x7f0d100cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d1666a640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d10161b90 0x7f0d10147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 -- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d10161b90 msgr2=0x7f0d10147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d10161b90 0x7f0d10147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 -- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d101f1310 msgr2=0x7f0d100cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d101f1310 0x7f0d100cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 -- 172.21.15.43:0/911077020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d101f4e00 con 0x7f0d101f4980 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d15668640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d101f4980 0x7f0d10161650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e600 tx=0x7f0d0c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d067fc640 1 -- 172.21.15.43:0/911077020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0053a0 con 0x7f0d101f4980 2024-03-30T13:11:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d067fc640 1 -- 172.21.15.43:0/911077020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0c018070 con 0x7f0d101f4980 2024-03-30T13:11:50.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d100cd960 con 0x7f0d101f4980 2024-03-30T13:11:50.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d067fc640 1 -- 172.21.15.43:0/911077020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c003840 con 0x7f0d101f4980 2024-03-30T13:11:50.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d100cdf20 con 0x7f0d101f4980 2024-03-30T13:11:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.923+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d10161fa0 con 0x7f0d101f4980 2024-03-30T13:11:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.927+0000 7f0d067fc640 1 -- 172.21.15.43:0/911077020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0d0c0039e0 con 0x7f0d101f4980 2024-03-30T13:11:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.927+0000 7f0d067fc640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0cf4067ff0 0x7f0cf406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.927+0000 7f0d067fc640 1 -- 172.21.15.43:0/911077020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0d0c086570 con 0x7f0d101f4980 2024-03-30T13:11:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.927+0000 7f0d15e69640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0cf4067ff0 0x7f0cf406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.927+0000 7f0d15e69640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0cf4067ff0 0x7f0cf406a4b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009b20 tx=0x7f0d00008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:50.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:50.935+0000 7f0d067fc640 1 -- 172.21.15.43:0/911077020 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0d10161fa0 con 0x7f0d101f4980 2024-03-30T13:11:51.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d10068900 con 0x7f0d101f4980 2024-03-30T13:11:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d067fc640 1 -- 172.21.15.43:0/911077020 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f0d0c056a40 con 0x7f0d101f4980 2024-03-30T13:11:51.106 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0cf4067ff0 msgr2=0x7f0cf406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0cf4067ff0 0x7f0cf406a4b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0d00009b20 tx=0x7f0d00008040 comp rx=0 tx=0).stop 2024-03-30T13:11:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d101f4980 msgr2=0x7f0d10161650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d101f4980 0x7f0d10161650 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00e600 tx=0x7f0d0c00eac0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 shutdown_connections 2024-03-30T13:11:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0cf4067ff0 0x7f0cf406a4b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d10161b90 0x7f0d10147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d101f4980 0x7f0d10161650 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 --2- 172.21.15.43:0/911077020 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d101f1310 0x7f0d100cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 >> 172.21.15.43:0/911077020 conn(0x7f0d100d1070 msgr2=0x7f0d100af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 shutdown_connections 2024-03-30T13:11:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.099+0000 7f0d180cd640 1 -- 172.21.15.43:0/911077020 wait complete. 2024-03-30T13:11:51.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:51.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.247+0000 7f7a91364640 1 Processor -- start 2024-03-30T13:11:51.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.247+0000 7f7a91364640 1 -- start start 2024-03-30T13:11:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a91364640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a8c0d0bc0 0x7f7a8c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a91364640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06cfe0 0x7f7a8c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a91364640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8c15d190 con 0x7f7a8c06f9e0 2024-03-30T13:11:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a91364640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8c06cb50 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a91364640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c06bf40 con 0x7f7a8c06cfe0 2024-03-30T13:11:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a8c0d0bc0 0x7f7a8c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7a8c06f9e0 0x7f7a8c06b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41288/0 (socket says 172.21.15.43:41288) 2024-03-30T13:11:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8affd640 1 -- 172.21.15.43:0/1378261652 learned_addr learned my addr 172.21.15.43:0/1378261652 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8b7fe640 1 --2- 172.21.15.43:0/1378261652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06cfe0 0x7f7a8c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8a7fc640 1 -- 172.21.15.43:0/1378261652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06cfe0 msgr2=0x7f7a8c06d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8a7fc640 1 --2- 172.21.15.43:0/1378261652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06cfe0 0x7f7a8c06d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8a7fc640 1 -- 172.21.15.43:0/1378261652 >> v1:172.21.15.43:6789/0 conn(0x7f7a8c06f9e0 legacy=0x7f7a8c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8a7fc640 1 -- 172.21.15.43:0/1378261652 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c06c220 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a89ffb640 1 -- 172.21.15.43:0/1378261652 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3586115586 0 0) 0x7f7a8c15d190 con 0x7f7a8c06f9e0 2024-03-30T13:11:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a8a7fc640 1 --2- 172.21.15.43:0/1378261652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c15c860 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097c0 tx=0x7f7a8001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520cdc9a4c9c2eec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a89ffb640 1 -- 172.21.15.43:0/1378261652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800b2070 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a89ffb640 1 -- 172.21.15.43:0/1378261652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a800ad640 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.251+0000 7f7a89ffb640 1 -- 172.21.15.43:0/1378261652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800ada10 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- 172.21.15.43:0/1378261652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 msgr2=0x7f7a8c15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 --2- 172.21.15.43:0/1378261652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c15c860 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097c0 tx=0x7f7a8001c4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- 172.21.15.43:0/1378261652 shutdown_connections 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 --2- 172.21.15.43:0/1378261652 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06cfe0 0x7f7a8c06d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 --2- 172.21.15.43:0/1378261652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c15c860 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- 172.21.15.43:0/1378261652 >> 172.21.15.43:0/1378261652 conn(0x7f7a8c0d1070 msgr2=0x7f7a8c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- 172.21.15.43:0/1378261652 shutdown_connections 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- 172.21.15.43:0/1378261652 wait complete. 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 Processor -- start 2024-03-30T13:11:51.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- start start 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a8c06cfe0 0x7f7a8c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06f9e0 0x7f7a8c1488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c070750 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0705d0 con 0x7f7a8c06cfe0 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c168ea0 con 0x7f7a8c06f9e0 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c139c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43038/0 (socket says 172.21.15.43:43038) 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 -- 172.21.15.43:0/2876085141 learned_addr learned my addr 172.21.15.43:0/2876085141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8affd640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a8c06cfe0 0x7f7a8c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8a7fc640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06f9e0 0x7f7a8c1488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 -- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06f9e0 msgr2=0x7f7a8c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06f9e0 0x7f7a8c1488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 -- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a8c06cfe0 msgr2=0x7f7a8c150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a8c06cfe0 0x7f7a8c150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 -- 172.21.15.43:0/2876085141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c0735c0 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8affd640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a8c06cfe0 0x7f7a8c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a8b7fe640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c139c90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400d560 tx=0x7f7a8400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a737fe640 1 -- 172.21.15.43:0/2876085141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840125f0 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a737fe640 1 -- 172.21.15.43:0/2876085141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a84019070 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c149030 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a737fe640 1 -- 172.21.15.43:0/2876085141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a840129f0 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.255+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c149e90 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.259+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c068900 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.259+0000 7f7a737fe640 1 -- 172.21.15.43:0/2876085141 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7a84012b90 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.259+0000 7f7a737fe640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a78067fc0 0x7f7a7806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.259+0000 7f7a737fe640 1 -- 172.21.15.43:0/2876085141 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f7a84086ca0 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.259+0000 7f7a8affd640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a78067fc0 0x7f7a7806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.259+0000 7f7a8affd640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a78067fc0 0x7f7a7806a480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005fd0 tx=0x7f7a7c005b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.263+0000 7f7a737fe640 1 -- 172.21.15.43:0/2876085141 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7a840570e0 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7a8c155cd0 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a737fe640 1 -- 172.21.15.43:0/2876085141 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f7a840569e0 con 0x7f7a8c0d0bc0 2024-03-30T13:11:51.434 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a78067fc0 msgr2=0x7f7a7806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a78067fc0 0x7f7a7806a480 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c005fd0 tx=0x7f7a7c005b80 comp rx=0 tx=0).stop 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 msgr2=0x7f7a8c139c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c139c90 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7a8400d560 tx=0x7f7a8400da20 comp rx=0 tx=0).stop 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 shutdown_connections 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a78067fc0 0x7f7a7806a480 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a8c0d0bc0 0x7f7a8c139c90 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7a8c06f9e0 0x7f7a8c1488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 --2- 172.21.15.43:0/2876085141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7a8c06cfe0 0x7f7a8c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 >> 172.21.15.43:0/2876085141 conn(0x7f7a8c0d1070 msgr2=0x7f7a8c06b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:51.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 shutdown_connections 2024-03-30T13:11:51.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.427+0000 7f7a91364640 1 -- 172.21.15.43:0/2876085141 wait complete. 2024-03-30T13:11:51.452 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f5405a56640 1 Processor -- start 2024-03-30T13:11:51.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f5405a56640 1 -- start start 2024-03-30T13:11:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f5405a56640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f540015fe20 0x7f5400160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f5405a56640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54000ccaa0 0x7f54000cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f5405a56640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54000cd410 con 0x7f540015c160 2024-03-30T13:11:51.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f5405a56640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f54001607d0 con 0x7f54000ccaa0 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f5405a56640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5400160c80 con 0x7f540015fe20 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53ff7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f54000ccaa0 0x7f54000cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53feffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f540015c160 0x7f540006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41298/0 (socket says 172.21.15.43:41298) 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53feffd640 1 -- 172.21.15.43:0/3477984664 learned_addr learned my addr 172.21.15.43:0/3477984664 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53fe7fc640 1 --2- 172.21.15.43:0/3477984664 >> v2:172.21.15.113:3300/0 conn(0x7f540015fe20 0x7f5400160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53ff7fe640 1 -- 172.21.15.43:0/3477984664 >> v2:172.21.15.113:3300/0 conn(0x7f540015fe20 msgr2=0x7f5400160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53ff7fe640 1 --2- 172.21.15.43:0/3477984664 >> v2:172.21.15.113:3300/0 conn(0x7f540015fe20 0x7f5400160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53ff7fe640 1 -- 172.21.15.43:0/3477984664 >> v1:172.21.15.43:6789/0 conn(0x7f540015c160 legacy=0x7f540006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53ff7fe640 1 -- 172.21.15.43:0/3477984664 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54000cd680 con 0x7f54000ccaa0 2024-03-30T13:11:51.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.571+0000 7f53fdffb640 1 -- 172.21.15.43:0/3477984664 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837503631 0 0) 0x7f54000cd410 con 0x7f540015c160 2024-03-30T13:11:51.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f53ff7fe640 1 --2- 172.21.15.43:0/3477984664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54000ccaa0 0x7f54000cced0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5400142bf0 tx=0x7f53f001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=645b8b785ad667df server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f53fdffb640 1 -- 172.21.15.43:0/3477984664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00ab650 con 0x7f54000ccaa0 2024-03-30T13:11:51.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f53fdffb640 1 -- 172.21.15.43:0/3477984664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f00b0070 con 0x7f54000ccaa0 2024-03-30T13:11:51.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f53fdffb640 1 -- 172.21.15.43:0/3477984664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00aba10 con 0x7f54000ccaa0 2024-03-30T13:11:51.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 -- 172.21.15.43:0/3477984664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54000ccaa0 msgr2=0x7f54000cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 --2- 172.21.15.43:0/3477984664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54000ccaa0 0x7f54000cced0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5400142bf0 tx=0x7f53f001c450 comp rx=0 tx=0).stop 2024-03-30T13:11:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 -- 172.21.15.43:0/3477984664 shutdown_connections 2024-03-30T13:11:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 --2- 172.21.15.43:0/3477984664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54000ccaa0 0x7f54000cced0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 --2- 172.21.15.43:0/3477984664 >> v2:172.21.15.113:3300/0 conn(0x7f540015fe20 0x7f5400160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 -- 172.21.15.43:0/3477984664 >> 172.21.15.43:0/3477984664 conn(0x7f54000d0c90 msgr2=0x7f54000d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 -- 172.21.15.43:0/3477984664 shutdown_connections 2024-03-30T13:11:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 -- 172.21.15.43:0/3477984664 wait complete. 2024-03-30T13:11:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 Processor -- start 2024-03-30T13:11:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.575+0000 7f5405a56640 1 -- start start 2024-03-30T13:11:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54000ccaa0 0x7f5400144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540015c160 0x7f540014aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f540015fe20 0x7f5400149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f54000cf420 con 0x7f540015fe20 2024-03-30T13:11:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f54000cf2a0 con 0x7f540015c160 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f540015d550 con 0x7f54000ccaa0 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f540015fe20 0x7f5400149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f540015fe20 0x7f5400149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43042/0 (socket says 172.21.15.43:43042) 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 -- 172.21.15.43:0/2929787064 learned_addr learned my addr 172.21.15.43:0/2929787064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53fe7fc640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540015c160 0x7f540014aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53feffd640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54000ccaa0 0x7f5400144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 -- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54000ccaa0 msgr2=0x7f5400144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54000ccaa0 0x7f5400144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 -- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540015c160 msgr2=0x7f540014aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540015c160 0x7f540014aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 -- 172.21.15.43:0/2929787064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5400145250 con 0x7f540015fe20 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53feffd640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54000ccaa0 0x7f5400144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53ff7fe640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f540015fe20 0x7f5400149cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53f00ad040 tx=0x7f53f001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53f7fff640 1 -- 172.21.15.43:0/2929787064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00bf610 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53f7fff640 1 -- 172.21.15.43:0/2929787064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f00bf7d0 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540015b540 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53f7fff640 1 -- 172.21.15.43:0/2929787064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00bfbc0 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54001494d0 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54001492c0 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53f7fff640 1 -- 172.21.15.43:0/2929787064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f53f00bd020 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53f7fff640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f53e0067fc0 0x7f53e006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53f7fff640 1 -- 172.21.15.43:0/2929787064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f53f0121440 con 0x7f540015fe20 2024-03-30T13:11:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.579+0000 7f53feffd640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f53e0067fc0 0x7f53e006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.583+0000 7f53feffd640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f53e0067fc0 0x7f53e006a480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53f8007fd0 tx=0x7f53f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.583+0000 7f53f7fff640 1 -- 172.21.15.43:0/2929787064 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f53f0016070 con 0x7f540015fe20 2024-03-30T13:11:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.743+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5400068900 con 0x7f540015fe20 2024-03-30T13:11:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f53f7fff640 1 -- 172.21.15.43:0/2929787064 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f53f00f17e0 con 0x7f540015fe20 2024-03-30T13:11:51.753 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f53e0067fc0 msgr2=0x7f53e006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f53e0067fc0 0x7f53e006a480 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f53f8007fd0 tx=0x7f53f8009040 comp rx=0 tx=0).stop 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f540015fe20 msgr2=0x7f5400149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f540015fe20 0x7f5400149cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f53f00ad040 tx=0x7f53f001ac90 comp rx=0 tx=0).stop 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 shutdown_connections 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f53e0067fc0 0x7f53e006a480 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f540015fe20 0x7f5400149cd0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f540015c160 0x7f540014aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 --2- 172.21.15.43:0/2929787064 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f54000ccaa0 0x7f5400144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 >> 172.21.15.43:0/2929787064 conn(0x7f54000d0c90 msgr2=0x7f540014f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:51.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 shutdown_connections 2024-03-30T13:11:51.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.747+0000 7f5405a56640 1 -- 172.21.15.43:0/2929787064 wait complete. 2024-03-30T13:11:51.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:51.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce7659640 1 Processor -- start 2024-03-30T13:11:51.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce7659640 1 -- start start 2024-03-30T13:11:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce7659640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 0x7f7ce00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce7659640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ce00ccfd0 con 0x7f7ce006c850 2024-03-30T13:11:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce7659640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ce00b4970 con 0x7f7ce006cc30 2024-03-30T13:11:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce7659640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce00b4c50 con 0x7f7ce015f5b0 2024-03-30T13:11:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce53f5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7ce006c850 0x7f7ce00b5730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41306/0 (socket says 172.21.15.43:41306) 2024-03-30T13:11:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce53f5640 1 -- 172.21.15.43:0/2170445081 learned_addr learned my addr 172.21.15.43:0/2170445081 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce4bf4640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 0x7f7ce00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838738451 0 0) 0x7f7ce00ccfd0 con 0x7f7ce006c850 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ccc003530 con 0x7f7ce006c850 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4055620726 0 0) 0x7f7ccc003530 con 0x7f7ce006c850 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 msgr2=0x7f7ce00ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 0x7f7ce00ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 >> v1:172.21.15.113:6789/0 conn(0x7f7ce006cc30 legacy=0x7f7ce00ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce00b4f30 con 0x7f7ce006c850 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f7cdc001680 con 0x7f7ce006c850 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 >> v1:172.21.15.43:6789/0 conn(0x7f7ce006c850 legacy=0x7f7ce00b5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ccc0039f0 0x7f7ccc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 0x7f7ccc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce00b4c50 con 0x7f7ccc004d50 2024-03-30T13:11:51.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ccc00b9d0 con 0x7f7ccc0039f0 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7ccc00bbd0 con 0x7f7ce015f5b0 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce5bf6640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce53f5640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ccc0039f0 0x7f7ccc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce4bf4640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 0x7f7ccc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce5bf6640 1 -- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 msgr2=0x7f7ccc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce5bf6640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 0x7f7ccc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce5bf6640 1 -- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ccc0039f0 msgr2=0x7f7ccc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce5bf6640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ccc0039f0 0x7f7ccc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce5bf6640 1 -- 172.21.15.43:0/2170445081 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ccc003530 con 0x7f7ccc004d50 2024-03-30T13:11:51.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.891+0000 7f7ce4bf4640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015f5b0 0x7f7ccc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:51.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce5bf6640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8001f60 tx=0x7f7cd801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859bee17e9eb6f88 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80b2070 con 0x7f7ccc004d50 2024-03-30T13:11:51.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd80a5770 con 0x7f7ccc004d50 2024-03-30T13:11:51.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7cd7fff640 1 -- 172.21.15.43:0/2170445081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8001b60 con 0x7f7ccc004d50 2024-03-30T13:11:51.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 -- 172.21.15.43:0/2170445081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ccc004d50 msgr2=0x7f7ccc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 --2- 172.21.15.43:0/2170445081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ccc004d50 0x7f7ccc0051d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8001f60 tx=0x7f7cd801c880 comp rx=0 tx=0).stop 2024-03-30T13:11:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce5bf6640 1 -- 172.21.15.43:0/2170445081 reap_dead start 2024-03-30T13:11:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 -- 172.21.15.43:0/2170445081 shutdown_connections 2024-03-30T13:11:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 -- 172.21.15.43:0/2170445081 >> 172.21.15.43:0/2170445081 conn(0x7f7ce00d0c90 msgr2=0x7f7ce00d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 -- 172.21.15.43:0/2170445081 shutdown_connections 2024-03-30T13:11:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 -- 172.21.15.43:0/2170445081 wait complete. 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 Processor -- start 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.895+0000 7f7ce7659640 1 -- start start 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce0156770 0x7f7ce015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015b780 0x7f7ce00ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ce00cedd0 0x7f7ce01f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce00aeb40 con 0x7f7ce0156770 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce00ae9c0 con 0x7f7ce00cedd0 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce00ad8e0 con 0x7f7ce015b780 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce0156770 0x7f7ce015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce0156770 0x7f7ce015cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43046/0 (socket says 172.21.15.43:43046) 2024-03-30T13:11:51.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 -- 172.21.15.43:0/910471754 learned_addr learned my addr 172.21.15.43:0/910471754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce5bf6640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ce00cedd0 0x7f7ce01f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce4bf4640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015b780 0x7f7ce00ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 -- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015b780 msgr2=0x7f7ce00ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015b780 0x7f7ce00ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 -- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ce00cedd0 msgr2=0x7f7ce01f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ce00cedd0 0x7f7ce01f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 -- 172.21.15.43:0/910471754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce015a6e0 con 0x7f7ce0156770 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce53f5640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce0156770 0x7f7ce015cf40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc012510 tx=0x7f7cdc0129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7cd5ffb640 1 -- 172.21.15.43:0/910471754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc01c070 con 0x7f7ce0156770 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7cd5ffb640 1 -- 172.21.15.43:0/910471754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc014830 con 0x7f7ce0156770 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7cd5ffb640 1 -- 172.21.15.43:0/910471754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc014c20 con 0x7f7ce0156770 2024-03-30T13:11:51.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce015a9c0 con 0x7f7ce0156770 2024-03-30T13:11:51.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce015af80 con 0x7f7ce0156770 2024-03-30T13:11:51.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ce0068900 con 0x7f7ce0156770 2024-03-30T13:11:51.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7cd5ffb640 1 -- 172.21.15.43:0/910471754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7cdc01f070 con 0x7f7ce0156770 2024-03-30T13:11:51.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7cd5ffb640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7cc4067ff0 0x7f7cc406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:51.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7cd5ffb640 1 -- 172.21.15.43:0/910471754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f7cdc0892a0 con 0x7f7ce0156770 2024-03-30T13:11:51.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.899+0000 7f7ce4bf4640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7cc4067ff0 0x7f7cc406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:51.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.903+0000 7f7ce4bf4640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7cc4067ff0 0x7f7cc406a4b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ce015bb60 tx=0x7f7cd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:51.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:51.907+0000 7f7cd5ffb640 1 -- 172.21.15.43:0/910471754 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7cdc059790 con 0x7f7ce0156770 2024-03-30T13:11:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.171+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7ce0158960 con 0x7f7ce0156770 2024-03-30T13:11:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.171+0000 7f7cd5ffb640 1 -- 172.21.15.43:0/910471754 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f7cdc059090 con 0x7f7ce0156770 2024-03-30T13:11:52.178 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7cc4067ff0 msgr2=0x7f7cc406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7cc4067ff0 0x7f7cc406a4b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ce015bb60 tx=0x7f7cd0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce0156770 msgr2=0x7f7ce015cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce0156770 0x7f7ce015cf40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc012510 tx=0x7f7cdc0129d0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 shutdown_connections 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7cc4067ff0 0x7f7cc406a4b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ce00cedd0 0x7f7ce01f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ce015b780 0x7f7ce00ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 --2- 172.21.15.43:0/910471754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ce0156770 0x7f7ce015cf40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 >> 172.21.15.43:0/910471754 conn(0x7f7ce00d0c90 msgr2=0x7f7ce0067370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 shutdown_connections 2024-03-30T13:11:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.175+0000 7f7ce7659640 1 -- 172.21.15.43:0/910471754 wait complete. 2024-03-30T13:11:52.205 INFO:tasks.cephfs_test_runner:Starting test: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:11:52.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbe307640 1 Processor -- start 2024-03-30T13:11:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbe307640 1 -- start start 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbe307640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0cb815b240 0x7f0cb80b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbe307640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cb80d1030 0x7f0cb80b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbe307640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cb80b4410 con 0x7f0cb80cdfa0 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbe307640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb806dab0 con 0x7f0cb815b240 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbe307640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0cb806e070 con 0x7f0cb80d1030 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0cb80cdfa0 0x7f0cb815ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41308/0 (socket says 172.21.15.43:41308) 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb77fe640 1 -- 172.21.15.43:0/1126406976 learned_addr learned my addr 172.21.15.43:0/1126406976 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb7fff640 1 --2- 172.21.15.43:0/1126406976 >> v2:172.21.15.43:3300/0 conn(0x7f0cb815b240 0x7f0cb80b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbc8a4640 1 --2- 172.21.15.43:0/1126406976 >> v2:172.21.15.113:3300/0 conn(0x7f0cb80d1030 0x7f0cb80b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb6ffd640 1 -- 172.21.15.43:0/1126406976 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908514862 0 0) 0x7f0cb80b4410 con 0x7f0cb80cdfa0 2024-03-30T13:11:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb6ffd640 1 -- 172.21.15.43:0/1126406976 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ca4003530 con 0x7f0cb80cdfa0 2024-03-30T13:11:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb7fff640 1 -- 172.21.15.43:0/1126406976 >> v2:172.21.15.113:3300/0 conn(0x7f0cb80d1030 msgr2=0x7f0cb80b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb7fff640 1 --2- 172.21.15.43:0/1126406976 >> v2:172.21.15.113:3300/0 conn(0x7f0cb80d1030 0x7f0cb80b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb7fff640 1 -- 172.21.15.43:0/1126406976 >> v1:172.21.15.43:6789/0 conn(0x7f0cb80cdfa0 legacy=0x7f0cb815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cb7fff640 1 -- 172.21.15.43:0/1126406976 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb80b46f0 con 0x7f0cb815b240 2024-03-30T13:11:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.331+0000 7f0cbc8a4640 1 --2- 172.21.15.43:0/1126406976 >> v2:172.21.15.113:3300/0 conn(0x7f0cb80d1030 0x7f0cb80b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:52.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cb7fff640 1 --2- 172.21.15.43:0/1126406976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb80b4ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80b23b0 tx=0x7f0cb001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3822b93fff120b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:52.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cb6ffd640 1 -- 172.21.15.43:0/1126406976 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00b0070 con 0x7f0cb815b240 2024-03-30T13:11:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cb6ffd640 1 -- 172.21.15.43:0/1126406976 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb00ab600 con 0x7f0cb815b240 2024-03-30T13:11:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cb6ffd640 1 -- 172.21.15.43:0/1126406976 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00ab9b0 con 0x7f0cb815b240 2024-03-30T13:11:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 -- 172.21.15.43:0/1126406976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 msgr2=0x7f0cb80b4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1126406976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb80b4ad0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0cb80b23b0 tx=0x7f0cb001c480 comp rx=0 tx=0).stop 2024-03-30T13:11:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 -- 172.21.15.43:0/1126406976 shutdown_connections 2024-03-30T13:11:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1126406976 >> v2:172.21.15.113:3300/0 conn(0x7f0cb80d1030 0x7f0cb80b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1126406976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb80b4ad0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 -- 172.21.15.43:0/1126406976 >> 172.21.15.43:0/1126406976 conn(0x7f0cb80b2b30 msgr2=0x7f0cb80d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 -- 172.21.15.43:0/1126406976 shutdown_connections 2024-03-30T13:11:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 -- 172.21.15.43:0/1126406976 wait complete. 2024-03-30T13:11:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 Processor -- start 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.335+0000 7f0cbe307640 1 -- start start 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb80cdfa0 0x7f0cb8161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cb80d1030 0x7f0cb8155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb81637e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb81681b0 con 0x7f0cb815b240 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8168030 con 0x7f0cb80cdfa0 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb8168330 con 0x7f0cb80d1030 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb81637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb81637e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43070/0 (socket says 172.21.15.43:43070) 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 -- 172.21.15.43:0/1099516824 learned_addr learned my addr 172.21.15.43:0/1099516824 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb7fff640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb80cdfa0 0x7f0cb8161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 -- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cb80d1030 msgr2=0x7f0cb8155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb77fe640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cb80d1030 0x7f0cb8155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cb80d1030 0x7f0cb8155990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 -- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb80cdfa0 msgr2=0x7f0cb8161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb80cdfa0 0x7f0cb8161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 -- 172.21.15.43:0/1099516824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb8161d60 con 0x7f0cb815b240 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb7fff640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb80cdfa0 0x7f0cb8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbc8a4640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb81637e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8007df0 tx=0x7f0ca800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb4ff9640 1 -- 172.21.15.43:0/1099516824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca801a070 con 0x7f0cb815b240 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb4ff9640 1 -- 172.21.15.43:0/1099516824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ca80039a0 con 0x7f0cb815b240 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb8162000 con 0x7f0cb815b240 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb4ff9640 1 -- 172.21.15.43:0/1099516824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca8003d90 con 0x7f0cb815b240 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb81625c0 con 0x7f0cb815b240 2024-03-30T13:11:52.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb8068890 con 0x7f0cb815b240 2024-03-30T13:11:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb4ff9640 1 -- 172.21.15.43:0/1099516824 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0ca801d070 con 0x7f0cb815b240 2024-03-30T13:11:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb4ff9640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0c98067fc0 0x7f0c9806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb4ff9640 1 -- 172.21.15.43:0/1099516824 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0ca8086b00 con 0x7f0cb815b240 2024-03-30T13:11:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.339+0000 7f0cb7fff640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0c98067fc0 0x7f0c9806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.343+0000 7f0cb7fff640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0c98067fc0 0x7f0c9806a480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00ad000 tx=0x7f0cb0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:52.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.343+0000 7f0cb4ff9640 1 -- 172.21.15.43:0/1099516824 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0ca8010070 con 0x7f0cb815b240 2024-03-30T13:11:52.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.651+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cb80b3700 con 0x7f0cb815b240 2024-03-30T13:11:52.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cb4ff9640 1 -- 172.21.15.43:0/1099516824 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7f0ca8056f30 con 0x7f0cb815b240 2024-03-30T13:11:52.662 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:52.662 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0c98067fc0 msgr2=0x7f0c9806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0c98067fc0 0x7f0c9806a480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00ad000 tx=0x7f0cb0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 msgr2=0x7f0cb81637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb81637e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ca8007df0 tx=0x7f0ca800d4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 shutdown_connections 2024-03-30T13:11:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0c98067fc0 0x7f0c9806a480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cb815b240 0x7f0cb81637e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cb80d1030 0x7f0cb8155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 --2- 172.21.15.43:0/1099516824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cb80cdfa0 0x7f0cb8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 >> 172.21.15.43:0/1099516824 conn(0x7f0cb80b2b30 msgr2=0x7f0cb806e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 shutdown_connections 2024-03-30T13:11:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.655+0000 7f0cbe307640 1 -- 172.21.15.43:0/1099516824 wait complete. 2024-03-30T13:11:52.683 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:52.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:52.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2ebc1dc640 1 Processor -- start 2024-03-30T13:11:52.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2ebc1dc640 1 -- start start 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2ebc1dc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 0x7f2eb406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2ebc1dc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb415d190 con 0x7f2eb40d0bc0 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2ebc1dc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb406bf40 con 0x7f2eb406cfe0 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2ebc1dc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb415cf70 con 0x7f2eb406f9e0 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2eb40d0bc0 0x7f2eb415c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41312/0 (socket says 172.21.15.43:41312) 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 -- 172.21.15.43:0/1093470703 learned_addr learned my addr 172.21.15.43:0/1093470703 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9f78640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 0x7f2eb406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1936145188 0 0) 0x7f2eb415d190 con 0x7f2eb40d0bc0 2024-03-30T13:11:52.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e9c003530 con 0x7f2eb40d0bc0 2024-03-30T13:11:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3384181195 0 0) 0x7f2e9c003530 con 0x7f2eb40d0bc0 2024-03-30T13:11:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 msgr2=0x7f2eb406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 0x7f2eb406b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 >> v1:172.21.15.113:6789/0 conn(0x7f2eb406cfe0 legacy=0x7f2eb406fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb406c220 con 0x7f2eb40d0bc0 2024-03-30T13:11:52.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f2ea8001680 con 0x7f2eb40d0bc0 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 >> v1:172.21.15.43:6789/0 conn(0x7f2eb40d0bc0 legacy=0x7f2eb415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 0x7f2e9c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e9c003f30 0x7f2e9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9c004d50 0x7f2e9c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb415cf70 con 0x7f2e9c003f30 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c00ba30 con 0x7f2e9c004d50 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9c00bc30 con 0x7f2eb406f9e0 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e9c003f30 0x7f2e9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9f78640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 0x7f2e9c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eba779640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9c004d50 0x7f2e9c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 -- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 msgr2=0x7f2e9c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb406f9e0 0x7f2e9c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 -- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9c004d50 msgr2=0x7f2e9c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:52.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2e9c004d50 0x7f2e9c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.811+0000 7f2eb9777640 1 -- 172.21.15.43:0/1093470703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9c003530 con 0x7f2e9c003f30 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2eb9777640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e9c003f30 0x7f2e9c004810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c2c0 tx=0x7f2ea801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98766752291b7d48 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8007d60 con 0x7f2e9c003f30 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea80a9070 con 0x7f2e9c003f30 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2eb8f76640 1 -- 172.21.15.43:0/1093470703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8005d80 con 0x7f2e9c003f30 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/1093470703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e9c003f30 msgr2=0x7f2e9c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2ebc1dc640 1 --2- 172.21.15.43:0/1093470703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2e9c003f30 0x7f2e9c004810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c2c0 tx=0x7f2ea801cc60 comp rx=0 tx=0).stop 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2eba779640 1 -- 172.21.15.43:0/1093470703 reap_dead start 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/1093470703 shutdown_connections 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/1093470703 >> 172.21.15.43:0/1093470703 conn(0x7f2eb40d1070 msgr2=0x7f2eb40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/1093470703 shutdown_connections 2024-03-30T13:11:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.815+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/1093470703 wait complete. 2024-03-30T13:11:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 Processor -- start 2024-03-30T13:11:52.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 -- start start 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb4154c40 0x7f2eb415e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb4155020 0x7f2eb413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb4153100 0x7f2eb413eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb41680b0 con 0x7f2eb4155020 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb406d820 con 0x7f2eb4153100 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb4168230 con 0x7f2eb4154c40 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb4155020 0x7f2eb413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb4155020 0x7f2eb413e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43082/0 (socket says 172.21.15.43:43082) 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 -- 172.21.15.43:0/108870095 learned_addr learned my addr 172.21.15.43:0/108870095 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9f78640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb4154c40 0x7f2eb415e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eba779640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb4153100 0x7f2eb413eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 -- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb4154c40 msgr2=0x7f2eb415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb4154c40 0x7f2eb415e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 -- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb4153100 msgr2=0x7f2eb413eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb4153100 0x7f2eb413eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 -- 172.21.15.43:0/108870095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb415ef40 con 0x7f2eb4155020 2024-03-30T13:11:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb9777640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb4155020 0x7f2eb413e5c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ea801ca90 tx=0x7f2e9c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb27fc640 1 -- 172.21.15.43:0/108870095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80048b0 con 0x7f2eb4155020 2024-03-30T13:11:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb27fc640 1 -- 172.21.15.43:0/108870095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea8004a50 con 0x7f2eb4155020 2024-03-30T13:11:52.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2eb27fc640 1 -- 172.21.15.43:0/108870095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004e40 con 0x7f2eb4155020 2024-03-30T13:11:52.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb415f220 con 0x7f2eb4155020 2024-03-30T13:11:52.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb415f7e0 con 0x7f2eb4155020 2024-03-30T13:11:52.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.819+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb4068900 con 0x7f2eb4155020 2024-03-30T13:11:52.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.823+0000 7f2eb27fc640 1 -- 172.21.15.43:0/108870095 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2ea80b6030 con 0x7f2eb4155020 2024-03-30T13:11:52.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.823+0000 7f2eb27fc640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2ea0067fc0 0x7f2ea006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:52.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.823+0000 7f2eb27fc640 1 -- 172.21.15.43:0/108870095 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f2ea811fa20 con 0x7f2eb4155020 2024-03-30T13:11:52.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.823+0000 7f2eb9f78640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2ea0067fc0 0x7f2ea006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:52.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.823+0000 7f2eb9f78640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2ea0067fc0 0x7f2ea006a480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009a30 tx=0x7f2eac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:52.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.827+0000 7f2eb27fc640 1 -- 172.21.15.43:0/108870095 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2ea8016970 con 0x7f2eb4155020 2024-03-30T13:11:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2eb415a800 con 0x7f2eb4155020 2024-03-30T13:11:52.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2eb27fc640 1 -- 172.21.15.43:0/108870095 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f2ea80efef0 con 0x7f2eb4155020 2024-03-30T13:11:52.997 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2ea0067fc0 msgr2=0x7f2ea006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2ea0067fc0 0x7f2ea006a480 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009a30 tx=0x7f2eac008040 comp rx=0 tx=0).stop 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb4155020 msgr2=0x7f2eb413e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb4155020 0x7f2eb413e5c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2ea801ca90 tx=0x7f2e9c005840 comp rx=0 tx=0).stop 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 shutdown_connections 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2ea0067fc0 0x7f2ea006a480 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2eb4153100 0x7f2eb413eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2eb4155020 0x7f2eb413e5c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 --2- 172.21.15.43:0/108870095 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2eb4154c40 0x7f2eb415e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 >> 172.21.15.43:0/108870095 conn(0x7f2eb40d1070 msgr2=0x7f2eb40cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:53.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 shutdown_connections 2024-03-30T13:11:53.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:52.991+0000 7f2ebc1dc640 1 -- 172.21.15.43:0/108870095 wait complete. 2024-03-30T13:11:53.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:53.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f82801e3640 1 Processor -- start 2024-03-30T13:11:53.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f82801e3640 1 -- start start 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f82801e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827815c160 0x7f827806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f82801e3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f827815fe20 0x7f8278160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f82801e3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8278160c80 con 0x7f82780ccaa0 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f82801e3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f82781607a0 con 0x7f827815fe20 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f82801e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f82780cd680 con 0x7f827815c160 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827d77e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f827815fe20 0x7f8278160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827d77e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f827815fe20 0x7f8278160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43094/0 (socket says 172.21.15.43:43094) 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827d77e640 1 -- 172.21.15.43:0/3140093188 learned_addr learned my addr 172.21.15.43:0/3140093188 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:53.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827df7f640 1 --2- 172.21.15.43:0/3140093188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827815c160 0x7f827806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827d77e640 1 -- 172.21.15.43:0/3140093188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827815c160 msgr2=0x7f827806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:53.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827d77e640 1 --2- 172.21.15.43:0/3140093188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827815c160 0x7f827806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827d77e640 1 -- 172.21.15.43:0/3140093188 >> v1:172.21.15.113:6789/0 conn(0x7f82780ccaa0 legacy=0x7f82780cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827d77e640 1 -- 172.21.15.43:0/3140093188 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82780cd960 con 0x7f827815fe20 2024-03-30T13:11:53.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.139+0000 7f827df7f640 1 --2- 172.21.15.43:0/3140093188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827815c160 0x7f827806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:53.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f827d77e640 1 --2- 172.21.15.43:0/3140093188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815fe20 0x7f8278160260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8268009870 tx=0x7f826801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708960cdd441213a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f827cf7d640 1 -- 172.21.15.43:0/3140093188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680a95f0 con 0x7f827815fe20 2024-03-30T13:11:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f827cf7d640 1 -- 172.21.15.43:0/3140093188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82680a9790 con 0x7f827815fe20 2024-03-30T13:11:53.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f827cf7d640 1 -- 172.21.15.43:0/3140093188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680a9b60 con 0x7f827815fe20 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 -- 172.21.15.43:0/3140093188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815fe20 msgr2=0x7f8278160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 --2- 172.21.15.43:0/3140093188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815fe20 0x7f8278160260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8268009870 tx=0x7f826801c530 comp rx=0 tx=0).stop 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 -- 172.21.15.43:0/3140093188 shutdown_connections 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 --2- 172.21.15.43:0/3140093188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815fe20 0x7f8278160260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 --2- 172.21.15.43:0/3140093188 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827815c160 0x7f827806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 -- 172.21.15.43:0/3140093188 >> 172.21.15.43:0/3140093188 conn(0x7f82780d0c90 msgr2=0x7f82780d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 -- 172.21.15.43:0/3140093188 shutdown_connections 2024-03-30T13:11:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.143+0000 7f82801e3640 1 -- 172.21.15.43:0/3140093188 wait complete. 2024-03-30T13:11:53.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 Processor -- start 2024-03-30T13:11:53.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 -- start start 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82780ccaa0 0x7f827813d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815c160 0x7f827806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827815fe20 0x7f827806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f82780cf420 con 0x7f827815c160 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f82780cf2a0 con 0x7f827815fe20 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f827815d550 con 0x7f82780ccaa0 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815c160 0x7f827806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815c160 0x7f827806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43110/0 (socket says 172.21.15.43:43110) 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 -- 172.21.15.43:0/1138077006 learned_addr learned my addr 172.21.15.43:0/1138077006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827df7f640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82780ccaa0 0x7f827813d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827e780640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827815fe20 0x7f827806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 -- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82780ccaa0 msgr2=0x7f827813d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82780ccaa0 0x7f827813d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 -- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827815fe20 msgr2=0x7f827806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827815fe20 0x7f827806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 -- 172.21.15.43:0/1138077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8278156280 con 0x7f827815c160 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827df7f640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82780ccaa0 0x7f827813d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f827d77e640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815c160 0x7f827806e360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f82680ab040 tx=0x7f826801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f826e7fc640 1 -- 172.21.15.43:0/1138077006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680b1800 con 0x7f827815c160 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f826e7fc640 1 -- 172.21.15.43:0/1138077006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82680b19c0 con 0x7f827815c160 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8278156560 con 0x7f827815c160 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f826e7fc640 1 -- 172.21.15.43:0/1138077006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82680b1db0 con 0x7f827815c160 2024-03-30T13:11:53.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.147+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8278156b20 con 0x7f827815c160 2024-03-30T13:11:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.151+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8278068900 con 0x7f827815c160 2024-03-30T13:11:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.151+0000 7f826e7fc640 1 -- 172.21.15.43:0/1138077006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f82680c53f0 con 0x7f827815c160 2024-03-30T13:11:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.151+0000 7f826e7fc640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.151+0000 7f826e7fc640 1 -- 172.21.15.43:0/1138077006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f82681217d0 con 0x7f827815c160 2024-03-30T13:11:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.151+0000 7f827df7f640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.151+0000 7f827df7f640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.155+0000 7f826e7fc640 1 -- 172.21.15.43:0/1138077006 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f82680ad020 con 0x7f827815c160 2024-03-30T13:11:53.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82781403b0 con 0x7f827815c160 2024-03-30T13:11:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f826e7fc640 1 -- 172.21.15.43:0/1138077006 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f8268016150 con 0x7f827815c160 2024-03-30T13:11:53.326 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:53.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 msgr2=0x7f825c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274008040 comp rx=0 tx=0).stop 2024-03-30T13:11:53.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815c160 msgr2=0x7f827806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815c160 0x7f827806e360 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f82680ab040 tx=0x7f826801ad70 comp rx=0 tx=0).stop 2024-03-30T13:11:53.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 shutdown_connections 2024-03-30T13:11:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827815fe20 0x7f827806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827815c160 0x7f827806e360 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 --2- 172.21.15.43:0/1138077006 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f82780ccaa0 0x7f827813d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 >> 172.21.15.43:0/1138077006 conn(0x7f82780d0c90 msgr2=0x7f827815e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 shutdown_connections 2024-03-30T13:11:53.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.319+0000 7f82801e3640 1 -- 172.21.15.43:0/1138077006 wait complete. 2024-03-30T13:11:53.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b83ed640 1 Processor -- start 2024-03-30T13:11:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b83ed640 1 -- start start 2024-03-30T13:11:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b83ed640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 0x7f93b0070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b83ed640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b00ce4d0 con 0x7f93b006f090 2024-03-30T13:11:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b83ed640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93b00cc7f0 con 0x7f93b015f880 2024-03-30T13:11:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b83ed640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f93b00b5440 con 0x7f93b00b4ff0 2024-03-30T13:11:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f93b006f090 0x7f93b00ce9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41328/0 (socket says 172.21.15.43:41328) 2024-03-30T13:11:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 -- 172.21.15.43:0/2457288373 learned_addr learned my addr 172.21.15.43:0/2457288373 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:53.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b698a640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 0x7f93b0070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1993571040 0 0) 0x7f93b00ce4d0 con 0x7f93b006f090 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f939c003530 con 0x7f93b006f090 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 420069924 0 0) 0x7f939c003530 con 0x7f93b006f090 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 msgr2=0x7f93b0070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 0x7f93b0070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 >> v1:172.21.15.113:6789/0 conn(0x7f93b015f880 legacy=0x7f93b006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b00ccad0 con 0x7f93b006f090 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f93a0001680 con 0x7f93b006f090 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 >> v1:172.21.15.43:6789/0 conn(0x7f93b006f090 legacy=0x7f93b00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0039f0 0x7f939c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c0043a0 0x7f939c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 0x7f939c005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93b00b5440 con 0x7f939c0043a0 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f939c00ba80 con 0x7f939c0039f0 2024-03-30T13:11:53.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f939c00bc80 con 0x7f93b00b4ff0 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c0043a0 0x7f939c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b698a640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 0x7f939c005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b6189640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0039f0 0x7f939c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 -- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 msgr2=0x7f939c005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b00b4ff0 0x7f939c005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 -- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0039f0 msgr2=0x7f939c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0039f0 0x7f939c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b5988640 1 -- 172.21.15.43:0/2457288373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c003530 con 0x7f939c0043a0 2024-03-30T13:11:53.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.471+0000 7f93b6189640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0039f0 0x7f939c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:53.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b5988640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c0043a0 0x7f939c004870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93a000c240 tx=0x7f93a001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722ba9e45890c6e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0007d40 con 0x7f939c0043a0 2024-03-30T13:11:53.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a00a9070 con 0x7f939c0043a0 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b5187640 1 -- 172.21.15.43:0/2457288373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0005d40 con 0x7f939c0043a0 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b83ed640 1 -- 172.21.15.43:0/2457288373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c0043a0 msgr2=0x7f939c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b83ed640 1 --2- 172.21.15.43:0/2457288373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f939c0043a0 0x7f939c004870 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93a000c240 tx=0x7f93a001cc80 comp rx=0 tx=0).stop 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b698a640 1 -- 172.21.15.43:0/2457288373 reap_dead start 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b83ed640 1 -- 172.21.15.43:0/2457288373 shutdown_connections 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b83ed640 1 -- 172.21.15.43:0/2457288373 >> 172.21.15.43:0/2457288373 conn(0x7f93b00b3cf0 msgr2=0x7f93b00b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b83ed640 1 -- 172.21.15.43:0/2457288373 shutdown_connections 2024-03-30T13:11:53.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b83ed640 1 -- 172.21.15.43:0/2457288373 wait complete. 2024-03-30T13:11:53.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.475+0000 7f93b83ed640 1 Processor -- start 2024-03-30T13:11:53.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 -- start start 2024-03-30T13:11:53.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0043a0 0x7f93b01576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93b0158440 0x7f93b00b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b0157e80 0x7f93b00b2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f93b00ccf30 con 0x7f93b0158440 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f93b00ccdb0 con 0x7f939c0043a0 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f93b00cf4d0 con 0x7f93b0157e80 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93b0158440 0x7f93b00b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93b0158440 0x7f93b00b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43128/0 (socket says 172.21.15.43:43128) 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 -- 172.21.15.43:0/2685902398 learned_addr learned my addr 172.21.15.43:0/2685902398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b6189640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0043a0 0x7f93b01576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b698a640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b0157e80 0x7f93b00b2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 -- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b0157e80 msgr2=0x7f93b00b2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b0157e80 0x7f93b00b2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 -- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0043a0 msgr2=0x7f93b01576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0043a0 0x7f93b01576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 -- 172.21.15.43:0/2685902398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b015a930 con 0x7f93b0158440 2024-03-30T13:11:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b5988640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93b0158440 0x7f93b00b1c40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f93a001cab0 tx=0x7f939c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93a6ffd640 1 -- 172.21.15.43:0/2685902398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0001010 con 0x7f93b0158440 2024-03-30T13:11:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93a6ffd640 1 -- 172.21.15.43:0/2685902398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93a00a5870 con 0x7f93b0158440 2024-03-30T13:11:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b015ac10 con 0x7f93b0158440 2024-03-30T13:11:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93a6ffd640 1 -- 172.21.15.43:0/2685902398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a00a5c60 con 0x7f93b0158440 2024-03-30T13:11:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b0157c20 con 0x7f93b0158440 2024-03-30T13:11:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b0068900 con 0x7f93b0158440 2024-03-30T13:11:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93a6ffd640 1 -- 172.21.15.43:0/2685902398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f93a00b7030 con 0x7f93b0158440 2024-03-30T13:11:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93a6ffd640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9394067fc0 0x7f939406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.479+0000 7f93a6ffd640 1 -- 172.21.15.43:0/2685902398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f93a011fb90 con 0x7f93b0158440 2024-03-30T13:11:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.483+0000 7f93b6189640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9394067fc0 0x7f939406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.483+0000 7f93b6189640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9394067fc0 0x7f939406a480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93ac000f70 tx=0x7f93ac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.483+0000 7f93a6ffd640 1 -- 172.21.15.43:0/2685902398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f93a0016990 con 0x7f93b0158440 2024-03-30T13:11:53.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.647+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f93b00d1790 con 0x7f93b0158440 2024-03-30T13:11:53.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93a6ffd640 1 -- 172.21.15.43:0/2685902398 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f93a00effd0 con 0x7f93b0158440 2024-03-30T13:11:53.657 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:53.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9394067fc0 msgr2=0x7f939406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9394067fc0 0x7f939406a480 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f93ac000f70 tx=0x7f93ac009040 comp rx=0 tx=0).stop 2024-03-30T13:11:53.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93b0158440 msgr2=0x7f93b00b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93b0158440 0x7f93b00b1c40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f93a001cab0 tx=0x7f939c005840 comp rx=0 tx=0).stop 2024-03-30T13:11:53.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 shutdown_connections 2024-03-30T13:11:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9394067fc0 0x7f939406a480 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f93b0157e80 0x7f93b00b2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f93b0158440 0x7f93b00b1c40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 --2- 172.21.15.43:0/2685902398 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f939c0043a0 0x7f93b01576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 >> 172.21.15.43:0/2685902398 conn(0x7f93b00b3cf0 msgr2=0x7f93b00b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 shutdown_connections 2024-03-30T13:11:53.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.651+0000 7f93b83ed640 1 -- 172.21.15.43:0/2685902398 wait complete. 2024-03-30T13:11:53.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:53.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 Processor -- start 2024-03-30T13:11:53.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 -- start start 2024-03-30T13:11:53.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62b415c390 0x7f62b40d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b406b660 0x7f62b415c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62b406f9e0 0x7f62b406fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f62b4070390 con 0x7f62b406f9e0 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f62b40cfd10 con 0x7f62b415c390 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62bbec9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f62b406baf0 con 0x7f62b406b660 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62b406f9e0 0x7f62b406fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62b406f9e0 0x7f62b406fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43140/0 (socket says 172.21.15.43:43140) 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 -- 172.21.15.43:0/2952114291 learned_addr learned my addr 172.21.15.43:0/2952114291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62b9c65640 1 --2- 172.21.15.43:0/2952114291 >> v2:172.21.15.113:3300/0 conn(0x7f62b415c390 0x7f62b40d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62b9464640 1 --2- 172.21.15.43:0/2952114291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b406b660 0x7f62b415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 -- 172.21.15.43:0/2952114291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b406b660 msgr2=0x7f62b415c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 --2- 172.21.15.43:0/2952114291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b406b660 0x7f62b415c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 -- 172.21.15.43:0/2952114291 >> v2:172.21.15.113:3300/0 conn(0x7f62b415c390 msgr2=0x7f62b40d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:53.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 --2- 172.21.15.43:0/2952114291 >> v2:172.21.15.113:3300/0 conn(0x7f62b415c390 0x7f62b40d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.787+0000 7f62ba466640 1 -- 172.21.15.43:0/2952114291 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b406bdd0 con 0x7f62b406f9e0 2024-03-30T13:11:53.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62ba466640 1 --2- 172.21.15.43:0/2952114291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b406fe50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f62b4071080 tx=0x7f62ac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e352131eb5a9c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62b8c63640 1 -- 172.21.15.43:0/2952114291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0a5760 con 0x7f62b406f9e0 2024-03-30T13:11:53.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62b8c63640 1 -- 172.21.15.43:0/2952114291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ac0a5900 con 0x7f62b406f9e0 2024-03-30T13:11:53.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62b8c63640 1 -- 172.21.15.43:0/2952114291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0a5cd0 con 0x7f62b406f9e0 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 -- 172.21.15.43:0/2952114291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 msgr2=0x7f62b406fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 --2- 172.21.15.43:0/2952114291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b406fe50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f62b4071080 tx=0x7f62ac01c9f0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 -- 172.21.15.43:0/2952114291 shutdown_connections 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 --2- 172.21.15.43:0/2952114291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b406fe50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 --2- 172.21.15.43:0/2952114291 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b406b660 0x7f62b415c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 --2- 172.21.15.43:0/2952114291 >> v2:172.21.15.113:3300/0 conn(0x7f62b415c390 0x7f62b40d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 -- 172.21.15.43:0/2952114291 >> 172.21.15.43:0/2952114291 conn(0x7f62b40d1070 msgr2=0x7f62b40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 -- 172.21.15.43:0/2952114291 shutdown_connections 2024-03-30T13:11:53.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 -- 172.21.15.43:0/2952114291 wait complete. 2024-03-30T13:11:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 Processor -- start 2024-03-30T13:11:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.791+0000 7f62bbec9640 1 -- start start 2024-03-30T13:11:53.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b414d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62b415c390 0x7f62b4157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b413bbd0 0x7f62b4159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62b406d380 con 0x7f62b406f9e0 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f62b415ccb0 con 0x7f62b415c390 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f62b406d500 con 0x7f62b413bbd0 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b414d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b414d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43156/0 (socket says 172.21.15.43:43156) 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 -- 172.21.15.43:0/1017875352 learned_addr learned my addr 172.21.15.43:0/1017875352 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9464640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62b415c390 0x7f62b4157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62ba466640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b413bbd0 0x7f62b4159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 -- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b413bbd0 msgr2=0x7f62b4159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b413bbd0 0x7f62b4159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 -- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62b415c390 msgr2=0x7f62b4157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62b415c390 0x7f62b4157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 -- 172.21.15.43:0/1017875352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b414ddc0 con 0x7f62b406f9e0 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9c65640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b414d750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f62b000d620 tx=0x7f62b000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62a67fc640 1 -- 172.21.15.43:0/1017875352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000b620 con 0x7f62b406f9e0 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62a67fc640 1 -- 172.21.15.43:0/1017875352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0019070 con 0x7f62b406f9e0 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b414e0a0 con 0x7f62b406f9e0 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62a67fc640 1 -- 172.21.15.43:0/1017875352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b000b9e0 con 0x7f62b406f9e0 2024-03-30T13:11:53.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b414e660 con 0x7f62b406f9e0 2024-03-30T13:11:53.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b414e860 con 0x7f62b406f9e0 2024-03-30T13:11:53.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62a67fc640 1 -- 172.21.15.43:0/1017875352 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f62b000bb80 con 0x7f62b406f9e0 2024-03-30T13:11:53.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62a67fc640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6298067ff0 0x7f629806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:53.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62a67fc640 1 -- 172.21.15.43:0/1017875352 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f62b0086b80 con 0x7f62b406f9e0 2024-03-30T13:11:53.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9464640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6298067ff0 0x7f629806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:53.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.795+0000 7f62b9464640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6298067ff0 0x7f629806a4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f62a8006fd0 tx=0x7f62a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:53.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.799+0000 7f62a67fc640 1 -- 172.21.15.43:0/1017875352 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f62b414e860 con 0x7f62b406f9e0 2024-03-30T13:11:53.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.959+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f62b4149b10 con 0x7f62b406f9e0 2024-03-30T13:11:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62a67fc640 1 -- 172.21.15.43:0/1017875352 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f62b0057050 con 0x7f62b406f9e0 2024-03-30T13:11:53.969 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6298067ff0 msgr2=0x7f629806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6298067ff0 0x7f629806a4b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f62a8006fd0 tx=0x7f62a8008040 comp rx=0 tx=0).stop 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 msgr2=0x7f62b414d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b414d750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f62b000d620 tx=0x7f62b000dae0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 shutdown_connections 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6298067ff0 0x7f629806a4b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62b413bbd0 0x7f62b4159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f62b415c390 0x7f62b4157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 --2- 172.21.15.43:0/1017875352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b406f9e0 0x7f62b414d750 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 >> 172.21.15.43:0/1017875352 conn(0x7f62b40d1070 msgr2=0x7f62b415dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 shutdown_connections 2024-03-30T13:11:53.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:53.963+0000 7f62bbec9640 1 -- 172.21.15.43:0/1017875352 wait complete. 2024-03-30T13:11:53.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-30T13:11:54.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.099+0000 7f4da35fc640 1 Processor -- start 2024-03-30T13:11:54.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da35fc640 1 -- start start 2024-03-30T13:11:54.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da35fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d9c145e10 0x7f4d9c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:54.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da35fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:54.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da35fc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d9c0d06c0 con 0x7f4d9c0d0220 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da35fc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4d9c0d09a0 con 0x7f4d9c145e10 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da35fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9c0d0c80 con 0x7f4d9c160470 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1398640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d9c145e10 0x7f4d9c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da0b97640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4d9c0d0220 0x7f4d9c15fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41332/0 (socket says 172.21.15.43:41332) 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1398640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d9c145e10 0x7f4d9c0cfce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43164/0 (socket says 172.21.15.43:43164) 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da0b97640 1 -- 172.21.15.43:0/606148520 learned_addr learned my addr 172.21.15.43:0/606148520 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1b99640 1 --2- 172.21.15.43:0/606148520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1398640 1 -- 172.21.15.43:0/606148520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 msgr2=0x7f4d9c0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1398640 1 --2- 172.21.15.43:0/606148520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c0b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1398640 1 -- 172.21.15.43:0/606148520 >> v1:172.21.15.43:6789/0 conn(0x7f4d9c0d0220 legacy=0x7f4d9c15fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1398640 1 -- 172.21.15.43:0/606148520 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9c0d0f60 con 0x7f4d9c145e10 2024-03-30T13:11:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4d93fff640 1 -- 172.21.15.43:0/606148520 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436629027 0 0) 0x7f4d9c0d06c0 con 0x7f4d9c0d0220 2024-03-30T13:11:54.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4da1398640 1 --2- 172.21.15.43:0/606148520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c0cfce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0097c0 tx=0x7f4d8c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4334eb1d81da214 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:54.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.103+0000 7f4d93fff640 1 -- 172.21.15.43:0/606148520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0b1070 con 0x7f4d9c145e10 2024-03-30T13:11:54.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4d93fff640 1 -- 172.21.15.43:0/606148520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8c0ac410 con 0x7f4d9c145e10 2024-03-30T13:11:54.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4d93fff640 1 -- 172.21.15.43:0/606148520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0ac7e0 con 0x7f4d9c145e10 2024-03-30T13:11:54.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- 172.21.15.43:0/606148520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 msgr2=0x7f4d9c0cfce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:54.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 --2- 172.21.15.43:0/606148520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c0cfce0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0097c0 tx=0x7f4d8c01c4a0 comp rx=0 tx=0).stop 2024-03-30T13:11:54.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- 172.21.15.43:0/606148520 shutdown_connections 2024-03-30T13:11:54.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 --2- 172.21.15.43:0/606148520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 --2- 172.21.15.43:0/606148520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c0cfce0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- 172.21.15.43:0/606148520 >> 172.21.15.43:0/606148520 conn(0x7f4d9c0ceb90 msgr2=0x7f4d9c0cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- 172.21.15.43:0/606148520 shutdown_connections 2024-03-30T13:11:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- 172.21.15.43:0/606148520 wait complete. 2024-03-30T13:11:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 Processor -- start 2024-03-30T13:11:54.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- start start 2024-03-30T13:11:54.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d9c0d0220 0x7f4d9c1613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c1f9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c1f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c0b28f0 con 0x7f4d9c145e10 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4d9c0b2770 con 0x7f4d9c0d0220 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4d9c0b2a70 con 0x7f4d9c160470 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c1f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c1f9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43176/0 (socket says 172.21.15.43:43176) 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 -- 172.21.15.43:0/3190788281 learned_addr learned my addr 172.21.15.43:0/3190788281 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da1398640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d9c0d0220 0x7f4d9c1613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 -- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 msgr2=0x7f4d9c1f50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da1b99640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c1f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c1f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 -- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d9c0d0220 msgr2=0x7f4d9c1613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:54.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d9c0d0220 0x7f4d9c1613d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 -- 172.21.15.43:0/3190788281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d9c1f5ae0 con 0x7f4d9c145e10 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da1398640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d9c0d0220 0x7f4d9c1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da0b97640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c1f9070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4d980011f0 tx=0x7f4d9800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4d91ffb640 1 -- 172.21.15.43:0/3190788281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98018070 con 0x7f4d9c145e10 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4d91ffb640 1 -- 172.21.15.43:0/3190788281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d980074e0 con 0x7f4d9c145e10 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d9c1f5dc0 con 0x7f4d9c145e10 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4d91ffb640 1 -- 172.21.15.43:0/3190788281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004a10 con 0x7f4d9c145e10 2024-03-30T13:11:54.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.107+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d9c13fc90 con 0x7f4d9c145e10 2024-03-30T13:11:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.111+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d9c068890 con 0x7f4d9c145e10 2024-03-30T13:11:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.111+0000 7f4d91ffb640 1 -- 172.21.15.43:0/3190788281 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4d9801b070 con 0x7f4d9c145e10 2024-03-30T13:11:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.111+0000 7f4d91ffb640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d80067fc0 0x7f4d8006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.111+0000 7f4d91ffb640 1 -- 172.21.15.43:0/3190788281 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f4d98085b70 con 0x7f4d9c145e10 2024-03-30T13:11:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.111+0000 7f4da1398640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d80067fc0 0x7f4d8006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:54.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.111+0000 7f4da1398640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d80067fc0 0x7f4d8006a480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ae000 tx=0x7f4d8c0a6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:54.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.115+0000 7f4d91ffb640 1 -- 172.21.15.43:0/3190788281 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4d98056040 con 0x7f4d9c145e10 2024-03-30T13:11:54.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:54.387+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f4d9c0b5940 con 0x7f4d9c145e10 2024-03-30T13:11:55.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.195+0000 7f4d91ffb640 1 -- 172.21.15.43:0/3190788281 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v76) ==== 150+0+0 (secure 0 0 0) 0x7f4d98055940 con 0x7f4d9c145e10 2024-03-30T13:11:55.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d80067fc0 msgr2=0x7f4d8006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d80067fc0 0x7f4d8006a480 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c0ae000 tx=0x7f4d8c0a6040 comp rx=0 tx=0).stop 2024-03-30T13:11:55.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 msgr2=0x7f4d9c1f9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c1f9070 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4d980011f0 tx=0x7f4d9800c9b0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 shutdown_connections 2024-03-30T13:11:55.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d80067fc0 0x7f4d8006a480 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d9c160470 0x7f4d9c1f50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d9c145e10 0x7f4d9c1f9070 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 --2- 172.21.15.43:0/3190788281 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d9c0d0220 0x7f4d9c1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 >> 172.21.15.43:0/3190788281 conn(0x7f4d9c0ceb90 msgr2=0x7f4d9c0b7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 shutdown_connections 2024-03-30T13:11:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.199+0000 7f4da35fc640 1 -- 172.21.15.43:0/3190788281 wait complete. 2024-03-30T13:11:55.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-30T13:11:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a8522b640 1 Processor -- start 2024-03-30T13:11:55.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a8522b640 1 -- start start 2024-03-30T13:11:55.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a8522b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5a8015afc0 0x7f5a800b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a8522b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 0x7f5a8006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a8522b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a800b4ba0 con 0x7f5a8006ff20 2024-03-30T13:11:55.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a8522b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5a80070c30 con 0x7f5a8015afc0 2024-03-30T13:11:55.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a8522b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5a800b4960 con 0x7f5a800b4520 2024-03-30T13:11:55.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7f59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5a8006ff20 0x7f5a8006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41336/0 (socket says 172.21.15.43:41336) 2024-03-30T13:11:55.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7f59e640 1 -- 172.21.15.43:0/3248881392 learned_addr learned my addr 172.21.15.43:0/3248881392 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:55.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7ed9d640 1 --2- 172.21.15.43:0/3248881392 >> v2:172.21.15.113:3300/0 conn(0x7f5a8015afc0 0x7f5a800b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7e59c640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 0x7f5a8006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3809863287 0 0) 0x7f5a800b4ba0 con 0x7f5a8006ff20 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a6c003530 con 0x7f5a8006ff20 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2093748139 0 0) 0x7f5a6c003530 con 0x7f5a8006ff20 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 msgr2=0x7f5a8006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 0x7f5a8006f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 >> v2:172.21.15.113:3300/0 conn(0x7f5a8015afc0 msgr2=0x7f5a800b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 --2- 172.21.15.43:0/3248881392 >> v2:172.21.15.113:3300/0 conn(0x7f5a8015afc0 0x7f5a800b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8015ec60 con 0x7f5a8006ff20 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f5a70001680 con 0x7f5a8006ff20 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.359+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 >> v1:172.21.15.43:6789/0 conn(0x7f5a8006ff20 legacy=0x7f5a8006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0039c0 0x7f5a6c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 0x7f5a6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c004d50 0x7f5a6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a800b4960 con 0x7f5a6c004d50 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c00b9d0 con 0x7f5a6c0039c0 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5a6c00bb50 con 0x7f5a800b4520 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c004d50 0x7f5a6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7ed9d640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0039c0 0x7f5a6c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7e59c640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 0x7f5a6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 -- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 msgr2=0x7f5a6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a800b4520 0x7f5a6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 -- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0039c0 msgr2=0x7f5a6c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0039c0 0x7f5a6c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 -- 172.21.15.43:0/3248881392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c003530 con 0x7f5a6c004d50 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7ed9d640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a6c0039c0 0x7f5a6c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c004d50 0x7f5a6c0051d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000c2c0 tx=0x7f5a7001cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7356c16cbc106f10 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70007d60 con 0x7f5a6c004d50 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a700a9020 con 0x7f5a6c004d50 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7dd9b640 1 -- 172.21.15.43:0/3248881392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a700a5bb0 con 0x7f5a6c004d50 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a8522b640 1 -- 172.21.15.43:0/3248881392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c004d50 msgr2=0x7f5a6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a8522b640 1 --2- 172.21.15.43:0/3248881392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a6c004d50 0x7f5a6c0051d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000c2c0 tx=0x7f5a7001cd40 comp rx=0 tx=0).stop 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a7f59e640 1 -- 172.21.15.43:0/3248881392 reap_dead start 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a8522b640 1 -- 172.21.15.43:0/3248881392 shutdown_connections 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a8522b640 1 -- 172.21.15.43:0/3248881392 >> 172.21.15.43:0/3248881392 conn(0x7f5a800d1070 msgr2=0x7f5a800d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a8522b640 1 -- 172.21.15.43:0/3248881392 shutdown_connections 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.363+0000 7f5a8522b640 1 -- 172.21.15.43:0/3248881392 wait complete. 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 Processor -- start 2024-03-30T13:11:55.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 -- start start 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a8014ae30 0x7f5a80153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8014b210 0x7f5a80157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80158280 0x7f5a80153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8015f500 con 0x7f5a8014b210 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8015f220 con 0x7f5a80158280 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5a800cf7c0 con 0x7f5a8014ae30 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8014b210 0x7f5a80157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8014b210 0x7f5a80157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43206/0 (socket says 172.21.15.43:43206) 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 -- 172.21.15.43:0/3621453708 learned_addr learned my addr 172.21.15.43:0/3621453708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7f59e640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80158280 0x7f5a80153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7ed9d640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a8014ae30 0x7f5a80153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 -- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a8014ae30 msgr2=0x7f5a80153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a8014ae30 0x7f5a80153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 -- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80158280 msgr2=0x7f5a80153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80158280 0x7f5a80153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 -- 172.21.15.43:0/3621453708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a80159580 con 0x7f5a8014b210 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7ed9d640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a8014ae30 0x7f5a80153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a7e59c640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8014b210 0x7f5a80157d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a680115e0 tx=0x7f5a68011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a777fe640 1 -- 172.21.15.43:0/3621453708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800f8f0 con 0x7f5a8014b210 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a777fe640 1 -- 172.21.15.43:0/3621453708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6800fa90 con 0x7f5a8014b210 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a777fe640 1 -- 172.21.15.43:0/3621453708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800fe80 con 0x7f5a8014b210 2024-03-30T13:11:55.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a80156250 con 0x7f5a8014b210 2024-03-30T13:11:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a80158aa0 con 0x7f5a8014b210 2024-03-30T13:11:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.367+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a80068900 con 0x7f5a8014b210 2024-03-30T13:11:55.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.371+0000 7f5a777fe640 1 -- 172.21.15.43:0/3621453708 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5a68015460 con 0x7f5a8014b210 2024-03-30T13:11:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.371+0000 7f5a777fe640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a60067ff0 0x7f5a6006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.371+0000 7f5a777fe640 1 -- 172.21.15.43:0/3621453708 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f5a68087390 con 0x7f5a8014b210 2024-03-30T13:11:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.371+0000 7f5a7ed9d640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a60067ff0 0x7f5a6006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.371+0000 7f5a7ed9d640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a60067ff0 0x7f5a6006a4b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a800b5980 tx=0x7f5a7800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:55.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.375+0000 7f5a777fe640 1 -- 172.21.15.43:0/3621453708 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5a68019030 con 0x7f5a8014b210 2024-03-30T13:11:55.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.535+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5a800706c0 con 0x7f5a8014b210 2024-03-30T13:11:55.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.539+0000 7f5a777fe640 1 -- 172.21.15.43:0/3621453708 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f5a68057940 con 0x7f5a8014b210 2024-03-30T13:11:55.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a60067ff0 msgr2=0x7f5a6006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a60067ff0 0x7f5a6006a4b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5a800b5980 tx=0x7f5a7800b040 comp rx=0 tx=0).stop 2024-03-30T13:11:55.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8014b210 msgr2=0x7f5a80157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8014b210 0x7f5a80157d40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a680115e0 tx=0x7f5a68011aa0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 shutdown_connections 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a60067ff0 0x7f5a6006a4b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a80158280 0x7f5a80153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a8014b210 0x7f5a80157d40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 --2- 172.21.15.43:0/3621453708 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a8014ae30 0x7f5a80153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 >> 172.21.15.43:0/3621453708 conn(0x7f5a800d1070 msgr2=0x7f5a8006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 shutdown_connections 2024-03-30T13:11:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.543+0000 7f5a8522b640 1 -- 172.21.15.43:0/3621453708 wait complete. 2024-03-30T13:11:55.568 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T13:11:55.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:11:55.622 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:11:55.678 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T13:11:55.679 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:11:55.679 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:55.684 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:11:55.684 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:55.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.903+0000 7feb012f2640 1 Processor -- start 2024-03-30T13:11:55.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.903+0000 7feb012f2640 1 -- start start 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feb012f2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feafc0cf080 0x7feafc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feb012f2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7feafc0b8140 0x7feafc0b8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feb012f2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feafc0d3af0 con 0x7feafc15e520 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feb012f2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7feafc0d3810 con 0x7feafc0cf080 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feb012f2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7feafc0b7660 con 0x7feafc0b8140 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafaffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feafc15e520 0x7feafc0b9c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41338/0 (socket says 172.21.15.43:41338) 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafaffd640 1 -- 172.21.15.43:0/2336575385 learned_addr learned my addr 172.21.15.43:0/2336575385 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:55.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafa7fc640 1 --2- 172.21.15.43:0/2336575385 >> v2:172.21.15.43:3300/0 conn(0x7feafc0cf080 0x7feafc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafb7fe640 1 --2- 172.21.15.43:0/2336575385 >> v2:172.21.15.113:3300/0 conn(0x7feafc0b8140 0x7feafc0b8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafa7fc640 1 -- 172.21.15.43:0/2336575385 >> v2:172.21.15.113:3300/0 conn(0x7feafc0b8140 msgr2=0x7feafc0b8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafa7fc640 1 --2- 172.21.15.43:0/2336575385 >> v2:172.21.15.113:3300/0 conn(0x7feafc0b8140 0x7feafc0b8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafa7fc640 1 -- 172.21.15.43:0/2336575385 >> v1:172.21.15.43:6789/0 conn(0x7feafc15e520 legacy=0x7feafc0b9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feaf9ffb640 1 -- 172.21.15.43:0/2336575385 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565029826 0 0) 0x7feafc0d3af0 con 0x7feafc15e520 2024-03-30T13:11:55.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.907+0000 7feafa7fc640 1 -- 172.21.15.43:0/2336575385 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc0b7940 con 0x7feafc0cf080 2024-03-30T13:11:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafa7fc640 1 --2- 172.21.15.43:0/2336575385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0cf080 0x7feafc0cf510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feafc0d07a0 tx=0x7feaf001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65969b2ceef29275 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feaf9ffb640 1 -- 172.21.15.43:0/2336575385 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00b0070 con 0x7feafc0cf080 2024-03-30T13:11:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feaf9ffb640 1 -- 172.21.15.43:0/2336575385 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf00ab600 con 0x7feafc0cf080 2024-03-30T13:11:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feaf9ffb640 1 -- 172.21.15.43:0/2336575385 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00ab9d0 con 0x7feafc0cf080 2024-03-30T13:11:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- 172.21.15.43:0/2336575385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0cf080 msgr2=0x7feafc0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 --2- 172.21.15.43:0/2336575385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0cf080 0x7feafc0cf510 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feafc0d07a0 tx=0x7feaf001c3f0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- 172.21.15.43:0/2336575385 shutdown_connections 2024-03-30T13:11:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 --2- 172.21.15.43:0/2336575385 >> v2:172.21.15.113:3300/0 conn(0x7feafc0b8140 0x7feafc0b8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 --2- 172.21.15.43:0/2336575385 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0cf080 0x7feafc0cf510 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- 172.21.15.43:0/2336575385 >> 172.21.15.43:0/2336575385 conn(0x7feafc0b5f90 msgr2=0x7feafc0b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- 172.21.15.43:0/2336575385 shutdown_connections 2024-03-30T13:11:55.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- 172.21.15.43:0/2336575385 wait complete. 2024-03-30T13:11:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 Processor -- start 2024-03-30T13:11:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- start start 2024-03-30T13:11:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0b8140 0x7feafc1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feafc0cf080 0x7feafc0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feafc15e520 0x7feafc1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7feafc16b2f0 con 0x7feafc0b8140 2024-03-30T13:11:55.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7feafc16b170 con 0x7feafc0cf080 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feb012f2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7feafc16b470 con 0x7feafc15e520 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0b8140 0x7feafc1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0b8140 0x7feafc1558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43224/0 (socket says 172.21.15.43:43224) 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 -- 172.21.15.43:0/2849005471 learned_addr learned my addr 172.21.15.43:0/2849005471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafa7fc640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feafc0cf080 0x7feafc0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafb7fe640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feafc15e520 0x7feafc1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 -- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feafc15e520 msgr2=0x7feafc1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feafc15e520 0x7feafc1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 -- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feafc0cf080 msgr2=0x7feafc0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feafc0cf080 0x7feafc0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.911+0000 7feafaffd640 1 -- 172.21.15.43:0/2849005471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feafc1536d0 con 0x7feafc0b8140 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feafaffd640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0b8140 0x7feafc1558d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c480 tx=0x7feaec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feae37fe640 1 -- 172.21.15.43:0/2849005471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec018070 con 0x7feafc0b8140 2024-03-30T13:11:55.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feae37fe640 1 -- 172.21.15.43:0/2849005471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaec007500 con 0x7feafc0b8140 2024-03-30T13:11:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feafc155e10 con 0x7feafc0b8140 2024-03-30T13:11:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feae37fe640 1 -- 172.21.15.43:0/2849005471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaec004a30 con 0x7feafc0b8140 2024-03-30T13:11:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feafc158ad0 con 0x7feafc0b8140 2024-03-30T13:11:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feafc068890 con 0x7feafc0b8140 2024-03-30T13:11:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feae37fe640 1 -- 172.21.15.43:0/2849005471 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7feaec01b070 con 0x7feafc0b8140 2024-03-30T13:11:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feae37fe640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7feae8067ff0 0x7feae806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feae37fe640 1 -- 172.21.15.43:0/2849005471 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7feaec085a30 con 0x7feafc0b8140 2024-03-30T13:11:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feafa7fc640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7feae8067ff0 0x7feae806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:55.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.915+0000 7feafa7fc640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7feae8067ff0 0x7feae806a4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feaf00ad000 tx=0x7feaf0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:55.919+0000 7feae37fe640 1 -- 172.21.15.43:0/2849005471 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7feaec0558b0 con 0x7feafc0b8140 2024-03-30T13:11:56.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.103+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feafc0008e0 con 0x7feafc0b8140 2024-03-30T13:11:56.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feae37fe640 1 -- 172.21.15.43:0/2849005471 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7feaec059bb0 con 0x7feafc0b8140 2024-03-30T13:11:56.114 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:56.115 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7feae8067ff0 msgr2=0x7feae806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7feae8067ff0 0x7feae806a4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7feaf00ad000 tx=0x7feaf0008040 comp rx=0 tx=0).stop 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0b8140 msgr2=0x7feafc1558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0b8140 0x7feafc1558d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c480 tx=0x7feaec00c940 comp rx=0 tx=0).stop 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 shutdown_connections 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7feae8067ff0 0x7feae806a4b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7feafc15e520 0x7feafc1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7feafc0cf080 0x7feafc0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 --2- 172.21.15.43:0/2849005471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feafc0b8140 0x7feafc1558d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 >> 172.21.15.43:0/2849005471 conn(0x7feafc0b5f90 msgr2=0x7feafc0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 shutdown_connections 2024-03-30T13:11:56.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.107+0000 7feb012f2640 1 -- 172.21.15.43:0/2849005471 wait complete. 2024-03-30T13:11:56.134 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:56.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:56.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff530461640 1 Processor -- start 2024-03-30T13:11:56.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff530461640 1 -- start start 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff530461640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff528158b40 0x7ff5280b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff530461640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5280cce60 0x7ff5280b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff530461640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff528158f20 con 0x7ff5280b5530 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff530461640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff528162d00 con 0x7ff5280cce60 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff530461640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff5281620f0 con 0x7ff528158b40 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e9fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5280cce60 0x7ff5280b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e9fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5280cce60 0x7ff5280b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43230/0 (socket says 172.21.15.43:43230) 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e9fe640 1 -- 172.21.15.43:0/609577283 learned_addr learned my addr 172.21.15.43:0/609577283 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e1fd640 1 --2- 172.21.15.43:0/609577283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff528158b40 0x7ff5280b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e9fe640 1 -- 172.21.15.43:0/609577283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff528158b40 msgr2=0x7ff5280b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e9fe640 1 --2- 172.21.15.43:0/609577283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff528158b40 0x7ff5280b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e9fe640 1 -- 172.21.15.43:0/609577283 >> v1:172.21.15.113:6789/0 conn(0x7ff5280b5530 legacy=0x7ff5280cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.255+0000 7ff52e9fe640 1 -- 172.21.15.43:0/609577283 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5281623d0 con 0x7ff5280cce60 2024-03-30T13:11:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff52e9fe640 1 --2- 172.21.15.43:0/609577283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5280cce60 0x7ff5280b6c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff520009e50 tx=0x7ff52001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f9bfbbff520837 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff52d1fb640 1 -- 172.21.15.43:0/609577283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200a57b0 con 0x7ff5280cce60 2024-03-30T13:11:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff52d1fb640 1 -- 172.21.15.43:0/609577283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5200a5950 con 0x7ff5280cce60 2024-03-30T13:11:56.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff52d1fb640 1 -- 172.21.15.43:0/609577283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200a5d20 con 0x7ff5280cce60 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- 172.21.15.43:0/609577283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5280cce60 msgr2=0x7ff5280b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 --2- 172.21.15.43:0/609577283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5280cce60 0x7ff5280b6c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff520009e50 tx=0x7ff52001c970 comp rx=0 tx=0).stop 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- 172.21.15.43:0/609577283 shutdown_connections 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 --2- 172.21.15.43:0/609577283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5280cce60 0x7ff5280b6c50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 --2- 172.21.15.43:0/609577283 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff528158b40 0x7ff5280b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- 172.21.15.43:0/609577283 >> 172.21.15.43:0/609577283 conn(0x7ff5280b3cf0 msgr2=0x7ff5280b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- 172.21.15.43:0/609577283 shutdown_connections 2024-03-30T13:11:56.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- 172.21.15.43:0/609577283 wait complete. 2024-03-30T13:11:56.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 Processor -- start 2024-03-30T13:11:56.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- start start 2024-03-30T13:11:56.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5280b5530 0x7ff52814a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5280cce60 0x7ff5280b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff528158b40 0x7ff5280b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280b79a0 con 0x7ff528158b40 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff5280d1f40 con 0x7ff5280b5530 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.259+0000 7ff530461640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff5280b7b20 con 0x7ff5280cce60 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff528158b40 0x7ff5280b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff528158b40 0x7ff5280b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43240/0 (socket says 172.21.15.43:43240) 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 -- 172.21.15.43:0/1124293700 learned_addr learned my addr 172.21.15.43:0/1124293700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e1fd640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5280b5530 0x7ff52814a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52d9fc640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5280cce60 0x7ff5280b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 -- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5280cce60 msgr2=0x7ff5280b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5280cce60 0x7ff5280b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 -- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5280b5530 msgr2=0x7ff52814a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5280b5530 0x7ff52814a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 -- 172.21.15.43:0/1124293700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5281529c0 con 0x7ff528158b40 2024-03-30T13:11:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e1fd640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5280b5530 0x7ff52814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e9fe640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff528158b40 0x7ff5280b2100 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff52001a5a0 tx=0x7ff520019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff51effd640 1 -- 172.21.15.43:0/1124293700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200bc020 con 0x7ff528158b40 2024-03-30T13:11:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff51effd640 1 -- 172.21.15.43:0/1124293700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5200b8670 con 0x7ff528158b40 2024-03-30T13:11:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff528152ca0 con 0x7ff528158b40 2024-03-30T13:11:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff51effd640 1 -- 172.21.15.43:0/1124293700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200b8a60 con 0x7ff528158b40 2024-03-30T13:11:56.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff528153260 con 0x7ff528158b40 2024-03-30T13:11:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff528068900 con 0x7ff528158b40 2024-03-30T13:11:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff51effd640 1 -- 172.21.15.43:0/1124293700 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff5200bf020 con 0x7ff528158b40 2024-03-30T13:11:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff51effd640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff50c067fc0 0x7ff50c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff51effd640 1 -- 172.21.15.43:0/1124293700 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7ff5201219d0 con 0x7ff528158b40 2024-03-30T13:11:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.263+0000 7ff52e1fd640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff50c067fc0 0x7ff50c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.267+0000 7ff52e1fd640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff50c067fc0 0x7ff50c06a480 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b9f0 tx=0x7ff524008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.267+0000 7ff51effd640 1 -- 172.21.15.43:0/1124293700 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff5200c6030 con 0x7ff528158b40 2024-03-30T13:11:56.401 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:56.403 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:11:56.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.431+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff528161bd0 con 0x7ff528158b40 2024-03-30T13:11:56.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.431+0000 7ff51effd640 1 -- 172.21.15.43:0/1124293700 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7ff520016620 con 0x7ff528158b40 2024-03-30T13:11:56.441 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff50c067fc0 msgr2=0x7ff50c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff50c067fc0 0x7ff50c06a480 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b9f0 tx=0x7ff524008040 comp rx=0 tx=0).stop 2024-03-30T13:11:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff528158b40 msgr2=0x7ff5280b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff528158b40 0x7ff5280b2100 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff52001a5a0 tx=0x7ff520019710 comp rx=0 tx=0).stop 2024-03-30T13:11:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 shutdown_connections 2024-03-30T13:11:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff50c067fc0 0x7ff50c06a480 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff528158b40 0x7ff5280b2100 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff5280cce60 0x7ff5280b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 --2- 172.21.15.43:0/1124293700 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff5280b5530 0x7ff52814a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 >> 172.21.15.43:0/1124293700 conn(0x7ff5280b3cf0 msgr2=0x7ff5280cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 shutdown_connections 2024-03-30T13:11:56.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.435+0000 7ff530461640 1 -- 172.21.15.43:0/1124293700 wait complete. 2024-03-30T13:11:56.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c3a1e8640 1 Processor -- start 2024-03-30T13:11:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c3a1e8640 1 -- start start 2024-03-30T13:11:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c3a1e8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 0x7f1c3406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c3a1e8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c340b4ba0 con 0x7f1c3406ff20 2024-03-30T13:11:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c3a1e8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c3415ec60 con 0x7f1c3415afc0 2024-03-30T13:11:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c3a1e8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1c34070c30 con 0x7f1c340b46f0 2024-03-30T13:11:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1c3406ff20 0x7f1c340ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41344/0 (socket says 172.21.15.43:41344) 2024-03-30T13:11:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 -- 172.21.15.43:0/749417705 learned_addr learned my addr 172.21.15.43:0/749417705 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:56.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 0x7f1c3406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495480788 0 0) 0x7f1c340b4ba0 con 0x7f1c3406ff20 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c1c003530 con 0x7f1c3406ff20 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1644863574 0 0) 0x7f1c3415ec60 con 0x7f1c3415afc0 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c340b4ba0 con 0x7f1c3415afc0 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 525184848 0 0) 0x7f1c1c003530 con 0x7f1c3406ff20 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 msgr2=0x7f1c3406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 0x7f1c3406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 >> v1:172.21.15.113:6789/0 conn(0x7f1c3415afc0 legacy=0x7f1c340b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3415ef40 con 0x7f1c3406ff20 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f1c2c001a00 con 0x7f1c3406ff20 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 >> v1:172.21.15.43:6789/0 conn(0x7f1c3406ff20 legacy=0x7f1c340ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c1c003b80 0x7f1c1c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 0x7f1c1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c004d50 0x7f1c1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c00dc90 con 0x7f1c1c004d50 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c00de10 con 0x7f1c1c003b80 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c00e010 con 0x7f1c340b46f0 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c004d50 0x7f1c1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c337fe640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c1c003b80 0x7f1c1c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 0x7f1c1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 -- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 msgr2=0x7f1c1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c340b46f0 0x7f1c1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 -- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c1c003b80 msgr2=0x7f1c1c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c1c003b80 0x7f1c1c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c33fff640 1 -- 172.21.15.43:0/749417705 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c003530 con 0x7f1c1c004d50 2024-03-30T13:11:56.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.615+0000 7f1c337fe640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c1c003b80 0x7f1c1c003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c33fff640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c004d50 0x7f1c1c0051d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00a730 tx=0x7f1c2c01f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1834d760097cbc3b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c00a350 con 0x7f1c1c004d50 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c2c0b1070 con 0x7f1c1c004d50 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c327fc640 1 -- 172.21.15.43:0/749417705 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0ac890 con 0x7f1c1c004d50 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/749417705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c004d50 msgr2=0x7f1c1c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 --2- 172.21.15.43:0/749417705 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c1c004d50 0x7f1c1c0051d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00a730 tx=0x7f1c2c01f980 comp rx=0 tx=0).stop 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c33fff640 1 -- 172.21.15.43:0/749417705 reap_dead start 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/749417705 shutdown_connections 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/749417705 >> 172.21.15.43:0/749417705 conn(0x7f1c340d1070 msgr2=0x7f1c340d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/749417705 shutdown_connections 2024-03-30T13:11:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/749417705 wait complete. 2024-03-30T13:11:56.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 Processor -- start 2024-03-30T13:11:56.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.619+0000 7f1c3a1e8640 1 -- start start 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c34151080 0x7f1c34150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c34151b50 0x7f1c3414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c34151460 0x7f1c3415b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3415fa20 con 0x7f1c34151b50 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3415f500 con 0x7f1c34151460 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1c34150c80 con 0x7f1c34151080 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c34151b50 0x7f1c3414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c34151b50 0x7f1c3414f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43254/0 (socket says 172.21.15.43:43254) 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 -- 172.21.15.43:0/1336418107 learned_addr learned my addr 172.21.15.43:0/1336418107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c33fff640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c34151460 0x7f1c3415b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c337fe640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c34151080 0x7f1c34150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 -- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c34151080 msgr2=0x7f1c34150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c34151080 0x7f1c34150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 -- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c34151460 msgr2=0x7f1c3415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c34151460 0x7f1c3415b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 -- 172.21.15.43:0/1336418107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c341f4cd0 con 0x7f1c34151b50 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c33fff640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c34151460 0x7f1c3415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c32ffd640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c34151b50 0x7f1c3414f930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011590 tx=0x7f1c28011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c1bfff640 1 -- 172.21.15.43:0/1336418107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28004220 con 0x7f1c34151b50 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c1bfff640 1 -- 172.21.15.43:0/1336418107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c280043c0 con 0x7f1c34151b50 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3414ee50 con 0x7f1c34151b50 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c1bfff640 1 -- 172.21.15.43:0/1336418107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28015870 con 0x7f1c34151b50 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c341f4980 con 0x7f1c34151b50 2024-03-30T13:11:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c34068900 con 0x7f1c34151b50 2024-03-30T13:11:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c1bfff640 1 -- 172.21.15.43:0/1336418107 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1c28015a10 con 0x7f1c34151b50 2024-03-30T13:11:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c1bfff640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c20067ff0 0x7f1c2006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c1bfff640 1 -- 172.21.15.43:0/1336418107 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f1c28086e60 con 0x7f1c34151b50 2024-03-30T13:11:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.623+0000 7f1c337fe640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c20067ff0 0x7f1c2006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.627+0000 7f1c337fe640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c20067ff0 0x7f1c2006a4b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c3415d2b0 tx=0x7f1c24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.631+0000 7f1c1bfff640 1 -- 172.21.15.43:0/1336418107 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1c28019030 con 0x7f1c34151b50 2024-03-30T13:11:56.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.791+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c3415fba0 con 0x7f1c34151b50 2024-03-30T13:11:56.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.791+0000 7f1c1bfff640 1 -- 172.21.15.43:0/1336418107 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f1c28057330 con 0x7f1c34151b50 2024-03-30T13:11:56.800 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c20067ff0 msgr2=0x7f1c2006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c20067ff0 0x7f1c2006a4b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1c3415d2b0 tx=0x7f1c24009040 comp rx=0 tx=0).stop 2024-03-30T13:11:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c34151b50 msgr2=0x7f1c3414f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c34151b50 0x7f1c3414f930 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1c28011590 tx=0x7f1c28011a50 comp rx=0 tx=0).stop 2024-03-30T13:11:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 shutdown_connections 2024-03-30T13:11:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c20067ff0 0x7f1c2006a4b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c34151460 0x7f1c3415b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c34151b50 0x7f1c3414f930 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 --2- 172.21.15.43:0/1336418107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c34151080 0x7f1c34150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 >> 172.21.15.43:0/1336418107 conn(0x7f1c340d1070 msgr2=0x7f1c340733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 shutdown_connections 2024-03-30T13:11:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.795+0000 7f1c3a1e8640 1 -- 172.21.15.43:0/1336418107 wait complete. 2024-03-30T13:11:56.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.931+0000 7f75ae106640 1 Processor -- start 2024-03-30T13:11:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.931+0000 7f75ae106640 1 -- start start 2024-03-30T13:11:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75ae106640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75a815f880 0x7f75a80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75ae106640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75a80ce4d0 con 0x7f75a8070640 2024-03-30T13:11:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75ae106640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75a80cc7f0 con 0x7f75a806f090 2024-03-30T13:11:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75ae106640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f75a8070ad0 con 0x7f75a815f880 2024-03-30T13:11:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a6ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f75a8070640 0x7f75a806e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41348/0 (socket says 172.21.15.43:41348) 2024-03-30T13:11:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a6ffd640 1 -- 172.21.15.43:0/2428591579 learned_addr learned my addr 172.21.15.43:0/2428591579 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:56.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a77fe640 1 --2- 172.21.15.43:0/2428591579 >> v2:172.21.15.113:3300/0 conn(0x7f75a815f880 0x7f75a80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1779913166 0 0) 0x7f75a80ce4d0 con 0x7f75a8070640 2024-03-30T13:11:56.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7594003530 con 0x7f75a8070640 2024-03-30T13:11:56.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3701102634 0 0) 0x7f7594003530 con 0x7f75a8070640 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 >> v2:172.21.15.113:3300/0 conn(0x7f75a815f880 msgr2=0x7f75a80b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 --2- 172.21.15.43:0/2428591579 >> v2:172.21.15.113:3300/0 conn(0x7f75a815f880 0x7f75a80b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 >> v1:172.21.15.113:6789/0 conn(0x7f75a806f090 legacy=0x7f75a80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a80ccad0 con 0x7f75a8070640 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f75900016a0 con 0x7f75a8070640 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 >> v1:172.21.15.43:6789/0 conn(0x7f75a8070640 legacy=0x7f75a806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75940039c0 0x7f7594003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7594004810 0x7f7594004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f759400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75a8070ad0 con 0x7f75940052f0 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f759400c1c0 con 0x7f75940039c0 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f759400c3c0 con 0x7f7594004810 2024-03-30T13:11:56.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a7fff640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f759400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a77fe640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75940039c0 0x7f7594003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a6ffd640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7594004810 0x7f7594004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a7fff640 1 -- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7594004810 msgr2=0x7f7594004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a7fff640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7594004810 0x7f7594004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a7fff640 1 -- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75940039c0 msgr2=0x7f7594003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a7fff640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75940039c0 0x7f7594003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a7fff640 1 -- 172.21.15.43:0/2428591579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7594003530 con 0x7f75940052f0 2024-03-30T13:11:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.935+0000 7f75a6ffd640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7594004810 0x7f7594004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a7fff640 1 -- 172.21.15.43:0/2428591579 reap_dead start 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a7fff640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f759400b9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75a80705f0 tx=0x7f75a001c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f3494c20d77a59 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0001a90 con 0x7f75940052f0 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a0003b80 con 0x7f75940052f0 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a67fc640 1 -- 172.21.15.43:0/2428591579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a00050d0 con 0x7f75940052f0 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- 172.21.15.43:0/2428591579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 msgr2=0x7f759400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f759400b9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f75a80705f0 tx=0x7f75a001c720 comp rx=0 tx=0).stop 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- 172.21.15.43:0/2428591579 shutdown_connections 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 --2- 172.21.15.43:0/2428591579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f759400b9d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- 172.21.15.43:0/2428591579 >> 172.21.15.43:0/2428591579 conn(0x7f75a80b3cf0 msgr2=0x7f75a80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- 172.21.15.43:0/2428591579 shutdown_connections 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- 172.21.15.43:0/2428591579 wait complete. 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 Processor -- start 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- start start 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f75a80b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a80c8160 0x7f75a8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75a81f8d80 0x7f75a80d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80cf650 con 0x7f75940052f0 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75a80cf4d0 con 0x7f75a80c8160 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75ae106640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f75a8160850 con 0x7f75a81f8d80 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f75a80b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f75a80b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43262/0 (socket says 172.21.15.43:43262) 2024-03-30T13:11:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 -- 172.21.15.43:0/517977860 learned_addr learned my addr 172.21.15.43:0/517977860 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a7fff640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75a81f8d80 0x7f75a80d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a6ffd640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a80c8160 0x7f75a8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 -- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75a81f8d80 msgr2=0x7f75a80d0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75a81f8d80 0x7f75a80d0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 -- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a80c8160 msgr2=0x7f75a8154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a80c8160 0x7f75a8154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 -- 172.21.15.43:0/517977860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a80c85e0 con 0x7f75940052f0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a6ffd640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a80c8160 0x7f75a8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f75a77fe640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f75a80b1ed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7598011560 tx=0x7f7598011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f759ffff640 1 -- 172.21.15.43:0/517977860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7598015930 con 0x7f75940052f0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.939+0000 7f759ffff640 1 -- 172.21.15.43:0/517977860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7598015ad0 con 0x7f75940052f0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f759ffff640 1 -- 172.21.15.43:0/517977860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759800f460 con 0x7f75940052f0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a80b24b0 con 0x7f75940052f0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a80b2a70 con 0x7f75940052f0 2024-03-30T13:11:56.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a8068900 con 0x7f75940052f0 2024-03-30T13:11:56.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f759ffff640 1 -- 172.21.15.43:0/517977860 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f759800f600 con 0x7f75940052f0 2024-03-30T13:11:56.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f759ffff640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7588067ff0 0x7f758806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:56.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f759ffff640 1 -- 172.21.15.43:0/517977860 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f7598087110 con 0x7f75940052f0 2024-03-30T13:11:56.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f75a6ffd640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7588067ff0 0x7f758806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:56.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.943+0000 7f75a6ffd640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7588067ff0 0x7f758806a4b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7590001810 tx=0x7f7590009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:56.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:56.947+0000 7f759ffff640 1 -- 172.21.15.43:0/517977860 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7598019030 con 0x7f75940052f0 2024-03-30T13:11:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.111+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75a8160350 con 0x7f75940052f0 2024-03-30T13:11:57.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.111+0000 7f759ffff640 1 -- 172.21.15.43:0/517977860 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f7598057550 con 0x7f75940052f0 2024-03-30T13:11:57.119 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7588067ff0 msgr2=0x7f758806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7588067ff0 0x7f758806a4b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7590001810 tx=0x7f7590009040 comp rx=0 tx=0).stop 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 msgr2=0x7f75a80b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f75a80b1ed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7598011560 tx=0x7f7598011a20 comp rx=0 tx=0).stop 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 shutdown_connections 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7588067ff0 0x7f758806a4b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75a81f8d80 0x7f75a80d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75a80c8160 0x7f75a8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 --2- 172.21.15.43:0/517977860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75940052f0 0x7f75a80b1ed0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 >> 172.21.15.43:0/517977860 conn(0x7f75a80b3cf0 msgr2=0x7f75a815df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 shutdown_connections 2024-03-30T13:11:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.115+0000 7f75ae106640 1 -- 172.21.15.43:0/517977860 wait complete. 2024-03-30T13:11:57.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:57.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1c07e6640 1 Processor -- start 2024-03-30T13:11:57.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1c07e6640 1 -- start start 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1c07e6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb1b815fe20 0x7fb1b8160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1c07e6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1b80ccaa0 0x7fb1b80cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1c07e6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b80cd410 con 0x7fb1b815c160 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1c07e6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b81607d0 con 0x7fb1b80ccaa0 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1c07e6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b8160c80 con 0x7fb1b815fe20 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bed83640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1b80ccaa0 0x7fb1b80cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bed83640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb1b80ccaa0 0x7fb1b80cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43268/0 (socket says 172.21.15.43:43268) 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bed83640 1 -- 172.21.15.43:0/2071122582 learned_addr learned my addr 172.21.15.43:0/2071122582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bdd81640 1 --2- 172.21.15.43:0/2071122582 >> v2:172.21.15.113:3300/0 conn(0x7fb1b815fe20 0x7fb1b8160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bed83640 1 -- 172.21.15.43:0/2071122582 >> v2:172.21.15.113:3300/0 conn(0x7fb1b815fe20 msgr2=0x7fb1b8160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bed83640 1 --2- 172.21.15.43:0/2071122582 >> v2:172.21.15.113:3300/0 conn(0x7fb1b815fe20 0x7fb1b8160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bed83640 1 -- 172.21.15.43:0/2071122582 >> v1:172.21.15.113:6789/0 conn(0x7fb1b815c160 legacy=0x7fb1b806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.267+0000 7fb1bed83640 1 -- 172.21.15.43:0/2071122582 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b80cd680 con 0x7fb1b80ccaa0 2024-03-30T13:11:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1bed83640 1 --2- 172.21.15.43:0/2071122582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b80ccaa0 0x7fb1b80cced0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8142bf0 tx=0x7fb1b401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd6dadbb21b4edc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1bd580640 1 -- 172.21.15.43:0/2071122582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40aa610 con 0x7fb1b80ccaa0 2024-03-30T13:11:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1bd580640 1 -- 172.21.15.43:0/2071122582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b40af070 con 0x7fb1b80ccaa0 2024-03-30T13:11:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1bd580640 1 -- 172.21.15.43:0/2071122582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40aa9d0 con 0x7fb1b80ccaa0 2024-03-30T13:11:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2071122582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b80ccaa0 msgr2=0x7fb1b80cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2071122582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b80ccaa0 0x7fb1b80cced0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8142bf0 tx=0x7fb1b401c4b0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2071122582 shutdown_connections 2024-03-30T13:11:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2071122582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b80ccaa0 0x7fb1b80cced0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2071122582 >> v2:172.21.15.113:3300/0 conn(0x7fb1b815fe20 0x7fb1b8160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2071122582 >> 172.21.15.43:0/2071122582 conn(0x7fb1b80d0c90 msgr2=0x7fb1b80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2071122582 shutdown_connections 2024-03-30T13:11:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2071122582 wait complete. 2024-03-30T13:11:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 Processor -- start 2024-03-30T13:11:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- start start 2024-03-30T13:11:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b80ccaa0 0x7fb1b8148870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b815c160 0x7fb1b8154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1b815fe20 0x7fb1b81561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80cf420 con 0x7fb1b815c160 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80cf2a0 con 0x7fb1b80ccaa0 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1c07e6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b815d550 con 0x7fb1b815fe20 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1bdd81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b815c160 0x7fb1b8154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1bdd81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b815c160 0x7fb1b8154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43276/0 (socket says 172.21.15.43:43276) 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1bdd81640 1 -- 172.21.15.43:0/2934125126 learned_addr learned my addr 172.21.15.43:0/2934125126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.271+0000 7fb1be582640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b80ccaa0 0x7fb1b8148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bed83640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1b815fe20 0x7fb1b81561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bdd81640 1 -- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1b815fe20 msgr2=0x7fb1b81561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bdd81640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1b815fe20 0x7fb1b81561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bdd81640 1 -- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b80ccaa0 msgr2=0x7fb1b8148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bdd81640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b80ccaa0 0x7fb1b8148870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bdd81640 1 -- 172.21.15.43:0/2934125126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b8156c60 con 0x7fb1b815c160 2024-03-30T13:11:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bed83640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1b815fe20 0x7fb1b81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1bdd81640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b815c160 0x7fb1b8154c60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800d5f0 tx=0x7fb1a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1aeffd640 1 -- 172.21.15.43:0/2934125126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80125f0 con 0x7fb1b815c160 2024-03-30T13:11:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1aeffd640 1 -- 172.21.15.43:0/2934125126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a8019070 con 0x7fb1b815c160 2024-03-30T13:11:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1aeffd640 1 -- 172.21.15.43:0/2934125126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a80129b0 con 0x7fb1b815c160 2024-03-30T13:11:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b8154180 con 0x7fb1b815c160 2024-03-30T13:11:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b8154740 con 0x7fb1b815c160 2024-03-30T13:11:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b8068900 con 0x7fb1b815c160 2024-03-30T13:11:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1aeffd640 1 -- 172.21.15.43:0/2934125126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb1a8012b50 con 0x7fb1b815c160 2024-03-30T13:11:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1aeffd640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb19c067ff0 0x7fb19c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1aeffd640 1 -- 172.21.15.43:0/2934125126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fb1a8086be0 con 0x7fb1b815c160 2024-03-30T13:11:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1be582640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb19c067ff0 0x7fb19c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.275+0000 7fb1be582640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb19c067ff0 0x7fb19c06a4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80b1600 tx=0x7fb1b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.279+0000 7fb1aeffd640 1 -- 172.21.15.43:0/2934125126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb1a80570b0 con 0x7fb1b815c160 2024-03-30T13:11:57.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.443+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb1b806e040 con 0x7fb1b815c160 2024-03-30T13:11:57.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.443+0000 7fb1aeffd640 1 -- 172.21.15.43:0/2934125126 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fb1a80569b0 con 0x7fb1b815c160 2024-03-30T13:11:57.451 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:57.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb19c067ff0 msgr2=0x7fb19c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb19c067ff0 0x7fb19c06a4b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80b1600 tx=0x7fb1b0009040 comp rx=0 tx=0).stop 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b815c160 msgr2=0x7fb1b8154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b815c160 0x7fb1b8154c60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800d5f0 tx=0x7fb1a800dab0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 shutdown_connections 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb19c067ff0 0x7fb19c06a4b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1b815fe20 0x7fb1b81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1b815c160 0x7fb1b8154c60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 --2- 172.21.15.43:0/2934125126 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb1b80ccaa0 0x7fb1b8148870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 >> 172.21.15.43:0/2934125126 conn(0x7fb1b80d0c90 msgr2=0x7fb1b80d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 shutdown_connections 2024-03-30T13:11:57.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.447+0000 7fb1c07e6640 1 -- 172.21.15.43:0/2934125126 wait complete. 2024-03-30T13:11:57.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:11:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 Processor -- start 2024-03-30T13:11:57.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 -- start start 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc5415e400 0x7fbc540b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5406d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5406dcc0 0x7fbc5406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbc5406f190 con 0x7fbc5406dcc0 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbc540b08a0 con 0x7fbc5415e400 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc58eec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbc5406f3e0 con 0x7fbc540ad8e0 2024-03-30T13:11:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5406dcc0 0x7fbc5406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5406dcc0 0x7fbc5406b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43290/0 (socket says 172.21.15.43:43290) 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 -- 172.21.15.43:0/4003756580 learned_addr learned my addr 172.21.15.43:0/4003756580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc5259c640 1 --2- 172.21.15.43:0/4003756580 >> v2:172.21.15.113:3300/0 conn(0x7fbc5415e400 0x7fbc540b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc4bfff640 1 --2- 172.21.15.43:0/4003756580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 -- 172.21.15.43:0/4003756580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 msgr2=0x7fbc5406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 --2- 172.21.15.43:0/4003756580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 -- 172.21.15.43:0/4003756580 >> v2:172.21.15.113:3300/0 conn(0x7fbc5415e400 msgr2=0x7fbc540b0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 --2- 172.21.15.43:0/4003756580 >> v2:172.21.15.113:3300/0 conn(0x7fbc5415e400 0x7fbc540b0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.591+0000 7fbc52d9d640 1 -- 172.21.15.43:0/4003756580 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5406f6c0 con 0x7fbc5406dcc0 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc52d9d640 1 --2- 172.21.15.43:0/4003756580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5406dcc0 0x7fbc5406b660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbc540b49d0 tx=0x7fbc4c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbf82bcaf43954f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc51d9b640 1 -- 172.21.15.43:0/4003756580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0ac070 con 0x7fbc5406dcc0 2024-03-30T13:11:57.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc51d9b640 1 -- 172.21.15.43:0/4003756580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0a8420 con 0x7fbc5406dcc0 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc51d9b640 1 -- 172.21.15.43:0/4003756580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0a87f0 con 0x7fbc5406dcc0 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- 172.21.15.43:0/4003756580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5406dcc0 msgr2=0x7fbc5406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 --2- 172.21.15.43:0/4003756580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5406dcc0 0x7fbc5406b660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbc540b49d0 tx=0x7fbc4c01c730 comp rx=0 tx=0).stop 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- 172.21.15.43:0/4003756580 shutdown_connections 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 --2- 172.21.15.43:0/4003756580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5406dcc0 0x7fbc5406b660 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 --2- 172.21.15.43:0/4003756580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 --2- 172.21.15.43:0/4003756580 >> v2:172.21.15.113:3300/0 conn(0x7fbc5415e400 0x7fbc540b0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- 172.21.15.43:0/4003756580 >> 172.21.15.43:0/4003756580 conn(0x7fbc54072500 msgr2=0x7fbc54072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- 172.21.15.43:0/4003756580 shutdown_connections 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- 172.21.15.43:0/4003756580 wait complete. 2024-03-30T13:11:57.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 Processor -- start 2024-03-30T13:11:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- start start 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc5406dcc0 0x7fbc54157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5414ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5415e400 0x7fbc540ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbc540affe0 con 0x7fbc5415e400 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbc540afe60 con 0x7fbc5406dcc0 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc58eec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbc540b54e0 con 0x7fbc540ad8e0 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc52d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5415e400 0x7fbc540ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc52d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5415e400 0x7fbc540ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43292/0 (socket says 172.21.15.43:43292) 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc52d9d640 1 -- 172.21.15.43:0/3385597627 learned_addr learned my addr 172.21.15.43:0/3385597627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc5259c640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc5406dcc0 0x7fbc54157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.595+0000 7fbc4bfff640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5414ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc52d9d640 1 -- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 msgr2=0x7fbc5414ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc52d9d640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5414ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc52d9d640 1 -- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc5406dcc0 msgr2=0x7fbc54157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc52d9d640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc5406dcc0 0x7fbc54157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc52d9d640 1 -- 172.21.15.43:0/3385597627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5414efc0 con 0x7fbc5415e400 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc52d9d640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5415e400 0x7fbc540ae5d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c01cfd0 tx=0x7fbc4c0194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc4affd640 1 -- 172.21.15.43:0/3385597627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0bf070 con 0x7fbc5415e400 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc4affd640 1 -- 172.21.15.43:0/3385597627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0a8e40 con 0x7fbc5415e400 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc540cd9c0 con 0x7fbc5415e400 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc4affd640 1 -- 172.21.15.43:0/3385597627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0ac040 con 0x7fbc5415e400 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc540cdf80 con 0x7fbc5415e400 2024-03-30T13:11:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc54068890 con 0x7fbc5415e400 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc4affd640 1 -- 172.21.15.43:0/3385597627 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbc4c0c2070 con 0x7fbc5415e400 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc4affd640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc34067fc0 0x7fbc3406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc4affd640 1 -- 172.21.15.43:0/3385597627 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fbc4c120f80 con 0x7fbc5415e400 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc5259c640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc34067fc0 0x7fbc3406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.599+0000 7fbc5259c640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc34067fc0 0x7fbc3406a480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00d8d0 tx=0x7fbc3c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.603+0000 7fbc4affd640 1 -- 172.21.15.43:0/3385597627 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbc4c0165e0 con 0x7fbc5415e400 2024-03-30T13:11:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbc540b0390 con 0x7fbc5415e400 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc4affd640 1 -- 172.21.15.43:0/3385597627 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fbc4c0f0dd0 con 0x7fbc5415e400 2024-03-30T13:11:57.782 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc34067fc0 msgr2=0x7fbc3406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc34067fc0 0x7fbc3406a480 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00d8d0 tx=0x7fbc3c008040 comp rx=0 tx=0).stop 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5415e400 msgr2=0x7fbc540ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5415e400 0x7fbc540ae5d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c01cfd0 tx=0x7fbc4c0194d0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 shutdown_connections 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc34067fc0 0x7fbc3406a480 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5415e400 0x7fbc540ae5d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc540ad8e0 0x7fbc5414ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 --2- 172.21.15.43:0/3385597627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc5406dcc0 0x7fbc54157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 >> 172.21.15.43:0/3385597627 conn(0x7fbc54072500 msgr2=0x7fbc540728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 shutdown_connections 2024-03-30T13:11:57.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.775+0000 7fbc58eec640 1 -- 172.21.15.43:0/3385597627 wait complete. 2024-03-30T13:11:57.800 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:11:57.800 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:57.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd32f8640 1 Processor -- start 2024-03-30T13:11:57.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd32f8640 1 -- start start 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd32f8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcdcc15aa10 0x7fcdcc0cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd32f8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc06bd70 0x7fcdcc15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd32f8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdcc06c1e0 con 0x7fcdcc0b05c0 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd32f8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcdcc0ad2f0 con 0x7fcdcc15aa10 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd32f8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcdcc0cca30 con 0x7fcdcc06bd70 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd0893640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcdcc0b05c0 0x7fcdcc06b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41364/0 (socket says 172.21.15.43:41364) 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd0893640 1 -- 172.21.15.43:0/2823148970 learned_addr learned my addr 172.21.15.43:0/2823148970 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd1094640 1 --2- 172.21.15.43:0/2823148970 >> v2:172.21.15.113:3300/0 conn(0x7fcdcc15aa10 0x7fcdcc0cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd1895640 1 --2- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc06bd70 0x7fcdcc15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdc3fff640 1 -- 172.21.15.43:0/2823148970 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3720174283 0 0) 0x7fcdcc06c1e0 con 0x7fcdcc0b05c0 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdc3fff640 1 -- 172.21.15.43:0/2823148970 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdb8003530 con 0x7fcdcc0b05c0 2024-03-30T13:11:57.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdc3fff640 1 -- 172.21.15.43:0/2823148970 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 788061853 0 0) 0x7fcdb8003530 con 0x7fcdcc0b05c0 2024-03-30T13:11:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd1094640 1 -- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc06bd70 msgr2=0x7fcdcc15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd1094640 1 --2- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc06bd70 0x7fcdcc15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd1094640 1 -- 172.21.15.43:0/2823148970 >> v1:172.21.15.43:6789/0 conn(0x7fcdcc0b05c0 legacy=0x7fcdcc06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.963+0000 7fcdd1094640 1 -- 172.21.15.43:0/2823148970 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc0737e0 con 0x7fcdcc15aa10 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd1094640 1 --2- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc15aa10 0x7fcdcc0cc4f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80097c0 tx=0x7fcdc801ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573c52a515bbb2c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdc3fff640 1 -- 172.21.15.43:0/2823148970 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80b5070 con 0x7fcdcc15aa10 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdc3fff640 1 -- 172.21.15.43:0/2823148970 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc80a85f0 con 0x7fcdcc15aa10 2024-03-30T13:11:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdc3fff640 1 -- 172.21.15.43:0/2823148970 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc001680 con 0x7fcdcc15aa10 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc15aa10 msgr2=0x7fcdcc0cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc15aa10 0x7fcdcc0cc4f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcdc80097c0 tx=0x7fcdc801ebb0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2823148970 shutdown_connections 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc06bd70 0x7fcdcc15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2823148970 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc15aa10 0x7fcdcc0cc4f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2823148970 >> 172.21.15.43:0/2823148970 conn(0x7fcdcc0b5060 msgr2=0x7fcdcc0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2823148970 shutdown_connections 2024-03-30T13:11:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2823148970 wait complete. 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 Processor -- start 2024-03-30T13:11:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.967+0000 7fcdd32f8640 1 -- start start 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc06bd70 0x7fcdcc1445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc0b05c0 0x7fcdcc14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc15aa10 0x7fcdcc09a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc0b0130 con 0x7fcdcc06bd70 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc0afb70 con 0x7fcdcc0b05c0 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcdcc0afdc0 con 0x7fcdcc15aa10 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc06bd70 0x7fcdcc1445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc06bd70 0x7fcdcc1445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43306/0 (socket says 172.21.15.43:43306) 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 -- 172.21.15.43:0/2906205893 learned_addr learned my addr 172.21.15.43:0/2906205893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd0893640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc0b05c0 0x7fcdcc14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1895640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc15aa10 0x7fcdcc09a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 -- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc15aa10 msgr2=0x7fcdcc09a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc15aa10 0x7fcdcc09a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 -- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc0b05c0 msgr2=0x7fcdcc14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc0b05c0 0x7fcdcc14d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 -- 172.21.15.43:0/2906205893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc14c460 con 0x7fcdcc06bd70 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1895640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc15aa10 0x7fcdcc09a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd1094640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc06bd70 0x7fcdcc1445f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdc801d260 tx=0x7fcdc801b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdc1ffb640 1 -- 172.21.15.43:0/2906205893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80bf070 con 0x7fcdcc06bd70 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdc1ffb640 1 -- 172.21.15.43:0/2906205893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc80a89a0 con 0x7fcdcc06bd70 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc14c740 con 0x7fcdcc06bd70 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdc1ffb640 1 -- 172.21.15.43:0/2906205893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80b5040 con 0x7fcdcc06bd70 2024-03-30T13:11:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc14ad40 con 0x7fcdcc06bd70 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdcc068890 con 0x7fcdcc06bd70 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdc1ffb640 1 -- 172.21.15.43:0/2906205893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcdc80c2070 con 0x7fcdcc06bd70 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.971+0000 7fcdc1ffb640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcdb0067fc0 0x7fcdb006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.975+0000 7fcdc1ffb640 1 -- 172.21.15.43:0/2906205893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fcdc8123010 con 0x7fcdcc06bd70 2024-03-30T13:11:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.975+0000 7fcdd0893640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcdb0067fc0 0x7fcdb006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.975+0000 7fcdd0893640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcdb0067fc0 0x7fcdb006a480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc1497a0 tx=0x7fcdbc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:57.979+0000 7fcdc1ffb640 1 -- 172.21.15.43:0/2906205893 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcdc8018710 con 0x7fcdcc06bd70 2024-03-30T13:11:58.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.139+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcdcc167620 con 0x7fcdcc06bd70 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdc1ffb640 1 -- 172.21.15.43:0/2906205893 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7fcdc80f3450 con 0x7fcdcc06bd70 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:11:58.150 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcdb0067fc0 msgr2=0x7fcdb006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcdb0067fc0 0x7fcdb006a480 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc1497a0 tx=0x7fcdbc009040 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc06bd70 msgr2=0x7fcdcc1445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc06bd70 0x7fcdcc1445f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcdc801d260 tx=0x7fcdc801b950 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 shutdown_connections 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcdb0067fc0 0x7fcdb006a480 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdcc15aa10 0x7fcdcc09a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.143+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdcc0b05c0 0x7fcdcc14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.147+0000 7fcdd32f8640 1 --2- 172.21.15.43:0/2906205893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdcc06bd70 0x7fcdcc1445f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.147+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 >> 172.21.15.43:0/2906205893 conn(0x7fcdcc0b5060 msgr2=0x7fcdcc0b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.147+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 shutdown_connections 2024-03-30T13:11:58.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.147+0000 7fcdd32f8640 1 -- 172.21.15.43:0/2906205893 wait complete. 2024-03-30T13:11:58.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:11:58.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:11:58.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fbe4a640 1 Processor -- start 2024-03-30T13:11:58.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fbe4a640 1 -- start start 2024-03-30T13:11:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fbe4a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21f406c4d0 0x7f21f40ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fbe4a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f40b5470 con 0x7f21f4147e00 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fbe4a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21f40b5750 con 0x7f21f406c070 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fbe4a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f21f40b5a30 con 0x7f21f406c4d0 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fa3e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21f406c4d0 0x7f21f40ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21f9be6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f21f4147e00 0x7f21f40b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41374/0 (socket says 172.21.15.43:41374) 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21f9be6640 1 -- 172.21.15.43:0/1448658066 learned_addr learned my addr 172.21.15.43:0/1448658066 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fa3e7640 1 -- 172.21.15.43:0/1448658066 >> v1:172.21.15.113:6789/0 conn(0x7f21f406c070 legacy=0x7f21f40adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fa3e7640 1 -- 172.21.15.43:0/1448658066 >> v1:172.21.15.43:6789/0 conn(0x7f21f4147e00 legacy=0x7f21f40b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21f8be4640 1 -- 172.21.15.43:0/1448658066 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1045023213 0 0) 0x7f21f40b5470 con 0x7f21f4147e00 2024-03-30T13:11:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.275+0000 7f21fa3e7640 1 -- 172.21.15.43:0/1448658066 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f40b5d10 con 0x7f21f406c4d0 2024-03-30T13:11:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fa3e7640 1 --2- 172.21.15.43:0/1448658066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c4d0 0x7f21f40ae2d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0097c0 tx=0x7f21ec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a63770186b0465 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f8be4640 1 -- 172.21.15.43:0/1448658066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0b0070 con 0x7f21f406c4d0 2024-03-30T13:11:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f8be4640 1 -- 172.21.15.43:0/1448658066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec0ab600 con 0x7f21f406c4d0 2024-03-30T13:11:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f8be4640 1 -- 172.21.15.43:0/1448658066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0ab9d0 con 0x7f21f406c4d0 2024-03-30T13:11:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- 172.21.15.43:0/1448658066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c4d0 msgr2=0x7f21f40ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/1448658066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c4d0 0x7f21f40ae2d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0097c0 tx=0x7f21ec01c460 comp rx=0 tx=0).stop 2024-03-30T13:11:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- 172.21.15.43:0/1448658066 shutdown_connections 2024-03-30T13:11:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/1448658066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c4d0 0x7f21f40ae2d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- 172.21.15.43:0/1448658066 >> 172.21.15.43:0/1448658066 conn(0x7f21f40d1070 msgr2=0x7f21f40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- 172.21.15.43:0/1448658066 shutdown_connections 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- 172.21.15.43:0/1448658066 wait complete. 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 Processor -- start 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- start start 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c070 0x7f21f413fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f406c4d0 0x7f21f413ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f4147e00 0x7f21f4160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f21f416c2b0 con 0x7f21f406c070 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f21f416c130 con 0x7f21f406c4d0 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21fbe4a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f21f416c430 con 0x7f21f4147e00 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c070 0x7f21f413fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c070 0x7f21f413fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43326/0 (socket says 172.21.15.43:43326) 2024-03-30T13:11:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 -- 172.21.15.43:0/4197811678 learned_addr learned my addr 172.21.15.43:0/4197811678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f93e5640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f406c4d0 0x7f21f413ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 -- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f4147e00 msgr2=0x7f21f4160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f4147e00 0x7f21f4160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 -- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f406c4d0 msgr2=0x7f21f413ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f406c4d0 0x7f21f413ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f9be6640 1 -- 172.21.15.43:0/4197811678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21f4147230 con 0x7f21f406c070 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.279+0000 7f21f93e5640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f406c4d0 0x7f21f413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21f9be6640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c070 0x7f21f413fec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c7c0 tx=0x7f21f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21ea7fc640 1 -- 172.21.15.43:0/4197811678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0018070 con 0x7f21f406c070 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21ea7fc640 1 -- 172.21.15.43:0/4197811678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f000a640 con 0x7f21f406c070 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21ea7fc640 1 -- 172.21.15.43:0/4197811678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f000aa30 con 0x7f21f406c070 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21f4144510 con 0x7f21f406c070 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21f4143a50 con 0x7f21f406c070 2024-03-30T13:11:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21f4144100 con 0x7f21f406c070 2024-03-30T13:11:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21ea7fc640 1 -- 172.21.15.43:0/4197811678 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f21f001b070 con 0x7f21f406c070 2024-03-30T13:11:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21ea7fc640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f21d8067fc0 0x7f21d806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21ea7fc640 1 -- 172.21.15.43:0/4197811678 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f21f0085e20 con 0x7f21f406c070 2024-03-30T13:11:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21f93e5640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f21d8067fc0 0x7f21d806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.283+0000 7f21f93e5640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f21d8067fc0 0x7f21d806a480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21f413fc80 tx=0x7f21e4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.287+0000 7f21ea7fc640 1 -- 172.21.15.43:0/4197811678 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f21f00562f0 con 0x7f21f406c070 2024-03-30T13:11:58.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.443+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21f4070a90 con 0x7f21f406c070 2024-03-30T13:11:58.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.443+0000 7f21ea7fc640 1 -- 172.21.15.43:0/4197811678 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f21f0055bf0 con 0x7f21f406c070 2024-03-30T13:11:58.452 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f21d8067fc0 msgr2=0x7f21d806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f21d8067fc0 0x7f21d806a480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21f413fc80 tx=0x7f21e4009040 comp rx=0 tx=0).stop 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c070 msgr2=0x7f21f413fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c070 0x7f21f413fec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f21f000c7c0 tx=0x7f21f000cc80 comp rx=0 tx=0).stop 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 shutdown_connections 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f21d8067fc0 0x7f21d806a480 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f21f4147e00 0x7f21f4160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f21f406c4d0 0x7f21f413ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 --2- 172.21.15.43:0/4197811678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21f406c070 0x7f21f413fec0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 >> 172.21.15.43:0/4197811678 conn(0x7f21f40d1070 msgr2=0x7f21f40b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 shutdown_connections 2024-03-30T13:11:58.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.447+0000 7f21fbe4a640 1 -- 172.21.15.43:0/4197811678 wait complete. 2024-03-30T13:11:58.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:11:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67d169d640 1 Processor -- start 2024-03-30T13:11:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67d169d640 1 -- start start 2024-03-30T13:11:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67d169d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f67cc15e4c0 0x7f67cc0b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67d169d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67cc0cf6c0 0x7f67cc0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67d169d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67cc06c850 con 0x7f67cc06c070 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67d169d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc15e8a0 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67d169d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc06cb30 con 0x7f67cc15e4c0 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67cc0cf6c0 0x7f67cc0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67cc0cf6c0 0x7f67cc0d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43328/0 (socket says 172.21.15.43:43328) 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 -- 172.21.15.43:0/2229250552 learned_addr learned my addr 172.21.15.43:0/2229250552 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67caffd640 1 --2- 172.21.15.43:0/2229250552 >> v2:172.21.15.113:3300/0 conn(0x7f67cc15e4c0 0x7f67cc0b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 -- 172.21.15.43:0/2229250552 >> v2:172.21.15.113:3300/0 conn(0x7f67cc15e4c0 msgr2=0x7f67cc0b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 --2- 172.21.15.43:0/2229250552 >> v2:172.21.15.113:3300/0 conn(0x7f67cc15e4c0 0x7f67cc0b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 -- 172.21.15.43:0/2229250552 >> v1:172.21.15.113:6789/0 conn(0x7f67cc06c070 legacy=0x7f67cc0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 -- 172.21.15.43:0/2229250552 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc06ce10 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.587+0000 7f67cb7fe640 1 --2- 172.21.15.43:0/2229250552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc0d2d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0d0cc0 tx=0x7f67c401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f9f3676dd11838 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67c9ffb640 1 -- 172.21.15.43:0/2229250552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40a5670 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67c9ffb640 1 -- 172.21.15.43:0/2229250552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c40a5810 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67c9ffb640 1 -- 172.21.15.43:0/2229250552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40a5be0 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- 172.21.15.43:0/2229250552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 msgr2=0x7f67cc0d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 --2- 172.21.15.43:0/2229250552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc0d2d70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0d0cc0 tx=0x7f67c401c9b0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- 172.21.15.43:0/2229250552 shutdown_connections 2024-03-30T13:11:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 --2- 172.21.15.43:0/2229250552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc0d2d70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 --2- 172.21.15.43:0/2229250552 >> v2:172.21.15.113:3300/0 conn(0x7f67cc15e4c0 0x7f67cc0b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- 172.21.15.43:0/2229250552 >> 172.21.15.43:0/2229250552 conn(0x7f67cc0b6410 msgr2=0x7f67cc0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- 172.21.15.43:0/2229250552 shutdown_connections 2024-03-30T13:11:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- 172.21.15.43:0/2229250552 wait complete. 2024-03-30T13:11:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 Processor -- start 2024-03-30T13:11:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- start start 2024-03-30T13:11:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67cc06c070 0x7f67cc158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67cc15e4c0 0x7f67cc1fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc0d3e90 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc0d3d10 con 0x7f67cc06c070 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc0d4010 con 0x7f67cc15e4c0 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43334/0 (socket says 172.21.15.43:43334) 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 -- 172.21.15.43:0/613658767 learned_addr learned my addr 172.21.15.43:0/613658767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67caffd640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67cc06c070 0x7f67cc158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67cb7fe640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67cc15e4c0 0x7f67cc1fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 -- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67cc15e4c0 msgr2=0x7f67cc1fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67cc15e4c0 0x7f67cc1fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 -- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67cc06c070 msgr2=0x7f67cc158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67cc06c070 0x7f67cc158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 -- 172.21.15.43:0/613658767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc152d60 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67caffd640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67cc06c070 0x7f67cc158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67ca7fc640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc159e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f67c000c480 tx=0x7f67c000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67b37fe640 1 -- 172.21.15.43:0/613658767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0012830 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67b37fe640 1 -- 172.21.15.43:0/613658767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67c0017070 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc153040 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67b37fe640 1 -- 172.21.15.43:0/613658767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0012c30 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.591+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc153600 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.595+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67cc068900 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.595+0000 7f67b37fe640 1 -- 172.21.15.43:0/613658767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f67c001c460 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.595+0000 7f67b37fe640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f67b8067fc0 0x7f67b806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.595+0000 7f67b37fe640 1 -- 172.21.15.43:0/613658767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f67c00866e0 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.595+0000 7f67caffd640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f67b8067fc0 0x7f67b806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.595+0000 7f67caffd640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f67b8067fc0 0x7f67b806a480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0097a0 tx=0x7f67bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.599+0000 7f67b37fe640 1 -- 172.21.15.43:0/613658767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f67c0056b50 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.755+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f67cc0b9d00 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67b37fe640 1 -- 172.21.15.43:0/613658767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f67c0056450 con 0x7f67cc0cf6c0 2024-03-30T13:11:58.765 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f67b8067fc0 msgr2=0x7f67b806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f67b8067fc0 0x7f67b806a480 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0097a0 tx=0x7f67bc008040 comp rx=0 tx=0).stop 2024-03-30T13:11:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 msgr2=0x7f67cc159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc159e20 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f67c000c480 tx=0x7f67c000c940 comp rx=0 tx=0).stop 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 shutdown_connections 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f67b8067fc0 0x7f67b806a480 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67cc15e4c0 0x7f67cc1fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67cc0cf6c0 0x7f67cc159e20 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 --2- 172.21.15.43:0/613658767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67cc06c070 0x7f67cc158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 >> 172.21.15.43:0/613658767 conn(0x7f67cc0b6410 msgr2=0x7f67cc0b7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 shutdown_connections 2024-03-30T13:11:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.759+0000 7f67d169d640 1 -- 172.21.15.43:0/613658767 wait complete. 2024-03-30T13:11:58.783 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:11:58.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.895+0000 7f915012e640 1 Processor -- start 2024-03-30T13:11:58.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f915012e640 1 -- start start 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f915012e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f914806fd20 0x7f914806eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f915012e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914806dc00 con 0x7f914815bd70 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f915012e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f914806dee0 con 0x7f91480afb90 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f915012e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f914806e1c0 con 0x7f914806fd20 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f914deca640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f914815bd70 0x7f914815c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41382/0 (socket says 172.21.15.43:41382) 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f914deca640 1 -- 172.21.15.43:0/1386580433 learned_addr learned my addr 172.21.15.43:0/1386580433 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f914e6cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f914806fd20 0x7f914806eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f914e6cb640 1 -- 172.21.15.43:0/1386580433 >> v1:172.21.15.113:6789/0 conn(0x7f91480afb90 legacy=0x7f914806f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:58.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f914e6cb640 1 -- 172.21.15.43:0/1386580433 >> v1:172.21.15.43:6789/0 conn(0x7f914815bd70 legacy=0x7f914815c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f914e6cb640 1 -- 172.21.15.43:0/1386580433 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914806e4a0 con 0x7f914806fd20 2024-03-30T13:11:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.899+0000 7f914cec8640 1 -- 172.21.15.43:0/1386580433 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2072656735 0 0) 0x7f914806dc00 con 0x7f914815bd70 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914e6cb640 1 --2- 172.21.15.43:0/1386580433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914806fd20 0x7f914806eca0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9140009dc0 tx=0x7f914001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9c882768ef6832 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914cec8640 1 -- 172.21.15.43:0/1386580433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91400b2070 con 0x7f914806fd20 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914cec8640 1 -- 172.21.15.43:0/1386580433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91400a5920 con 0x7f914806fd20 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914cec8640 1 -- 172.21.15.43:0/1386580433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91400a5cf0 con 0x7f914806fd20 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- 172.21.15.43:0/1386580433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914806fd20 msgr2=0x7f914806eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 --2- 172.21.15.43:0/1386580433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914806fd20 0x7f914806eca0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9140009dc0 tx=0x7f914001c950 comp rx=0 tx=0).stop 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- 172.21.15.43:0/1386580433 shutdown_connections 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 --2- 172.21.15.43:0/1386580433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f914806fd20 0x7f914806eca0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- 172.21.15.43:0/1386580433 >> 172.21.15.43:0/1386580433 conn(0x7f91480d0c90 msgr2=0x7f91480d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:58.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- 172.21.15.43:0/1386580433 shutdown_connections 2024-03-30T13:11:58.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- 172.21.15.43:0/1386580433 wait complete. 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 Processor -- start 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- start start 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914806fd20 0x7f914813a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91480afb90 0x7f91481f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f914815bd70 0x7f9148152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f91480b4400 con 0x7f91480afb90 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f91480b4280 con 0x7f914806fd20 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f91480b4580 con 0x7f914815bd70 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91480afb90 0x7f91481f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91480afb90 0x7f91481f7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43360/0 (socket says 172.21.15.43:43360) 2024-03-30T13:11:58.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 -- 172.21.15.43:0/3468898520 learned_addr learned my addr 172.21.15.43:0/3468898520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914deca640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914806fd20 0x7f914813a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914e6cb640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f914815bd70 0x7f9148152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 -- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f914815bd70 msgr2=0x7f9148152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f914815bd70 0x7f9148152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 -- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914806fd20 msgr2=0x7f914813a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914806fd20 0x7f914813a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 -- 172.21.15.43:0/3468898520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9148147c80 con 0x7f91480afb90 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914deca640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914806fd20 0x7f914813a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914e6cb640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f914815bd70 0x7f9148152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f914d6c9640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91480afb90 0x7f91481f7000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f913800c390 tx=0x7f913800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f913e7fc640 1 -- 172.21.15.43:0/3468898520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138017070 con 0x7f91480afb90 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f913e7fc640 1 -- 172.21.15.43:0/3468898520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91380052d0 con 0x7f91480afb90 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9148147f60 con 0x7f91480afb90 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f913e7fc640 1 -- 172.21.15.43:0/3468898520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138003820 con 0x7f91480afb90 2024-03-30T13:11:58.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.903+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9148148520 con 0x7f91480afb90 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.907+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9148068900 con 0x7f91480afb90 2024-03-30T13:11:58.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.907+0000 7f913e7fc640 1 -- 172.21.15.43:0/3468898520 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f913801a070 con 0x7f91480afb90 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.907+0000 7f913e7fc640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f912c067fc0 0x7f912c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.907+0000 7f913e7fc640 1 -- 172.21.15.43:0/3468898520 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f9138085c40 con 0x7f91480afb90 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.907+0000 7f914deca640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f912c067fc0 0x7f912c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:58.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.907+0000 7f914deca640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f912c067fc0 0x7f912c06a480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9144000f70 tx=0x7f9144009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:58.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:58.911+0000 7f913e7fc640 1 -- 172.21.15.43:0/3468898520 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9138005480 con 0x7f91480afb90 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.083+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f914814c2a0 con 0x7f91480afb90 2024-03-30T13:11:59.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.083+0000 7f913e7fc640 1 -- 172.21.15.43:0/3468898520 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f9138056080 con 0x7f91480afb90 2024-03-30T13:11:59.091 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:11:59.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f912c067fc0 msgr2=0x7f912c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f912c067fc0 0x7f912c06a480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9144000f70 tx=0x7f9144009040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91480afb90 msgr2=0x7f91481f7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91480afb90 0x7f91481f7000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f913800c390 tx=0x7f913800c850 comp rx=0 tx=0).stop 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 shutdown_connections 2024-03-30T13:11:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f912c067fc0 0x7f912c06a480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f914815bd70 0x7f9148152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91480afb90 0x7f91481f7000 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 --2- 172.21.15.43:0/3468898520 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f914806fd20 0x7f914813a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 >> 172.21.15.43:0/3468898520 conn(0x7f91480d0c90 msgr2=0x7f914814b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 shutdown_connections 2024-03-30T13:11:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.087+0000 7f915012e640 1 -- 172.21.15.43:0/3468898520 wait complete. 2024-03-30T13:11:59.113 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:11:59.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa35168640 1 Processor -- start 2024-03-30T13:11:59.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa35168640 1 -- start start 2024-03-30T13:11:59.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa35168640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 0x7faa3006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa35168640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa30070340 con 0x7faa300b42c0 2024-03-30T13:11:59.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa35168640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa3015e9e0 con 0x7faa3006c790 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa35168640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7faa3006bd70 con 0x7faa3006f9e0 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2f59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faa300b42c0 0x7faa300b4730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41386/0 (socket says 172.21.15.43:41386) 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2f59e640 1 -- 172.21.15.43:0/942702650 learned_addr learned my addr 172.21.15.43:0/942702650 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2e59c640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 0x7faa3006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291565954 0 0) 0x7faa30070340 con 0x7faa300b42c0 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa18003530 con 0x7faa300b42c0 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719864447 0 0) 0x7faa3015e9e0 con 0x7faa3006c790 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa30070340 con 0x7faa3006c790 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1443568603 0 0) 0x7faa18003530 con 0x7faa300b42c0 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 msgr2=0x7faa3006fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 0x7faa3006fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 >> v1:172.21.15.113:6789/0 conn(0x7faa3006c790 legacy=0x7faa3006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3015eca0 con 0x7faa300b42c0 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7faa28001980 con 0x7faa300b42c0 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 >> v1:172.21.15.43:6789/0 conn(0x7faa300b42c0 legacy=0x7faa300b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18003b80 0x7faa18003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 0x7faa18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa18004d50 0x7faa180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa3006bd70 con 0x7faa18003b80 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faa1800bab0 con 0x7faa18004d50 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7faa1800bcb0 con 0x7faa3006f9e0 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2ed9d640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18003b80 0x7faa18003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2e59c640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 0x7faa18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2f59e640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa18004d50 0x7faa180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2ed9d640 1 -- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 msgr2=0x7faa18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2ed9d640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa3006f9e0 0x7faa18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2ed9d640 1 -- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa18004d50 msgr2=0x7faa180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2ed9d640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa18004d50 0x7faa180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.227+0000 7faa2ed9d640 1 -- 172.21.15.43:0/942702650 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa18003530 con 0x7faa18003b80 2024-03-30T13:11:59.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa2ed9d640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18003b80 0x7faa18003f80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa240018d0 tx=0x7faa2401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cfafec75123b42 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24001520 con 0x7faa18003b80 2024-03-30T13:11:59.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa240052f0 con 0x7faa18003b80 2024-03-30T13:11:59.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa2dd9b640 1 -- 172.21.15.43:0/942702650 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24003840 con 0x7faa18003b80 2024-03-30T13:11:59.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa35168640 1 -- 172.21.15.43:0/942702650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18003b80 msgr2=0x7faa18003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa35168640 1 --2- 172.21.15.43:0/942702650 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa18003b80 0x7faa18003f80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7faa240018d0 tx=0x7faa2401c580 comp rx=0 tx=0).stop 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa2f59e640 1 -- 172.21.15.43:0/942702650 reap_dead start 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa35168640 1 -- 172.21.15.43:0/942702650 shutdown_connections 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa35168640 1 -- 172.21.15.43:0/942702650 >> 172.21.15.43:0/942702650 conn(0x7faa300d1070 msgr2=0x7faa300d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa35168640 1 -- 172.21.15.43:0/942702650 shutdown_connections 2024-03-30T13:11:59.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.231+0000 7faa35168640 1 -- 172.21.15.43:0/942702650 wait complete. 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 Processor -- start 2024-03-30T13:11:59.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 -- start start 2024-03-30T13:11:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa30144b80 0x7faa3015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa3015dbf0 0x7faa300ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa30144f60 0x7faa30159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7faa3006cb70 con 0x7faa3015dbf0 2024-03-30T13:11:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7faa30070b20 con 0x7faa30144b80 2024-03-30T13:11:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa35168640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7faa3006ccf0 con 0x7faa30144f60 2024-03-30T13:11:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa3015dbf0 0x7faa300ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa3015dbf0 0x7faa300ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43376/0 (socket says 172.21.15.43:43376) 2024-03-30T13:11:59.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 -- 172.21.15.43:0/1689884399 learned_addr learned my addr 172.21.15.43:0/1689884399 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2ed9d640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa30144b80 0x7faa3015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 -- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa30144f60 msgr2=0x7faa30159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa30144f60 0x7faa30159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 -- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa30144b80 msgr2=0x7faa3015d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa30144b80 0x7faa3015d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 -- 172.21.15.43:0/1689884399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa300cd730 con 0x7faa3015dbf0 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa2e59c640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa3015dbf0 0x7faa300ce7d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faa200115e0 tx=0x7faa20011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa177fe640 1 -- 172.21.15.43:0/1689884399 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2000f8f0 con 0x7faa3015dbf0 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa177fe640 1 -- 172.21.15.43:0/1689884399 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa2000fa90 con 0x7faa3015dbf0 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.235+0000 7faa177fe640 1 -- 172.21.15.43:0/1689884399 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2000fe40 con 0x7faa3015dbf0 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa300cda10 con 0x7faa3015dbf0 2024-03-30T13:11:59.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa300cdfd0 con 0x7faa3015dbf0 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa30068900 con 0x7faa3015dbf0 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa177fe640 1 -- 172.21.15.43:0/1689884399 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7faa20015460 con 0x7faa3015dbf0 2024-03-30T13:11:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa177fe640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa1c067fc0 0x7faa1c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa177fe640 1 -- 172.21.15.43:0/1689884399 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7faa200872e0 con 0x7faa3015dbf0 2024-03-30T13:11:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa2ed9d640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa1c067fc0 0x7faa1c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.239+0000 7faa2ed9d640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa1c067fc0 0x7faa1c06a480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faa240ac000 tx=0x7faa24009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.247+0000 7faa177fe640 1 -- 172.21.15.43:0/1689884399 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7faa20019030 con 0x7faa3015dbf0 2024-03-30T13:11:59.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.407+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faa300706c0 con 0x7faa3015dbf0 2024-03-30T13:11:59.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.407+0000 7faa177fe640 1 -- 172.21.15.43:0/1689884399 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7faa20057690 con 0x7faa3015dbf0 2024-03-30T13:11:59.416 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa1c067fc0 msgr2=0x7faa1c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa1c067fc0 0x7faa1c06a480 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faa240ac000 tx=0x7faa24009040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa3015dbf0 msgr2=0x7faa300ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa3015dbf0 0x7faa300ce7d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faa200115e0 tx=0x7faa20011aa0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 shutdown_connections 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7faa1c067fc0 0x7faa1c06a480 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7faa30144f60 0x7faa30159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faa3015dbf0 0x7faa300ce7d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 --2- 172.21.15.43:0/1689884399 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7faa30144b80 0x7faa3015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 >> 172.21.15.43:0/1689884399 conn(0x7faa300d1070 msgr2=0x7faa300af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 shutdown_connections 2024-03-30T13:11:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.411+0000 7faa35168640 1 -- 172.21.15.43:0/1689884399 wait complete. 2024-03-30T13:11:59.433 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe09603c640 1 Processor -- start 2024-03-30T13:11:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe09603c640 1 -- start start 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe09603c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0901420a0 0x7fe0900cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe09603c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0900cdc10 con 0x7fe0900cfb00 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe09603c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0900cf060 con 0x7fe09015c0a0 2024-03-30T13:11:59.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe09603c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe0900cf670 con 0x7fe0901420a0 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe08ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0901420a0 0x7fe0900cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe08effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0900cfb00 0x7fe090141990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41394/0 (socket says 172.21.15.43:41394) 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.551+0000 7fe08effd640 1 -- 172.21.15.43:0/160578676 learned_addr learned my addr 172.21.15.43:0/160578676 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe08ffff640 1 -- 172.21.15.43:0/160578676 >> v1:172.21.15.113:6789/0 conn(0x7fe09015c0a0 legacy=0x7fe09006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe08ffff640 1 -- 172.21.15.43:0/160578676 >> v1:172.21.15.43:6789/0 conn(0x7fe0900cfb00 legacy=0x7fe090141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe08ffff640 1 -- 172.21.15.43:0/160578676 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0900cd650 con 0x7fe0901420a0 2024-03-30T13:11:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe08ffff640 1 --2- 172.21.15.43:0/160578676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe0900cff60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe09006d1a0 tx=0x7fe08801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797e8bc5601c3c75 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe08e7fc640 1 -- 172.21.15.43:0/160578676 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0880b2070 con 0x7fe0901420a0 2024-03-30T13:11:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe08e7fc640 1 -- 172.21.15.43:0/160578676 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0880a58a0 con 0x7fe0901420a0 2024-03-30T13:11:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe08e7fc640 1 -- 172.21.15.43:0/160578676 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0880a5c70 con 0x7fe0901420a0 2024-03-30T13:11:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe09603c640 1 -- 172.21.15.43:0/160578676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 msgr2=0x7fe0900cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe09603c640 1 --2- 172.21.15.43:0/160578676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe0900cff60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe09006d1a0 tx=0x7fe08801c8f0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe09603c640 1 -- 172.21.15.43:0/160578676 shutdown_connections 2024-03-30T13:11:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe09603c640 1 --2- 172.21.15.43:0/160578676 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe0900cff60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe09603c640 1 -- 172.21.15.43:0/160578676 >> 172.21.15.43:0/160578676 conn(0x7fe0900b2980 msgr2=0x7fe0900d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe09603c640 1 -- 172.21.15.43:0/160578676 shutdown_connections 2024-03-30T13:11:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.555+0000 7fe09603c640 1 -- 172.21.15.43:0/160578676 wait complete. 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 Processor -- start 2024-03-30T13:11:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 -- start start 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0900cfb00 0x7fe09013fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe09015a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe09015c0a0 0x7fe09015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe09015c600 con 0x7fe0901420a0 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe09015c480 con 0x7fe09015c0a0 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe0900cd0f0 con 0x7fe0900cfb00 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe09015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe09015a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43388/0 (socket says 172.21.15.43:43388) 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 -- 172.21.15.43:0/3614214805 learned_addr learned my addr 172.21.15.43:0/3614214805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08ffff640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe09015c0a0 0x7fe09015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08f7fe640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0900cfb00 0x7fe09013fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 -- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0900cfb00 msgr2=0x7fe09013fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0900cfb00 0x7fe09013fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 -- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe09015c0a0 msgr2=0x7fe09015ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe09015c0a0 0x7fe09015ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 -- 172.21.15.43:0/3614214805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09015eab0 con 0x7fe0901420a0 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08f7fe640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0900cfb00 0x7fe09013fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08ffff640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe09015c0a0 0x7fe09015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08effd640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe09015a920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe08400c420 tx=0x7fe08400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe077fff640 1 -- 172.21.15.43:0/3614214805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe084017070 con 0x7fe0901420a0 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe077fff640 1 -- 172.21.15.43:0/3614214805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe084005cf0 con 0x7fe0901420a0 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09015ed90 con 0x7fe0901420a0 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe077fff640 1 -- 172.21.15.43:0/3614214805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840072f0 con 0x7fe0901420a0 2024-03-30T13:11:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe09015f350 con 0x7fe0901420a0 2024-03-30T13:11:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe090068890 con 0x7fe0901420a0 2024-03-30T13:11:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe077fff640 1 -- 172.21.15.43:0/3614214805 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe08401a070 con 0x7fe0901420a0 2024-03-30T13:11:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe077fff640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe07c067fc0 0x7fe07c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:11:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe077fff640 1 -- 172.21.15.43:0/3614214805 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fe084085c50 con 0x7fe0901420a0 2024-03-30T13:11:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.559+0000 7fe08f7fe640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe07c067fc0 0x7fe07c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:11:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.563+0000 7fe08f7fe640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe07c067fc0 0x7fe07c06a480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b1c80 tx=0x7fe080008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:11:59.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.571+0000 7fe077fff640 1 -- 172.21.15.43:0/3614214805 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe08408b020 con 0x7fe0901420a0 2024-03-30T13:11:59.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.739+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe090074e60 con 0x7fe0901420a0 2024-03-30T13:11:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.739+0000 7fe077fff640 1 -- 172.21.15.43:0/3614214805 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fe084056090 con 0x7fe0901420a0 2024-03-30T13:11:59.746 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe07c067fc0 msgr2=0x7fe07c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe07c067fc0 0x7fe07c06a480 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe0900b1c80 tx=0x7fe080008040 comp rx=0 tx=0).stop 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 msgr2=0x7fe09015a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe09015a920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe08400c420 tx=0x7fe08400c8e0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 shutdown_connections 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe07c067fc0 0x7fe07c06a480 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe09015c0a0 0x7fe09015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0901420a0 0x7fe09015a920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 --2- 172.21.15.43:0/3614214805 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0900cfb00 0x7fe09013fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 >> 172.21.15.43:0/3614214805 conn(0x7fe0900b2980 msgr2=0x7fe09006eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:11:59.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 shutdown_connections 2024-03-30T13:11:59.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.743+0000 7fe09603c640 1 -- 172.21.15.43:0/3614214805 wait complete. 2024-03-30T13:11:59.765 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-03-30T13:11:59.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:59.765 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T13:11:59.776 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:11:59.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:11:59.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:11:59.777 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T13:11:59.825 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:11:59.825 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:59.832 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:11:59.832 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:11:59.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.999+0000 7fd622c73640 1 Processor -- start 2024-03-30T13:12:00.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:11:59.999+0000 7fd622c73640 1 -- start start 2024-03-30T13:12:00.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd622c73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 0x7fd61c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd622c73640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd61c0af590 con 0x7fd61c15c410 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd622c73640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd61c0af870 con 0x7fd61c0b15c0 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd622c73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd61c0cc900 con 0x7fd61c0b5e30 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd61c15c410 0x7fd61c0b0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41398/0 (socket says 172.21.15.43:41398) 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 -- 172.21.15.43:0/2131662531 learned_addr learned my addr 172.21.15.43:0/2131662531 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61bfff640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 0x7fd61c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4265926110 0 0) 0x7fd61c0af590 con 0x7fd61c15c410 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd608003530 con 0x7fd61c15c410 2024-03-30T13:12:00.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 184205842 0 0) 0x7fd608003530 con 0x7fd61c15c410 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 msgr2=0x7fd61c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 0x7fd61c15c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 >> v1:172.21.15.113:6789/0 conn(0x7fd61c0b15c0 legacy=0x7fd61c06bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c0cc590 con 0x7fd61c15c410 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fd60c001680 con 0x7fd61c15c410 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 >> v1:172.21.15.43:6789/0 conn(0x7fd61c15c410 legacy=0x7fd61c0b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6080039f0 0x7fd608003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 0x7fd608004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd608004d50 0x7fd6080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c0cc900 con 0x7fd6080039f0 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd60800ba80 con 0x7fd608004d50 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd60800bc80 con 0x7fd61c0b5e30 2024-03-30T13:12:00.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6080039f0 0x7fd608003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd621210640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd608004d50 0x7fd6080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd61bfff640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 0x7fd608004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 -- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 msgr2=0x7fd608004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c0b5e30 0x7fd608004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 -- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd608004d50 msgr2=0x7fd6080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd608004d50 0x7fd6080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.003+0000 7fd620a0f640 1 -- 172.21.15.43:0/2131662531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd608003530 con 0x7fd6080039f0 2024-03-30T13:12:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd620a0f640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6080039f0 0x7fd608003dd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c2c0 tx=0x7fd60c01cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfb0a3380cbaa91 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c007d60 con 0x7fd6080039f0 2024-03-30T13:12:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd60c0a9070 con 0x7fd6080039f0 2024-03-30T13:12:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61b7fe640 1 -- 172.21.15.43:0/2131662531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c005d40 con 0x7fd6080039f0 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- 172.21.15.43:0/2131662531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6080039f0 msgr2=0x7fd608003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 --2- 172.21.15.43:0/2131662531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6080039f0 0x7fd608003dd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c2c0 tx=0x7fd60c01cd40 comp rx=0 tx=0).stop 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd621210640 1 -- 172.21.15.43:0/2131662531 reap_dead start 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- 172.21.15.43:0/2131662531 shutdown_connections 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- 172.21.15.43:0/2131662531 >> 172.21.15.43:0/2131662531 conn(0x7fd61c0b5060 msgr2=0x7fd61c0b5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- 172.21.15.43:0/2131662531 shutdown_connections 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- 172.21.15.43:0/2131662531 wait complete. 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 Processor -- start 2024-03-30T13:12:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- start start 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6080039f0 0x7fd61c166380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c151930 0x7fd61c14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c151db0 0x7fd61c14fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1684d0 con 0x7fd61c151930 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c168350 con 0x7fd6080039f0 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd622c73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd61c168650 con 0x7fd61c151db0 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c151930 0x7fd61c14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c151930 0x7fd61c14f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43416/0 (socket says 172.21.15.43:43416) 2024-03-30T13:12:00.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 -- 172.21.15.43:0/2749893706 learned_addr learned my addr 172.21.15.43:0/2749893706 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd620a0f640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6080039f0 0x7fd61c166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd621210640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c151db0 0x7fd61c14fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 -- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c151db0 msgr2=0x7fd61c14fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c151db0 0x7fd61c14fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 -- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6080039f0 msgr2=0x7fd61c166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6080039f0 0x7fd61c166380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.007+0000 7fd61bfff640 1 -- 172.21.15.43:0/2749893706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c0ae960 con 0x7fd61c151930 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd61bfff640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c151930 0x7fd61c14f6c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd604007f60 tx=0x7fd604007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd6197fa640 1 -- 172.21.15.43:0/2749893706 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60401a020 con 0x7fd61c151930 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd6197fa640 1 -- 172.21.15.43:0/2749893706 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd604016470 con 0x7fd61c151930 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c0aec40 con 0x7fd61c151930 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd6197fa640 1 -- 172.21.15.43:0/2749893706 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604016860 con 0x7fd61c151930 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c15d670 con 0x7fd61c151930 2024-03-30T13:12:00.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c068890 con 0x7fd61c151930 2024-03-30T13:12:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd6197fa640 1 -- 172.21.15.43:0/2749893706 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd60401d020 con 0x7fd61c151930 2024-03-30T13:12:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd6197fa640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd5fc067fc0 0x7fd5fc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd6197fa640 1 -- 172.21.15.43:0/2749893706 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fd6040875e0 con 0x7fd61c151930 2024-03-30T13:12:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd620a0f640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd5fc067fc0 0x7fd5fc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.011+0000 7fd620a0f640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd5fc067fc0 0x7fd5fc06a480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0ab000 tx=0x7fd60c0b1040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.015+0000 7fd6197fa640 1 -- 172.21.15.43:0/2749893706 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd604024030 con 0x7fd61c151930 2024-03-30T13:12:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.211+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd61c168130 con 0x7fd61c151930 2024-03-30T13:12:00.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd6197fa640 1 -- 172.21.15.43:0/2749893706 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7fd604057ab0 con 0x7fd61c151930 2024-03-30T13:12:00.222 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:00.222 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd5fc067fc0 msgr2=0x7fd5fc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd5fc067fc0 0x7fd5fc06a480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0ab000 tx=0x7fd60c0b1040 comp rx=0 tx=0).stop 2024-03-30T13:12:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c151930 msgr2=0x7fd61c14f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c151930 0x7fd61c14f6c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd604007f60 tx=0x7fd604007f90 comp rx=0 tx=0).stop 2024-03-30T13:12:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 shutdown_connections 2024-03-30T13:12:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd5fc067fc0 0x7fd5fc06a480 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd61c151db0 0x7fd61c14fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd61c151930 0x7fd61c14f6c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 --2- 172.21.15.43:0/2749893706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6080039f0 0x7fd61c166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 >> 172.21.15.43:0/2749893706 conn(0x7fd61c0b5060 msgr2=0x7fd61c152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 shutdown_connections 2024-03-30T13:12:00.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.215+0000 7fd622c73640 1 -- 172.21.15.43:0/2749893706 wait complete. 2024-03-30T13:12:00.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:12:00.242 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:00.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f1876035640 1 Processor -- start 2024-03-30T13:12:00.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f1876035640 1 -- start start 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f1876035640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18700700f0 0x7f1870165ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f1876035640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18700b4ba0 con 0x7f18700b46f0 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f1876035640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f187015ec60 con 0x7f187015afc0 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f1876035640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1870070c30 con 0x7f18700700f0 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f18700b46f0 0x7f187006f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41406/0 (socket says 172.21.15.43:41406) 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186effd640 1 -- 172.21.15.43:0/3297210333 learned_addr learned my addr 172.21.15.43:0/3297210333 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186ffff640 1 --2- 172.21.15.43:0/3297210333 >> v2:172.21.15.113:3300/0 conn(0x7f18700700f0 0x7f1870165ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1468817533 0 0) 0x7f18700b4ba0 con 0x7f18700b46f0 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1858003530 con 0x7f18700b46f0 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1823726389 0 0) 0x7f1858003530 con 0x7f18700b46f0 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 >> v2:172.21.15.113:3300/0 conn(0x7f18700700f0 msgr2=0x7f1870165ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 --2- 172.21.15.43:0/3297210333 >> v2:172.21.15.113:3300/0 conn(0x7f18700700f0 0x7f1870165ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 >> v1:172.21.15.113:6789/0 conn(0x7f187015afc0 legacy=0x7f18700b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187015ef40 con 0x7f18700b46f0 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f1860001680 con 0x7f18700b46f0 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 >> v1:172.21.15.43:6789/0 conn(0x7f18700b46f0 legacy=0x7f187006f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580039c0 0x7f1858003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1858004810 0x7f1858004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18580051d0 0x7f185800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f185800c040 con 0x7f18580051d0 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f185800c1f0 con 0x7f1858004810 2024-03-30T13:12:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f185800c3f0 con 0x7f18580039c0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186ffff640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18580051d0 0x7f185800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186effd640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1858004810 0x7f1858004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186f7fe640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580039c0 0x7f1858003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186ffff640 1 -- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580039c0 msgr2=0x7f1858003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186ffff640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580039c0 0x7f1858003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186ffff640 1 -- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1858004810 msgr2=0x7f1858004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186ffff640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1858004810 0x7f1858004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.355+0000 7f186ffff640 1 -- 172.21.15.43:0/3297210333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1858003530 con 0x7f18580051d0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f186ffff640 1 -- 172.21.15.43:0/3297210333 reap_dead start 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f186ffff640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18580051d0 0x7f185800b9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1868009fc0 tx=0x7f186801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4959a93d9586b61d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868009c10 con 0x7f18580051d0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18680a5d10 con 0x7f18580051d0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f186e7fc640 1 -- 172.21.15.43:0/3297210333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868003be0 con 0x7f18580051d0 2024-03-30T13:12:00.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f1876035640 1 -- 172.21.15.43:0/3297210333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18580051d0 msgr2=0x7f185800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f1876035640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18580051d0 0x7f185800b9d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1868009fc0 tx=0x7f186801ccc0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f1876035640 1 -- 172.21.15.43:0/3297210333 shutdown_connections 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f1876035640 1 --2- 172.21.15.43:0/3297210333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18580051d0 0x7f185800b9d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f1876035640 1 -- 172.21.15.43:0/3297210333 >> 172.21.15.43:0/3297210333 conn(0x7f18700d1070 msgr2=0x7f18700d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f1876035640 1 -- 172.21.15.43:0/3297210333 shutdown_connections 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.359+0000 7f1876035640 1 -- 172.21.15.43:0/3297210333 wait complete. 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 Processor -- start 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 -- start start 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580051d0 0x7f1870147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18700ce7b0 0x7f18700cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f187015c630 0x7f187015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f187015fa20 con 0x7f187015c630 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f187015f500 con 0x7f18700ce7b0 2024-03-30T13:12:00.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f18701482b0 con 0x7f18580051d0 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f187015c630 0x7f187015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f187015c630 0x7f187015cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43428/0 (socket says 172.21.15.43:43428) 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 -- 172.21.15.43:0/1073843086 learned_addr learned my addr 172.21.15.43:0/1073843086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186f7fe640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580051d0 0x7f1870147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186effd640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18700ce7b0 0x7f18700cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 -- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580051d0 msgr2=0x7f1870147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580051d0 0x7f1870147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 -- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18700ce7b0 msgr2=0x7f18700cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18700ce7b0 0x7f18700cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 -- 172.21.15.43:0/1073843086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187015d240 con 0x7f187015c630 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186effd640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18700ce7b0 0x7f18700cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f186ffff640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f187015c630 0x7f187015cab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f18680987f0 tx=0x7f1868009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1857fff640 1 -- 172.21.15.43:0/1073843086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1868009c10 con 0x7f187015c630 2024-03-30T13:12:00.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1857fff640 1 -- 172.21.15.43:0/1073843086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18680bd070 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187015d520 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1857fff640 1 -- 172.21.15.43:0/1073843086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18680039a0 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187015dae0 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1870068900 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1857fff640 1 -- 172.21.15.43:0/1073843086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1868003b40 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1857fff640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f185c067fc0 0x7f185c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.363+0000 7f1857fff640 1 -- 172.21.15.43:0/1073843086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f18681223a0 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.367+0000 7f186f7fe640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f185c067fc0 0x7f185c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.367+0000 7f186f7fe640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f185c067fc0 0x7f185c06a480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c10 tx=0x7f1864009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.367+0000 7f1857fff640 1 -- 172.21.15.43:0/1073843086 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f18680169d0 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.523+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f187015fba0 con 0x7f187015c630 2024-03-30T13:12:00.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.523+0000 7f1857fff640 1 -- 172.21.15.43:0/1073843086 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f18680f27e0 con 0x7f187015c630 2024-03-30T13:12:00.586 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f185c067fc0 msgr2=0x7f185c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f185c067fc0 0x7f185c06a480 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1864000c10 tx=0x7f1864009040 comp rx=0 tx=0).stop 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f187015c630 msgr2=0x7f187015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f187015c630 0x7f187015cab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f18680987f0 tx=0x7f1868009750 comp rx=0 tx=0).stop 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 shutdown_connections 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f185c067fc0 0x7f185c06a480 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f187015c630 0x7f187015cab0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18700ce7b0 0x7f18700cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 --2- 172.21.15.43:0/1073843086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18580051d0 0x7f1870147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 >> 172.21.15.43:0/1073843086 conn(0x7f18700d1070 msgr2=0x7f18700d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 shutdown_connections 2024-03-30T13:12:00.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.527+0000 7f1876035640 1 -- 172.21.15.43:0/1073843086 wait complete. 2024-03-30T13:12:00.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43d0f2640 1 Processor -- start 2024-03-30T13:12:00.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43d0f2640 1 -- start start 2024-03-30T13:12:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43d0f2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff43806c4d0 0x7ff4380ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43d0f2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4380b5470 con 0x7ff43806c070 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43d0f2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4380b5750 con 0x7ff438147e00 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43d0f2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff4380b5a30 con 0x7ff43806c4d0 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff43806c070 0x7ff4380adbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41412/0 (socket says 172.21.15.43:41412) 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 -- 172.21.15.43:0/533731004 learned_addr learned my addr 172.21.15.43:0/533731004 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863409806 0 0) 0x7ff4380b5470 con 0x7ff43806c070 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43759e640 1 --2- 172.21.15.43:0/533731004 >> v2:172.21.15.113:3300/0 conn(0x7ff43806c4d0 0x7ff4380ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff420003530 con 0x7ff43806c070 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3316936929 0 0) 0x7ff420003530 con 0x7ff43806c070 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 >> v2:172.21.15.113:3300/0 conn(0x7ff43806c4d0 msgr2=0x7ff4380ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 --2- 172.21.15.43:0/533731004 >> v2:172.21.15.113:3300/0 conn(0x7ff43806c4d0 0x7ff4380ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 >> v1:172.21.15.113:6789/0 conn(0x7ff438147e00 legacy=0x7ff4380b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4380b5d10 con 0x7ff43806c070 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ff42c001680 con 0x7ff43806c070 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 >> v1:172.21.15.43:6789/0 conn(0x7ff43806c070 legacy=0x7ff4380adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4200039c0 0x7ff420003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff420004810 0x7ff420004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4200051d0 0x7ff42000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380b5a30 con 0x7ff420004810 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff42000c070 con 0x7ff4200051d0 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff42000c270 con 0x7ff4200039c0 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff420004810 0x7ff420004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43759e640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4200051d0 0x7ff42000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff436d9d640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4200039c0 0x7ff420003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 -- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4200039c0 msgr2=0x7ff420003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4200039c0 0x7ff420003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 -- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4200051d0 msgr2=0x7ff42000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff4200051d0 0x7ff42000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43659c640 1 -- 172.21.15.43:0/533731004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff420003530 con 0x7ff420004810 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff43759e640 1 -- 172.21.15.43:0/533731004 reap_dead start 2024-03-30T13:12:00.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.699+0000 7ff436d9d640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4200039c0 0x7ff420003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43659c640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff420004810 0x7ff420004c90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c240 tx=0x7ff42c01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04b30cdafb8aa0c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c007d20 con 0x7ff420004810 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff42c005ae0 con 0x7ff420004810 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff435d9b640 1 -- 172.21.15.43:0/533731004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c007070 con 0x7ff420004810 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- 172.21.15.43:0/533731004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff420004810 msgr2=0x7ff420004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff420004810 0x7ff420004c90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c240 tx=0x7ff42c01f3b0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- 172.21.15.43:0/533731004 shutdown_connections 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/533731004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff420004810 0x7ff420004c90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- 172.21.15.43:0/533731004 >> 172.21.15.43:0/533731004 conn(0x7ff4380d1070 msgr2=0x7ff4380d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- 172.21.15.43:0/533731004 shutdown_connections 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- 172.21.15.43:0/533731004 wait complete. 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 Processor -- start 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- start start 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff420004810 0x7ff438143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4380d0380 0x7ff4380d0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43813eb20 0x7ff43813efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff4380b49f0 con 0x7ff43813eb20 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff43806b880 con 0x7ff420004810 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43d0f2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff4380b4b70 con 0x7ff4380d0380 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43813eb20 0x7ff43813efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43813eb20 0x7ff43813efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43446/0 (socket says 172.21.15.43:43446) 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.703+0000 7ff43759e640 1 -- 172.21.15.43:0/3574323457 learned_addr learned my addr 172.21.15.43:0/3574323457 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff436d9d640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff420004810 0x7ff438143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43659c640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4380d0380 0x7ff4380d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43759e640 1 -- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4380d0380 msgr2=0x7ff4380d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43759e640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4380d0380 0x7ff4380d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43759e640 1 -- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff420004810 msgr2=0x7ff438143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43759e640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff420004810 0x7ff438143c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43759e640 1 -- 172.21.15.43:0/3574323457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43813f810 con 0x7ff43813eb20 2024-03-30T13:12:00.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43759e640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43813eb20 0x7ff43813efa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff43806d9a0 tx=0x7ff4300115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff41f7fe640 1 -- 172.21.15.43:0/3574323457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff430011e10 con 0x7ff43813eb20 2024-03-30T13:12:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff41f7fe640 1 -- 172.21.15.43:0/3574323457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43001b070 con 0x7ff43813eb20 2024-03-30T13:12:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43813faf0 con 0x7ff43813eb20 2024-03-30T13:12:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff41f7fe640 1 -- 172.21.15.43:0/3574323457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4300039a0 con 0x7ff43813eb20 2024-03-30T13:12:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4381400b0 con 0x7ff43813eb20 2024-03-30T13:12:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff438068900 con 0x7ff43813eb20 2024-03-30T13:12:00.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff41f7fe640 1 -- 172.21.15.43:0/3574323457 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff430003b40 con 0x7ff43813eb20 2024-03-30T13:12:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff41f7fe640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff424067fc0 0x7ff42406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff41f7fe640 1 -- 172.21.15.43:0/3574323457 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7ff430087160 con 0x7ff43813eb20 2024-03-30T13:12:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff436d9d640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff424067fc0 0x7ff42406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:00.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.707+0000 7ff436d9d640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff424067fc0 0x7ff42406a480 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff438074a70 tx=0x7ff428009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:00.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.711+0000 7ff41f7fe640 1 -- 172.21.15.43:0/3574323457 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff430025020 con 0x7ff43813eb20 2024-03-30T13:12:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.867+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff438070a90 con 0x7ff43813eb20 2024-03-30T13:12:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.867+0000 7ff41f7fe640 1 -- 172.21.15.43:0/3574323457 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7ff4300575a0 con 0x7ff43813eb20 2024-03-30T13:12:00.875 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:00.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff424067fc0 msgr2=0x7ff42406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff424067fc0 0x7ff42406a480 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff438074a70 tx=0x7ff428009040 comp rx=0 tx=0).stop 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43813eb20 msgr2=0x7ff43813efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43813eb20 0x7ff43813efa0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff43806d9a0 tx=0x7ff4300115a0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 shutdown_connections 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff424067fc0 0x7ff42406a480 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff43813eb20 0x7ff43813efa0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff4380d0380 0x7ff4380d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 --2- 172.21.15.43:0/3574323457 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff420004810 0x7ff438143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 >> 172.21.15.43:0/3574323457 conn(0x7ff4380d1070 msgr2=0x7ff438070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 shutdown_connections 2024-03-30T13:12:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:00.871+0000 7ff43d0f2640 1 -- 172.21.15.43:0/3574323457 wait complete. 2024-03-30T13:12:00.893 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:01.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.003+0000 7f6df308b640 1 Processor -- start 2024-03-30T13:12:01.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.003+0000 7f6df308b640 1 -- start start 2024-03-30T13:12:01.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6df308b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6dec06f090 0x7f6dec0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6df308b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dec0ce4d0 con 0x7f6dec15f880 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6df308b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dec070a50 con 0x7f6dec070640 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6df308b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6dec0cc7f0 con 0x7f6dec06f090 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6df0e27640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6dec15f880 0x7f6dec06e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41418/0 (socket says 172.21.15.43:41418) 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6df0e27640 1 -- 172.21.15.43:0/1397377614 learned_addr learned my addr 172.21.15.43:0/1397377614 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6debfff640 1 --2- 172.21.15.43:0/1397377614 >> v2:172.21.15.43:3300/0 conn(0x7f6dec06f090 0x7f6dec0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6deb7fe640 1 -- 172.21.15.43:0/1397377614 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2415135214 0 0) 0x7f6dec0ce4d0 con 0x7f6dec15f880 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6deb7fe640 1 -- 172.21.15.43:0/1397377614 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6dd4003530 con 0x7f6dec15f880 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6debfff640 1 -- 172.21.15.43:0/1397377614 >> v1:172.21.15.113:6789/0 conn(0x7f6dec070640 legacy=0x7f6dec0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6debfff640 1 -- 172.21.15.43:0/1397377614 >> v1:172.21.15.43:6789/0 conn(0x7f6dec15f880 legacy=0x7f6dec06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6deb7fe640 1 -- 172.21.15.43:0/1397377614 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1268085462 0 0) 0x7f6dd4003530 con 0x7f6dec15f880 2024-03-30T13:12:01.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.007+0000 7f6debfff640 1 -- 172.21.15.43:0/1397377614 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec0ccad0 con 0x7f6dec06f090 2024-03-30T13:12:01.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6debfff640 1 --2- 172.21.15.43:0/1397377614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec06f090 0x7f6dec0b4ff0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6de00097c0 tx=0x7f6de001c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6a3894fdab0401 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6deb7fe640 1 -- 172.21.15.43:0/1397377614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de00b0070 con 0x7f6dec06f090 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6deb7fe640 1 -- 172.21.15.43:0/1397377614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de00ab400 con 0x7f6dec06f090 2024-03-30T13:12:01.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6deb7fe640 1 -- 172.21.15.43:0/1397377614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0016a0 con 0x7f6dec06f090 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6df308b640 1 -- 172.21.15.43:0/1397377614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec06f090 msgr2=0x7f6dec0b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6df308b640 1 --2- 172.21.15.43:0/1397377614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec06f090 0x7f6dec0b4ff0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6de00097c0 tx=0x7f6de001c2a0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6df308b640 1 -- 172.21.15.43:0/1397377614 shutdown_connections 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6df308b640 1 --2- 172.21.15.43:0/1397377614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec06f090 0x7f6dec0b4ff0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6df308b640 1 -- 172.21.15.43:0/1397377614 >> 172.21.15.43:0/1397377614 conn(0x7f6dec0b3cf0 msgr2=0x7f6dec0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6df308b640 1 -- 172.21.15.43:0/1397377614 shutdown_connections 2024-03-30T13:12:01.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.011+0000 7f6df308b640 1 -- 172.21.15.43:0/1397377614 wait complete. 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 Processor -- start 2024-03-30T13:12:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 -- start start 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dec06f090 0x7f6dec0c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dec070640 0x7f6dec0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec15f880 0x7f6dec157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec0cf650 con 0x7f6dec15f880 2024-03-30T13:12:01.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec0cf4d0 con 0x7f6dec070640 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6dec160850 con 0x7f6dec06f090 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec15f880 0x7f6dec157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec15f880 0x7f6dec157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43468/0 (socket says 172.21.15.43:43468) 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 -- 172.21.15.43:0/3122760935 learned_addr learned my addr 172.21.15.43:0/3122760935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 -- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dec06f090 msgr2=0x7f6dec0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dec06f090 0x7f6dec0c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 -- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dec070640 msgr2=0x7f6dec0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dec070640 0x7f6dec0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 -- 172.21.15.43:0/3122760935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec0c9ee0 con 0x7f6dec15f880 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df1628640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec15f880 0x7f6dec157610 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c730 tx=0x7f6de400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6de97fa640 1 -- 172.21.15.43:0/3122760935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4003b80 con 0x7f6dec15f880 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6de97fa640 1 -- 172.21.15.43:0/3122760935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6de4003d20 con 0x7f6dec15f880 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec0ca1c0 con 0x7f6dec15f880 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6de97fa640 1 -- 172.21.15.43:0/3122760935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de400aa30 con 0x7f6dec15f880 2024-03-30T13:12:01.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec15a3f0 con 0x7f6dec15f880 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.015+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dec15a200 con 0x7f6dec15f880 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.019+0000 7f6de97fa640 1 -- 172.21.15.43:0/3122760935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6de400abd0 con 0x7f6dec15f880 2024-03-30T13:12:01.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.019+0000 7f6de97fa640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6dd8067ff0 0x7f6dd806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.019+0000 7f6de97fa640 1 -- 172.21.15.43:0/3122760935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f6de4086710 con 0x7f6dec15f880 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.019+0000 7f6df0e27640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6dd8067ff0 0x7f6dd806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.019+0000 7f6df0e27640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6dd8067ff0 0x7f6dd806a4b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc001210 tx=0x7f6ddc001180 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.023+0000 7f6de97fa640 1 -- 172.21.15.43:0/3122760935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6de4056b40 con 0x7f6dec15f880 2024-03-30T13:12:01.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6dec068900 con 0x7f6dec15f880 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6de97fa640 1 -- 172.21.15.43:0/3122760935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f6de4056440 con 0x7f6dec15f880 2024-03-30T13:12:01.194 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6dd8067ff0 msgr2=0x7f6dd806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6dd8067ff0 0x7f6dd806a4b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc001210 tx=0x7f6ddc001180 comp rx=0 tx=0).stop 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec15f880 msgr2=0x7f6dec157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec15f880 0x7f6dec157610 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c730 tx=0x7f6de400cbf0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 shutdown_connections 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6dd8067ff0 0x7f6dd806a4b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6dec15f880 0x7f6dec157610 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6dec070640 0x7f6dec0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 --2- 172.21.15.43:0/3122760935 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6dec06f090 0x7f6dec0c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.187+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 >> 172.21.15.43:0/3122760935 conn(0x7f6dec0b3cf0 msgr2=0x7f6dec06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.191+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 shutdown_connections 2024-03-30T13:12:01.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.191+0000 7f6df308b640 1 -- 172.21.15.43:0/3122760935 wait complete. 2024-03-30T13:12:01.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:12:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59397e3640 1 Processor -- start 2024-03-30T13:12:01.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59397e3640 1 -- start start 2024-03-30T13:12:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59397e3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f593415afc0 0x7f59340b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59397e3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f59340700f0 0x7f59340ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59397e3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59340b4ba0 con 0x7f59340b4520 2024-03-30T13:12:01.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59397e3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5934070c30 con 0x7f593415afc0 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59397e3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f59340b4960 con 0x7f59340700f0 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f5932ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f593415afc0 0x7f59340b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f5932ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f593415afc0 0x7f59340b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57194/0 (socket says 172.21.15.43:57194) 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f5932ffd640 1 -- 172.21.15.43:0/687029840 learned_addr learned my addr 172.21.15.43:0/687029840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f59337fe640 1 --2- 172.21.15.43:0/687029840 >> v2:172.21.15.113:3300/0 conn(0x7f59340700f0 0x7f59340ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f5932ffd640 1 -- 172.21.15.43:0/687029840 >> v2:172.21.15.113:3300/0 conn(0x7f59340700f0 msgr2=0x7f59340ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f5932ffd640 1 --2- 172.21.15.43:0/687029840 >> v2:172.21.15.113:3300/0 conn(0x7f59340700f0 0x7f59340ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f5932ffd640 1 -- 172.21.15.43:0/687029840 >> v1:172.21.15.113:6789/0 conn(0x7f59340b4520 legacy=0x7f593406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.335+0000 7f5932ffd640 1 -- 172.21.15.43:0/687029840 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593415ec60 con 0x7f593415afc0 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f5932ffd640 1 --2- 172.21.15.43:0/687029840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59340b3fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f593406e500 tx=0x7f592c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8499b358fb69be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f5931ffb640 1 -- 172.21.15.43:0/687029840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0a5880 con 0x7f593415afc0 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f5931ffb640 1 -- 172.21.15.43:0/687029840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c0a5a20 con 0x7f593415afc0 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f5931ffb640 1 -- 172.21.15.43:0/687029840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0a5df0 con 0x7f593415afc0 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- 172.21.15.43:0/687029840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 msgr2=0x7f59340b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 --2- 172.21.15.43:0/687029840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59340b3fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f593406e500 tx=0x7f592c01c9c0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- 172.21.15.43:0/687029840 shutdown_connections 2024-03-30T13:12:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 --2- 172.21.15.43:0/687029840 >> v2:172.21.15.113:3300/0 conn(0x7f59340700f0 0x7f59340ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 --2- 172.21.15.43:0/687029840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59340b3fe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- 172.21.15.43:0/687029840 >> 172.21.15.43:0/687029840 conn(0x7f59340d1070 msgr2=0x7f59340d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- 172.21.15.43:0/687029840 shutdown_connections 2024-03-30T13:12:01.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- 172.21.15.43:0/687029840 wait complete. 2024-03-30T13:12:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 Processor -- start 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- start start 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340700f0 0x7f5934162db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340b4520 0x7f5934152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59341645a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f593415fa20 con 0x7f593415afc0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f593415b3a0 con 0x7f59340700f0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.339+0000 7f59397e3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5934161fc0 con 0x7f59340b4520 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59341645a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59341645a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57210/0 (socket says 172.21.15.43:57210) 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 -- 172.21.15.43:0/594256606 learned_addr learned my addr 172.21.15.43:0/594256606 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59327fc640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340b4520 0x7f5934152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f5932ffd640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340700f0 0x7f5934162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 -- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340b4520 msgr2=0x7f5934152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340b4520 0x7f5934152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 -- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340700f0 msgr2=0x7f5934162db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340700f0 0x7f5934162db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 -- 172.21.15.43:0/594256606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5934152670 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f5932ffd640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340700f0 0x7f5934162db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59337fe640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59341645a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f592400d560 tx=0x7f592400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f591b7fe640 1 -- 172.21.15.43:0/594256606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59240125f0 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f591b7fe640 1 -- 172.21.15.43:0/594256606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5924019070 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5934162310 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f591b7fe640 1 -- 172.21.15.43:0/594256606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59240129f0 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341625b0 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5934068900 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f591b7fe640 1 -- 172.21.15.43:0/594256606 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5924012b90 con 0x7f593415afc0 2024-03-30T13:12:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f591b7fe640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5920067fc0 0x7f592006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f591b7fe640 1 -- 172.21.15.43:0/594256606 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f5924086c50 con 0x7f593415afc0 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.343+0000 7f5932ffd640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5920067fc0 0x7f592006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.347+0000 7f5932ffd640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5920067fc0 0x7f592006a480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f592c0ae000 tx=0x7f592c0a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.347+0000 7f591b7fe640 1 -- 172.21.15.43:0/594256606 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f592408c020 con 0x7f593415afc0 2024-03-30T13:12:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.687+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f59340706c0 con 0x7f593415afc0 2024-03-30T13:12:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.687+0000 7f591b7fe640 1 -- 172.21.15.43:0/594256606 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f5924057090 con 0x7f593415afc0 2024-03-30T13:12:01.695 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5920067fc0 msgr2=0x7f592006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5920067fc0 0x7f592006a480 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f592c0ae000 tx=0x7f592c0a7070 comp rx=0 tx=0).stop 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 msgr2=0x7f59341645a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59341645a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f592400d560 tx=0x7f592400da20 comp rx=0 tx=0).stop 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 shutdown_connections 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5920067fc0 0x7f592006a480 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f593415afc0 0x7f59341645a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f59340b4520 0x7f5934152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 --2- 172.21.15.43:0/594256606 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f59340700f0 0x7f5934162db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 >> 172.21.15.43:0/594256606 conn(0x7f59340d1070 msgr2=0x7f59340b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 shutdown_connections 2024-03-30T13:12:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.691+0000 7f59397e3640 1 -- 172.21.15.43:0/594256606 wait complete. 2024-03-30T13:12:01.720 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:01.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1df5998640 1 Processor -- start 2024-03-30T13:12:01.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1df5998640 1 -- start start 2024-03-30T13:12:01.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1df5998640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1df00c70b0 0x7f1df006db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1df5998640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df006cc80 0x7f1df00c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1df5998640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1df006e0b0 con 0x7f1df00cc860 2024-03-30T13:12:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1df5998640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1df00b2850 con 0x7f1df00c70b0 2024-03-30T13:12:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1df5998640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1df00c79d0 con 0x7f1df006cc80 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1de7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df006cc80 0x7f1df00c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1deeffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1df00c70b0 0x7f1df006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1deeffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1df00c70b0 0x7f1df006db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:33546/0 (socket says 172.21.15.43:33546) 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1def7fe640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f1df00cc860 0x7f1df00b2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:60252/0 (socket says 172.21.15.43:60252) 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1deeffd640 1 -- 172.21.15.43:0/3996811824 learned_addr learned my addr 172.21.15.43:0/3996811824 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1deeffd640 1 -- 172.21.15.43:0/3996811824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df006cc80 msgr2=0x7f1df00c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1deeffd640 1 --2- 172.21.15.43:0/3996811824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df006cc80 0x7f1df00c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1deeffd640 1 -- 172.21.15.43:0/3996811824 >> v1:172.21.15.113:6789/0 conn(0x7f1df00cc860 legacy=0x7f1df00b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.835+0000 7f1deeffd640 1 -- 172.21.15.43:0/3996811824 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df00b1c40 con 0x7f1df00c70b0 2024-03-30T13:12:01.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1deeffd640 1 --2- 172.21.15.43:0/3996811824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00c70b0 0x7f1df006db70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009870 tx=0x7f1de001c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871a5b640a64ed08 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1dee7fc640 1 -- 172.21.15.43:0/3996811824 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a95f0 con 0x7f1df00c70b0 2024-03-30T13:12:01.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1dee7fc640 1 -- 172.21.15.43:0/3996811824 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de00a9790 con 0x7f1df00c70b0 2024-03-30T13:12:01.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1dee7fc640 1 -- 172.21.15.43:0/3996811824 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a9b60 con 0x7f1df00c70b0 2024-03-30T13:12:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 -- 172.21.15.43:0/3996811824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00c70b0 msgr2=0x7f1df006db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 --2- 172.21.15.43:0/3996811824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00c70b0 0x7f1df006db70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009870 tx=0x7f1de001c530 comp rx=0 tx=0).stop 2024-03-30T13:12:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 -- 172.21.15.43:0/3996811824 shutdown_connections 2024-03-30T13:12:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 --2- 172.21.15.43:0/3996811824 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df006cc80 0x7f1df00c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 --2- 172.21.15.43:0/3996811824 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00c70b0 0x7f1df006db70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 -- 172.21.15.43:0/3996811824 >> 172.21.15.43:0/3996811824 conn(0x7f1df00cec90 msgr2=0x7f1df00cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:01.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 -- 172.21.15.43:0/3996811824 shutdown_connections 2024-03-30T13:12:01.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 -- 172.21.15.43:0/3996811824 wait complete. 2024-03-30T13:12:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 Processor -- start 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.839+0000 7f1df5998640 1 -- start start 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df006cc80 0x7f1df015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df00c70b0 0x7f1df00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00cc860 0x7f1df00c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1df01482e0 con 0x7f1df006cc80 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1df0148160 con 0x7f1df00cc860 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1df016b970 con 0x7f1df00c70b0 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df006cc80 0x7f1df015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df006cc80 0x7f1df015f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57216/0 (socket says 172.21.15.43:57216) 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 -- 172.21.15.43:0/1534978145 learned_addr learned my addr 172.21.15.43:0/1534978145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:01.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1de7fff640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df00c70b0 0x7f1df00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1def7fe640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00cc860 0x7f1df00c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 -- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df00c70b0 msgr2=0x7f1df00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df00c70b0 0x7f1df00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 -- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00cc860 msgr2=0x7f1df00c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00cc860 0x7f1df00c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 -- 172.21.15.43:0/1534978145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df006de70 con 0x7f1df006cc80 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1def7fe640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00cc860 0x7f1df00c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1deeffd640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df006cc80 0x7f1df015f060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1de00ab040 tx=0x7f1de001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1decff9640 1 -- 172.21.15.43:0/1534978145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a95f0 con 0x7f1df006cc80 2024-03-30T13:12:01.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1decff9640 1 -- 172.21.15.43:0/1534978145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de00b1800 con 0x7f1df006cc80 2024-03-30T13:12:01.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df00cd0b0 con 0x7f1df006cc80 2024-03-30T13:12:01.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1decff9640 1 -- 172.21.15.43:0/1534978145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de00b1bf0 con 0x7f1df006cc80 2024-03-30T13:12:01.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df00cd650 con 0x7f1df006cc80 2024-03-30T13:12:01.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1df0068890 con 0x7f1df006cc80 2024-03-30T13:12:01.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1decff9640 1 -- 172.21.15.43:0/1534978145 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1de00bc450 con 0x7f1df006cc80 2024-03-30T13:12:01.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1decff9640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1dd0067fc0 0x7f1dd006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:01.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1decff9640 1 -- 172.21.15.43:0/1534978145 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f1de0121700 con 0x7f1df006cc80 2024-03-30T13:12:01.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.843+0000 7f1de7fff640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1dd0067fc0 0x7f1dd006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:01.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.847+0000 7f1de7fff640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1dd0067fc0 0x7f1dd006a480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1df00c9400 tx=0x7f1dd8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:01.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:01.847+0000 7f1decff9640 1 -- 172.21.15.43:0/1534978145 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1de00cb080 con 0x7f1df006cc80 2024-03-30T13:12:02.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.019+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1df006e280 con 0x7f1df006cc80 2024-03-30T13:12:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.019+0000 7f1decff9640 1 -- 172.21.15.43:0/1534978145 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f1de00ad020 con 0x7f1df006cc80 2024-03-30T13:12:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1dd0067fc0 msgr2=0x7f1dd006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1dd0067fc0 0x7f1dd006a480 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1df00c9400 tx=0x7f1dd8008040 comp rx=0 tx=0).stop 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df006cc80 msgr2=0x7f1df015f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df006cc80 0x7f1df015f060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f1de00ab040 tx=0x7f1de001ad70 comp rx=0 tx=0).stop 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 shutdown_connections 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1dd0067fc0 0x7f1dd006a480 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1df00cc860 0x7f1df00c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1df00c70b0 0x7f1df00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 --2- 172.21.15.43:0/1534978145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df006cc80 0x7f1df015f060 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 >> 172.21.15.43:0/1534978145 conn(0x7f1df00cec90 msgr2=0x7f1df006ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 shutdown_connections 2024-03-30T13:12:02.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.023+0000 7f1df5998640 1 -- 172.21.15.43:0/1534978145 wait complete. 2024-03-30T13:12:02.045 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:12:02.045 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:02.052 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:02.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.171+0000 7fadc198e640 1 Processor -- start 2024-03-30T13:12:02.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.171+0000 7fadc198e640 1 -- start start 2024-03-30T13:12:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.171+0000 7fadc198e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadbc147e80 0x7fadbc06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.171+0000 7fadc198e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.171+0000 7fadc198e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadbc06bc40 con 0x7fadbc06c1c0 2024-03-30T13:12:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.171+0000 7fadc198e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fadbc070c50 con 0x7fadbc147e80 2024-03-30T13:12:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.171+0000 7fadc198e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc070600 con 0x7fadbc06c640 2024-03-30T13:12:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadbc147e80 0x7fadbc06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadbc147e80 0x7fadbc06b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57228/0 (socket says 172.21.15.43:57228) 2024-03-30T13:12:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 -- 172.21.15.43:0/407060792 learned_addr learned my addr 172.21.15.43:0/407060792 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbb7fe640 1 --2- 172.21.15.43:0/407060792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 -- 172.21.15.43:0/407060792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 msgr2=0x7fadbc15fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 --2- 172.21.15.43:0/407060792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 -- 172.21.15.43:0/407060792 >> v1:172.21.15.113:6789/0 conn(0x7fadbc06c1c0 legacy=0x7fadbc161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 -- 172.21.15.43:0/407060792 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc0708e0 con 0x7fadbc147e80 2024-03-30T13:12:02.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbb7fe640 1 --2- 172.21.15.43:0/407060792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadbaffd640 1 --2- 172.21.15.43:0/407060792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc06b660 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009870 tx=0x7fadb401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f37199e580a370c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadb9ffb640 1 -- 172.21.15.43:0/407060792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40ab5f0 con 0x7fadbc147e80 2024-03-30T13:12:02.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadb9ffb640 1 -- 172.21.15.43:0/407060792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb40b0070 con 0x7fadbc147e80 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadb9ffb640 1 -- 172.21.15.43:0/407060792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40ab9b0 con 0x7fadbc147e80 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 -- 172.21.15.43:0/407060792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 msgr2=0x7fadbc06b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 --2- 172.21.15.43:0/407060792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc06b660 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009870 tx=0x7fadb401c520 comp rx=0 tx=0).stop 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 -- 172.21.15.43:0/407060792 shutdown_connections 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 --2- 172.21.15.43:0/407060792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 --2- 172.21.15.43:0/407060792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc06b660 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 -- 172.21.15.43:0/407060792 >> 172.21.15.43:0/407060792 conn(0x7fadbc06e410 msgr2=0x7fadbc06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 -- 172.21.15.43:0/407060792 shutdown_connections 2024-03-30T13:12:02.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.175+0000 7fadc198e640 1 -- 172.21.15.43:0/407060792 wait complete. 2024-03-30T13:12:02.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 Processor -- start 2024-03-30T13:12:02.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 -- start start 2024-03-30T13:12:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadbc06c1c0 0x7fadbc144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc160c40 con 0x7fadbc147e80 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc160ac0 con 0x7fadbc06c1c0 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc160dc0 con 0x7fadbc06c640 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc140cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57242/0 (socket says 172.21.15.43:57242) 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 -- 172.21.15.43:0/818008223 learned_addr learned my addr 172.21.15.43:0/818008223 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadba7fc640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbaffd640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadbc06c1c0 0x7fadbc144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 -- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 msgr2=0x7fadbc15dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 -- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadbc06c1c0 msgr2=0x7fadbc144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadbc06c1c0 0x7fadbc144f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 -- 172.21.15.43:0/818008223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc144170 con 0x7fadbc147e80 2024-03-30T13:12:02.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbaffd640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadbc06c1c0 0x7fadbc144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadbb7fe640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc140cd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fadac00cea0 tx=0x7fadac00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fada3fff640 1 -- 172.21.15.43:0/818008223 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac003ba0 con 0x7fadbc147e80 2024-03-30T13:12:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fada3fff640 1 -- 172.21.15.43:0/818008223 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac003d60 con 0x7fadbc147e80 2024-03-30T13:12:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc144450 con 0x7fadbc147e80 2024-03-30T13:12:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fada3fff640 1 -- 172.21.15.43:0/818008223 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac00d880 con 0x7fadbc147e80 2024-03-30T13:12:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.179+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc15e1d0 con 0x7fadbc147e80 2024-03-30T13:12:02.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.183+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadbc068890 con 0x7fadbc147e80 2024-03-30T13:12:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.183+0000 7fada3fff640 1 -- 172.21.15.43:0/818008223 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fadac00da20 con 0x7fadbc147e80 2024-03-30T13:12:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.183+0000 7fada3fff640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fada8067fc0 0x7fada806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.183+0000 7fada3fff640 1 -- 172.21.15.43:0/818008223 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fadac0871d0 con 0x7fadbc147e80 2024-03-30T13:12:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.183+0000 7fadbaffd640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fada8067fc0 0x7fada806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.183+0000 7fadbaffd640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fada8067fc0 0x7fada806a480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadb40af000 tx=0x7fadb4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.187+0000 7fada3fff640 1 -- 172.21.15.43:0/818008223 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fadac0576a0 con 0x7fadbc147e80 2024-03-30T13:12:02.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.351+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadbc1623f0 con 0x7fadbc147e80 2024-03-30T13:12:02.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.355+0000 7fada3fff640 1 -- 172.21.15.43:0/818008223 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7fadac056fa0 con 0x7fadbc147e80 2024-03-30T13:12:02.362 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:02.362 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fada8067fc0 msgr2=0x7fada806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fada8067fc0 0x7fada806a480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadb40af000 tx=0x7fadb4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 msgr2=0x7fadbc140cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc140cd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fadac00cea0 tx=0x7fadac00cf90 comp rx=0 tx=0).stop 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 shutdown_connections 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fada8067fc0 0x7fada806a480 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadbc147e80 0x7fadbc140cd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadbc06c640 0x7fadbc15dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 --2- 172.21.15.43:0/818008223 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadbc06c1c0 0x7fadbc144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 >> 172.21.15.43:0/818008223 conn(0x7fadbc06e410 msgr2=0x7fadbc0724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:02.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 shutdown_connections 2024-03-30T13:12:02.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.359+0000 7fadc198e640 1 -- 172.21.15.43:0/818008223 wait complete. 2024-03-30T13:12:02.387 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:12:02.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:02.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4bceb640 1 Processor -- start 2024-03-30T13:12:02.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4bceb640 1 -- start start 2024-03-30T13:12:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4bceb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d4415f880 0x7f0d440b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4bceb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4406f090 0x7f0d440ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4bceb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d440ce4d0 con 0x7f0d44070640 2024-03-30T13:12:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4bceb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0d44070ad0 con 0x7f0d4415f880 2024-03-30T13:12:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4bceb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d440cc7f0 con 0x7f0d4406f090 2024-03-30T13:12:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4a288640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4406f090 0x7f0d440ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d49a87640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0d4415f880 0x7f0d440b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4a288640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4406f090 0x7f0d440ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:48000/0 (socket says 172.21.15.43:48000) 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4a288640 1 -- 172.21.15.43:0/3512865999 learned_addr learned my addr 172.21.15.43:0/3512865999 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d49286640 1 --1- 172.21.15.43:0/3512865999 >> v1:172.21.15.113:6789/0 conn(0x7f0d44070640 0x7f0d4406e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:60268/0 (socket says 172.21.15.43:60268) 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4a288640 1 -- 172.21.15.43:0/3512865999 >> v2:172.21.15.113:3300/0 conn(0x7f0d4415f880 msgr2=0x7f0d440b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4a288640 1 --2- 172.21.15.43:0/3512865999 >> v2:172.21.15.113:3300/0 conn(0x7f0d4415f880 0x7f0d440b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4a288640 1 -- 172.21.15.43:0/3512865999 >> v1:172.21.15.113:6789/0 conn(0x7f0d44070640 legacy=0x7f0d4406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.511+0000 7f0d4a288640 1 -- 172.21.15.43:0/3512865999 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d440ccad0 con 0x7f0d4406f090 2024-03-30T13:12:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4a288640 1 --2- 172.21.15.43:0/3512865999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4406f090 0x7f0d440ce9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009870 tx=0x7f0d4001c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb99c85a8c2f193 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d48a85640 1 -- 172.21.15.43:0/3512865999 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400b0070 con 0x7f0d4406f090 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d48a85640 1 -- 172.21.15.43:0/3512865999 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d40004230 con 0x7f0d4406f090 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d48a85640 1 -- 172.21.15.43:0/3512865999 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40003620 con 0x7f0d4406f090 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3512865999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4406f090 msgr2=0x7f0d440ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3512865999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4406f090 0x7f0d440ce9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009870 tx=0x7f0d4001c440 comp rx=0 tx=0).stop 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3512865999 shutdown_connections 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3512865999 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4406f090 0x7f0d440ce9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3512865999 >> v2:172.21.15.113:3300/0 conn(0x7f0d4415f880 0x7f0d440b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3512865999 >> 172.21.15.43:0/3512865999 conn(0x7f0d440b3cf0 msgr2=0x7f0d440b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3512865999 shutdown_connections 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3512865999 wait complete. 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 Processor -- start 2024-03-30T13:12:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- start start 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4406f090 0x7f0d441540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d44070640 0x7f0d440c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4415f880 0x7f0d44159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441609d0 con 0x7f0d44070640 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44160850 con 0x7f0d4406f090 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d4bceb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d441691e0 con 0x7f0d4415f880 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d49286640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d44070640 0x7f0d440c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d49286640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d44070640 0x7f0d440c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57256/0 (socket says 172.21.15.43:57256) 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d49286640 1 -- 172.21.15.43:0/3639687242 learned_addr learned my addr 172.21.15.43:0/3639687242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:02.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.515+0000 7f0d49a87640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4406f090 0x7f0d441540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d4a288640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4415f880 0x7f0d44159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49286640 1 -- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4415f880 msgr2=0x7f0d44159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49286640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4415f880 0x7f0d44159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49286640 1 -- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4406f090 msgr2=0x7f0d441540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49286640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4406f090 0x7f0d441540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49286640 1 -- 172.21.15.43:0/3639687242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d44158560 con 0x7f0d44070640 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49a87640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4406f090 0x7f0d441540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49286640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d44070640 0x7f0d440c91d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c450 tx=0x7f0d3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d367fc640 1 -- 172.21.15.43:0/3639687242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38018070 con 0x7f0d44070640 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d367fc640 1 -- 172.21.15.43:0/3639687242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d38007480 con 0x7f0d44070640 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d44158840 con 0x7f0d44070640 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d367fc640 1 -- 172.21.15.43:0/3639687242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d380047f0 con 0x7f0d44070640 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d44155560 con 0x7f0d44070640 2024-03-30T13:12:02.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d44068900 con 0x7f0d44070640 2024-03-30T13:12:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d367fc640 1 -- 172.21.15.43:0/3639687242 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0d3801b070 con 0x7f0d44070640 2024-03-30T13:12:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d367fc640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d28067fc0 0x7f0d2806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d367fc640 1 -- 172.21.15.43:0/3639687242 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0d38085a10 con 0x7f0d44070640 2024-03-30T13:12:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49a87640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d28067fc0 0x7f0d2806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.519+0000 7f0d49a87640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d28067fc0 0x7f0d2806a480 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0074c0 tx=0x7f0d3c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.527+0000 7f0d367fc640 1 -- 172.21.15.43:0/3639687242 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0d38055ee0 con 0x7f0d44070640 2024-03-30T13:12:02.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.687+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d440d1790 con 0x7f0d44070640 2024-03-30T13:12:02.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.687+0000 7f0d367fc640 1 -- 172.21.15.43:0/3639687242 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f0d380557e0 con 0x7f0d44070640 2024-03-30T13:12:02.695 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d28067fc0 msgr2=0x7f0d2806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d28067fc0 0x7f0d2806a480 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0074c0 tx=0x7f0d3c00b040 comp rx=0 tx=0).stop 2024-03-30T13:12:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d44070640 msgr2=0x7f0d440c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d44070640 0x7f0d440c91d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c450 tx=0x7f0d3800c910 comp rx=0 tx=0).stop 2024-03-30T13:12:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 shutdown_connections 2024-03-30T13:12:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d28067fc0 0x7f0d2806a480 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d4415f880 0x7f0d44159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d44070640 0x7f0d440c91d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 --2- 172.21.15.43:0/3639687242 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d4406f090 0x7f0d441540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 >> 172.21.15.43:0/3639687242 conn(0x7f0d440b3cf0 msgr2=0x7f0d440d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:02.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 shutdown_connections 2024-03-30T13:12:02.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.691+0000 7f0d4bceb640 1 -- 172.21.15.43:0/3639687242 wait complete. 2024-03-30T13:12:02.718 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f6650640 1 Processor -- start 2024-03-30T13:12:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f6650640 1 -- start start 2024-03-30T13:12:02.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f6650640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 0x7fe7f006fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f6650640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe7f00b42c0 0x7fe7f00b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f6650640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f0070340 con 0x7fe7f006c790 2024-03-30T13:12:02.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f6650640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f006bd70 con 0x7fe7f00b42c0 2024-03-30T13:12:02.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f6650640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f015f8b0 con 0x7fe7f006f9e0 2024-03-30T13:12:02.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7effff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe7f006c790 0x7fe7f006b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38972/0 (socket says 172.21.15.43:38972) 2024-03-30T13:12:02.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7effff640 1 -- 172.21.15.43:0/4143054607 learned_addr learned my addr 172.21.15.43:0/4143054607 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 0x7fe7f006fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f4bed640 1 --2- 172.21.15.43:0/4143054607 >> v2:172.21.15.113:3300/0 conn(0x7fe7f00b42c0 0x7fe7f00b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123425208 0 0) 0x7fe7f0070340 con 0x7fe7f006c790 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7d8003530 con 0x7fe7f006c790 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1919707790 0 0) 0x7fe7d8003530 con 0x7fe7f006c790 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 msgr2=0x7fe7f006fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 0x7fe7f006fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 >> v2:172.21.15.113:3300/0 conn(0x7fe7f00b42c0 msgr2=0x7fe7f00b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 --2- 172.21.15.43:0/4143054607 >> v2:172.21.15.113:3300/0 conn(0x7fe7f00b42c0 0x7fe7f00b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f015e9e0 con 0x7fe7f006c790 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fe7e0001680 con 0x7fe7f006c790 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 >> v1:172.21.15.43:6789/0 conn(0x7fe7f006c790 legacy=0x7fe7f006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d80039c0 0x7fe7d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 0x7fe7d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f006bd70 con 0x7fe7d8004810 2024-03-30T13:12:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f015f8b0 con 0x7fe7d80039c0 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d800b9d0 con 0x7fe7f006f9e0 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7effff640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d80039c0 0x7fe7d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7f4bed640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 0x7fe7d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7ef7fe640 1 -- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 msgr2=0x7fe7d80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7f006f9e0 0x7fe7d80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7ef7fe640 1 -- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d80039c0 msgr2=0x7fe7d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7d80039c0 0x7fe7d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.835+0000 7fe7ef7fe640 1 -- 172.21.15.43:0/4143054607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d8003530 con 0x7fe7d8004810 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009ae0 tx=0x7fe7e401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68c797046c2c686 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40096b0 con 0x7fe7d8004810 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e40a6480 con 0x7fe7d8004810 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7eeffd640 1 -- 172.21.15.43:0/4143054607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40a6870 con 0x7fe7d8004810 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- 172.21.15.43:0/4143054607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8004810 msgr2=0x7fe7d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 --2- 172.21.15.43:0/4143054607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7d8004810 0x7fe7d8004c90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009ae0 tx=0x7fe7e401ca70 comp rx=0 tx=0).stop 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f4bed640 1 -- 172.21.15.43:0/4143054607 reap_dead start 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- 172.21.15.43:0/4143054607 shutdown_connections 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- 172.21.15.43:0/4143054607 >> 172.21.15.43:0/4143054607 conn(0x7fe7f00d1070 msgr2=0x7fe7f00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:02.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- 172.21.15.43:0/4143054607 shutdown_connections 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- 172.21.15.43:0/4143054607 wait complete. 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 Processor -- start 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- start start 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d8004810 0x7fe7f01509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f00cfca0 0x7fe7f00d0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f01f4950 0x7fe7f01f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f015f260 con 0x7fe7f00cfca0 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f015ef80 con 0x7fe7f01f4950 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f6650640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f015f540 con 0x7fe7d8004810 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f00cfca0 0x7fe7f00d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7ef7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f00cfca0 0x7fe7f00d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57278/0 (socket says 172.21.15.43:57278) 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7ef7fe640 1 -- 172.21.15.43:0/3517145591 learned_addr learned my addr 172.21.15.43:0/3517145591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.839+0000 7fe7f4bed640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f01f4950 0x7fe7f01f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7effff640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d8004810 0x7fe7f01509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ef7fe640 1 -- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d8004810 msgr2=0x7fe7f01509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d8004810 0x7fe7f01509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ef7fe640 1 -- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f01f4950 msgr2=0x7fe7f01f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f01f4950 0x7fe7f01f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ef7fe640 1 -- 172.21.15.43:0/3517145591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f0160d80 con 0x7fe7f00cfca0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ef7fe640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f00cfca0 0x7fe7f00d0120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009ae0 tx=0x7fe7d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ecff9640 1 -- 172.21.15.43:0/3517145591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40c0070 con 0x7fe7f00cfca0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ecff9640 1 -- 172.21.15.43:0/3517145591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e40a6e40 con 0x7fe7f00cfca0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f0160ff0 con 0x7fe7f00cfca0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ecff9640 1 -- 172.21.15.43:0/3517145591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40a7460 con 0x7fe7f00cfca0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f01615b0 con 0x7fe7f00cfca0 2024-03-30T13:12:02.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f0068900 con 0x7fe7f00cfca0 2024-03-30T13:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ecff9640 1 -- 172.21.15.43:0/3517145591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe7e40c3070 con 0x7fe7f00cfca0 2024-03-30T13:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ecff9640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7dc067fc0 0x7fe7dc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7ecff9640 1 -- 172.21.15.43:0/3517145591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fe7e4120910 con 0x7fe7f00cfca0 2024-03-30T13:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7effff640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7dc067fc0 0x7fe7dc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.843+0000 7fe7effff640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7dc067fc0 0x7fe7dc06a480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000fe0 tx=0x7fe7e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:02.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:02.847+0000 7fe7ecff9640 1 -- 172.21.15.43:0/3517145591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe7e40ac020 con 0x7fe7f00cfca0 2024-03-30T13:12:03.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.019+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7f00706c0 con 0x7fe7f00cfca0 2024-03-30T13:12:03.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.019+0000 7fe7ecff9640 1 -- 172.21.15.43:0/3517145591 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fe7e4016780 con 0x7fe7f00cfca0 2024-03-30T13:12:03.028 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:03.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7dc067fc0 msgr2=0x7fe7dc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7dc067fc0 0x7fe7dc06a480 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000fe0 tx=0x7fe7e0009040 comp rx=0 tx=0).stop 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f00cfca0 msgr2=0x7fe7f00d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f00cfca0 0x7fe7f00d0120 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009ae0 tx=0x7fe7d8005840 comp rx=0 tx=0).stop 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 shutdown_connections 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe7dc067fc0 0x7fe7dc06a480 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe7f01f4950 0x7fe7f01f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe7f00cfca0 0x7fe7f00d0120 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 --2- 172.21.15.43:0/3517145591 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe7d8004810 0x7fe7f01509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 >> 172.21.15.43:0/3517145591 conn(0x7fe7f00d1070 msgr2=0x7fe7f00b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 shutdown_connections 2024-03-30T13:12:03.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.023+0000 7fe7f6650640 1 -- 172.21.15.43:0/3517145591 wait complete. 2024-03-30T13:12:03.048 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:03.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff569892640 1 Processor -- start 2024-03-30T13:12:03.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff569892640 1 -- start start 2024-03-30T13:12:03.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff569892640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff56406f090 0x7ff5640b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff569892640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff564070640 0x7ff5640ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff569892640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5640ce4d0 con 0x7ff56415f880 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff569892640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff564070a50 con 0x7ff56406f090 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff569892640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff5640cc7f0 con 0x7ff564070640 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff5627fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff56406f090 0x7ff5640b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff562ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff56415f880 0x7ff56406e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38974/0 (socket says 172.21.15.43:38974) 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff562ffd640 1 -- 172.21.15.43:0/3420384442 learned_addr learned my addr 172.21.15.43:0/3420384442 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff5637fe640 1 --2- 172.21.15.43:0/3420384442 >> v2:172.21.15.113:3300/0 conn(0x7ff564070640 0x7ff5640ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff5627fc640 1 -- 172.21.15.43:0/3420384442 >> v2:172.21.15.113:3300/0 conn(0x7ff564070640 msgr2=0x7ff5640ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff5627fc640 1 --2- 172.21.15.43:0/3420384442 >> v2:172.21.15.113:3300/0 conn(0x7ff564070640 0x7ff5640ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff5627fc640 1 -- 172.21.15.43:0/3420384442 >> v1:172.21.15.43:6789/0 conn(0x7ff56415f880 legacy=0x7ff56406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff5627fc640 1 -- 172.21.15.43:0/3420384442 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5640ccad0 con 0x7ff56406f090 2024-03-30T13:12:03.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff561ffb640 1 -- 172.21.15.43:0/3420384442 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804679581 0 0) 0x7ff5640ce4d0 con 0x7ff56415f880 2024-03-30T13:12:03.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.159+0000 7ff5637fe640 1 --2- 172.21.15.43:0/3420384442 >> v2:172.21.15.113:3300/0 conn(0x7ff564070640 0x7ff5640ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:03.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff5627fc640 1 --2- 172.21.15.43:0/3420384442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff5640b4ff0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5640cc6e0 tx=0x7ff55401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557c4c6cd10b3ccb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff561ffb640 1 -- 172.21.15.43:0/3420384442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540b0070 con 0x7ff56406f090 2024-03-30T13:12:03.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff561ffb640 1 -- 172.21.15.43:0/3420384442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5540ab600 con 0x7ff56406f090 2024-03-30T13:12:03.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff561ffb640 1 -- 172.21.15.43:0/3420384442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5540ab9d0 con 0x7ff56406f090 2024-03-30T13:12:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- 172.21.15.43:0/3420384442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 msgr2=0x7ff5640b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 --2- 172.21.15.43:0/3420384442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff5640b4ff0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5640cc6e0 tx=0x7ff55401c360 comp rx=0 tx=0).stop 2024-03-30T13:12:03.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- 172.21.15.43:0/3420384442 shutdown_connections 2024-03-30T13:12:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 --2- 172.21.15.43:0/3420384442 >> v2:172.21.15.113:3300/0 conn(0x7ff564070640 0x7ff5640ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 --2- 172.21.15.43:0/3420384442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff5640b4ff0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- 172.21.15.43:0/3420384442 >> 172.21.15.43:0/3420384442 conn(0x7ff5640b3cf0 msgr2=0x7ff5640b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- 172.21.15.43:0/3420384442 shutdown_connections 2024-03-30T13:12:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- 172.21.15.43:0/3420384442 wait complete. 2024-03-30T13:12:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 Processor -- start 2024-03-30T13:12:03.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- start start 2024-03-30T13:12:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff56406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff564070640 0x7ff5641484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff56415f880 0x7ff56415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff5641609d0 con 0x7ff56406f090 2024-03-30T13:12:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff564160850 con 0x7ff56415f880 2024-03-30T13:12:03.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff569892640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff5641691e0 con 0x7ff564070640 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff56406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff562ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff56406d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57292/0 (socket says 172.21.15.43:57292) 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.163+0000 7ff562ffd640 1 -- 172.21.15.43:0/1610214227 learned_addr learned my addr 172.21.15.43:0/1610214227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff5627fc640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff564070640 0x7ff5641484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff5637fe640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff56415f880 0x7ff56415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff562ffd640 1 -- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff564070640 msgr2=0x7ff5641484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff562ffd640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff564070640 0x7ff5641484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff562ffd640 1 -- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff56415f880 msgr2=0x7ff56415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff562ffd640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff56415f880 0x7ff56415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff562ffd640 1 -- 172.21.15.43:0/1610214227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5640c9620 con 0x7ff56406f090 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff5637fe640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff56415f880 0x7ff56415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff562ffd640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff56406d4a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff558007fd0 tx=0x7ff55800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff54bfff640 1 -- 172.21.15.43:0/1610214227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558017070 con 0x7ff56406f090 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff54bfff640 1 -- 172.21.15.43:0/1610214227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff558005d10 con 0x7ff56406f090 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56406c9c0 con 0x7ff56406f090 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff54bfff640 1 -- 172.21.15.43:0/1610214227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff558007310 con 0x7ff56406f090 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56406cf80 con 0x7ff56406f090 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff564068900 con 0x7ff56406f090 2024-03-30T13:12:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff54bfff640 1 -- 172.21.15.43:0/1610214227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff55801a070 con 0x7ff56406f090 2024-03-30T13:12:03.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff54bfff640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff550067fc0 0x7ff55006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff54bfff640 1 -- 172.21.15.43:0/1610214227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7ff558085c80 con 0x7ff56406f090 2024-03-30T13:12:03.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff5627fc640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff550067fc0 0x7ff55006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.167+0000 7ff5627fc640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff550067fc0 0x7ff55006a480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff5540ad000 tx=0x7ff554008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.171+0000 7ff54bfff640 1 -- 172.21.15.43:0/1610214227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff5580560c0 con 0x7ff56406f090 2024-03-30T13:12:03.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.347+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff5640d1790 con 0x7ff56406f090 2024-03-30T13:12:03.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.347+0000 7ff54bfff640 1 -- 172.21.15.43:0/1610214227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7ff5580559c0 con 0x7ff56406f090 2024-03-30T13:12:03.354 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:12:03.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff550067fc0 msgr2=0x7ff55006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff550067fc0 0x7ff55006a480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff5540ad000 tx=0x7ff554008040 comp rx=0 tx=0).stop 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 msgr2=0x7ff56406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff56406d4a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff558007fd0 tx=0x7ff55800c8e0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 shutdown_connections 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff550067fc0 0x7ff55006a480 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff56415f880 0x7ff56415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff564070640 0x7ff5641484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 --2- 172.21.15.43:0/1610214227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff56406f090 0x7ff56406d4a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 >> 172.21.15.43:0/1610214227 conn(0x7ff5640b3cf0 msgr2=0x7ff5640b7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 shutdown_connections 2024-03-30T13:12:03.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.351+0000 7ff569892640 1 -- 172.21.15.43:0/1610214227 wait complete. 2024-03-30T13:12:03.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:12:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c8cd0b640 1 Processor -- start 2024-03-30T13:12:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c8cd0b640 1 -- start start 2024-03-30T13:12:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c8cd0b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c880b46f0 0x7f1c8806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c8cd0b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c8806ff20 0x7f1c880ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c8cd0b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c880b4ba0 con 0x7f1c8815afc0 2024-03-30T13:12:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c8cd0b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1c88070390 con 0x7f1c8806ff20 2024-03-30T13:12:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c8cd0b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1c88070c30 con 0x7f1c880b46f0 2024-03-30T13:12:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c86d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c8806ff20 0x7f1c880ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c86d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1c8806ff20 0x7f1c880ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57300/0 (socket says 172.21.15.43:57300) 2024-03-30T13:12:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c85d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1c880b46f0 0x7f1c8806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c86d9d640 1 -- 172.21.15.43:0/1881767900 learned_addr learned my addr 172.21.15.43:0/1881767900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c86d9d640 1 -- 172.21.15.43:0/1881767900 >> v2:172.21.15.113:3300/0 conn(0x7f1c880b46f0 msgr2=0x7f1c8806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c86d9d640 1 --2- 172.21.15.43:0/1881767900 >> v2:172.21.15.113:3300/0 conn(0x7f1c880b46f0 0x7f1c8806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c86d9d640 1 -- 172.21.15.43:0/1881767900 >> v1:172.21.15.113:6789/0 conn(0x7f1c8815afc0 legacy=0x7f1c880b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.495+0000 7f1c86d9d640 1 -- 172.21.15.43:0/1881767900 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c8815ec60 con 0x7f1c8806ff20 2024-03-30T13:12:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c86d9d640 1 --2- 172.21.15.43:0/1881767900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c880ce7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1c880737e0 tx=0x7f1c8001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed5f70a87478fa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8559a640 1 -- 172.21.15.43:0/1881767900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c800aa650 con 0x7f1c8806ff20 2024-03-30T13:12:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8559a640 1 -- 172.21.15.43:0/1881767900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c800af070 con 0x7f1c8806ff20 2024-03-30T13:12:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8559a640 1 -- 172.21.15.43:0/1881767900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c800aaa10 con 0x7f1c8806ff20 2024-03-30T13:12:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/1881767900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 msgr2=0x7f1c880ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/1881767900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c880ce7b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1c880737e0 tx=0x7f1c8001c4f0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/1881767900 shutdown_connections 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/1881767900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c880ce7b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/1881767900 >> v2:172.21.15.113:3300/0 conn(0x7f1c880b46f0 0x7f1c8806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/1881767900 >> 172.21.15.43:0/1881767900 conn(0x7f1c880d1070 msgr2=0x7f1c880d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/1881767900 shutdown_connections 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/1881767900 wait complete. 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 Processor -- start 2024-03-30T13:12:03.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.499+0000 7f1c8cd0b640 1 -- start start 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c88151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c880b46f0 0x7f1c8815c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c8815afc0 0x7f1c881f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8815fa20 con 0x7f1c8806ff20 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1c8815b3a0 con 0x7f1c880b46f0 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1c8815a0e0 con 0x7f1c8815afc0 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c88151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c88151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57302/0 (socket says 172.21.15.43:57302) 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 -- 172.21.15.43:0/2237014900 learned_addr learned my addr 172.21.15.43:0/2237014900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c85d9b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c880b46f0 0x7f1c8815c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c86d9d640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c8815afc0 0x7f1c881f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 -- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c8815afc0 msgr2=0x7f1c881f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c8815afc0 0x7f1c881f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 -- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c880b46f0 msgr2=0x7f1c8815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c880b46f0 0x7f1c8815c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 -- 172.21.15.43:0/2237014900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c881594d0 con 0x7f1c8806ff20 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8659c640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c88151950 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c7c0 tx=0x7f1c7800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c76ffd640 1 -- 172.21.15.43:0/2237014900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78016020 con 0x7f1c8806ff20 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c76ffd640 1 -- 172.21.15.43:0/2237014900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c78006d60 con 0x7f1c8806ff20 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c881597b0 con 0x7f1c8806ff20 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c76ffd640 1 -- 172.21.15.43:0/2237014900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78003bc0 con 0x7f1c8806ff20 2024-03-30T13:12:03.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c88159d70 con 0x7f1c8806ff20 2024-03-30T13:12:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c88068900 con 0x7f1c8806ff20 2024-03-30T13:12:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c76ffd640 1 -- 172.21.15.43:0/2237014900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1c7800a850 con 0x7f1c8806ff20 2024-03-30T13:12:03.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c76ffd640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c68067fc0 0x7f1c6806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c76ffd640 1 -- 172.21.15.43:0/2237014900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f1c78086550 con 0x7f1c8806ff20 2024-03-30T13:12:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.503+0000 7f1c85d9b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c68067fc0 0x7f1c6806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.507+0000 7f1c85d9b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c68067fc0 0x7f1c6806a480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1c8815a720 tx=0x7f1c7c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.507+0000 7f1c76ffd640 1 -- 172.21.15.43:0/2237014900 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1c78020020 con 0x7f1c8806ff20 2024-03-30T13:12:03.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.667+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1c88155d10 con 0x7f1c8806ff20 2024-03-30T13:12:03.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.667+0000 7f1c76ffd640 1 -- 172.21.15.43:0/2237014900 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f1c78019310 con 0x7f1c8806ff20 2024-03-30T13:12:03.675 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:03.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c68067fc0 msgr2=0x7f1c6806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c68067fc0 0x7f1c6806a480 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1c8815a720 tx=0x7f1c7c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:03.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 msgr2=0x7f1c88151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c88151950 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c7c0 tx=0x7f1c7800cc80 comp rx=0 tx=0).stop 2024-03-30T13:12:03.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 shutdown_connections 2024-03-30T13:12:03.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1c68067fc0 0x7f1c6806a480 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1c8815afc0 0x7f1c881f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1c880b46f0 0x7f1c8815c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 --2- 172.21.15.43:0/2237014900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1c8806ff20 0x7f1c88151950 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 >> 172.21.15.43:0/2237014900 conn(0x7f1c880d1070 msgr2=0x7f1c88156230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:03.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 shutdown_connections 2024-03-30T13:12:03.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.671+0000 7f1c8cd0b640 1 -- 172.21.15.43:0/2237014900 wait complete. 2024-03-30T13:12:03.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:03.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.799+0000 7f8281002640 1 Processor -- start 2024-03-30T13:12:03.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.799+0000 7f8281002640 1 -- start start 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.799+0000 7f8281002640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 0x7f827c0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.799+0000 7f8281002640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f827c0700f0 0x7f827c06b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.799+0000 7f8281002640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f827c0cfc00 con 0x7f827c070a00 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.799+0000 7f8281002640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f827c06e110 con 0x7f827c0700f0 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.799+0000 7f8281002640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f827c146100 con 0x7f827c145d20 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827b59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f827c0700f0 0x7f827c06b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f827c070a00 0x7f827c06f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38976/0 (socket says 172.21.15.43:38976) 2024-03-30T13:12:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827a59c640 1 -- 172.21.15.43:0/4056264510 learned_addr learned my addr 172.21.15.43:0/4056264510 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827ad9d640 1 --2- 172.21.15.43:0/4056264510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 0x7f827c0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827b59e640 1 -- 172.21.15.43:0/4056264510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 msgr2=0x7f827c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827b59e640 1 --2- 172.21.15.43:0/4056264510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 0x7f827c0cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827b59e640 1 -- 172.21.15.43:0/4056264510 >> v1:172.21.15.43:6789/0 conn(0x7f827c070a00 legacy=0x7f827c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827b59e640 1 -- 172.21.15.43:0/4056264510 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c0cfe80 con 0x7f827c0700f0 2024-03-30T13:12:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8279d9b640 1 -- 172.21.15.43:0/4056264510 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655947120 0 0) 0x7f8264001120 con 0x7f827c070a00 2024-03-30T13:12:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f827b59e640 1 --2- 172.21.15.43:0/4056264510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c06b920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f82740097c0 tx=0x7f827401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d48c5e1d466f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8279d9b640 1 -- 172.21.15.43:0/4056264510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740b0070 con 0x7f827c0700f0 2024-03-30T13:12:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8279d9b640 1 -- 172.21.15.43:0/4056264510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82740ab600 con 0x7f827c0700f0 2024-03-30T13:12:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8279d9b640 1 -- 172.21.15.43:0/4056264510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740ab9d0 con 0x7f827c0700f0 2024-03-30T13:12:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 -- 172.21.15.43:0/4056264510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 msgr2=0x7f827c06b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 --2- 172.21.15.43:0/4056264510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c06b920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f82740097c0 tx=0x7f827401c410 comp rx=0 tx=0).stop 2024-03-30T13:12:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 -- 172.21.15.43:0/4056264510 shutdown_connections 2024-03-30T13:12:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 --2- 172.21.15.43:0/4056264510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c06b920 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 --2- 172.21.15.43:0/4056264510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 0x7f827c0cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 -- 172.21.15.43:0/4056264510 >> 172.21.15.43:0/4056264510 conn(0x7f827c06e410 msgr2=0x7f827c06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 -- 172.21.15.43:0/4056264510 shutdown_connections 2024-03-30T13:12:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.803+0000 7f8281002640 1 -- 172.21.15.43:0/4056264510 wait complete. 2024-03-30T13:12:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 Processor -- start 2024-03-30T13:12:03.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 -- start start 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c1656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c070a00 0x7f827c13d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 0x7f827c13dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f827c06c7f0 con 0x7f827c0700f0 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f827c06c670 con 0x7f827c070a00 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f827c06c970 con 0x7f827c145d20 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c1656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c1656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57330/0 (socket says 172.21.15.43:57330) 2024-03-30T13:12:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 -- 172.21.15.43:0/4074084491 learned_addr learned my addr 172.21.15.43:0/4074084491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 -- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 msgr2=0x7f827c13dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 0x7f827c13dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 -- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c070a00 msgr2=0x7f827c13d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c070a00 0x7f827c13d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 -- 172.21.15.43:0/4074084491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c1597f0 con 0x7f827c0700f0 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f827ad9d640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c1656a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f826c00efc0 tx=0x7f826c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f82737fe640 1 -- 172.21.15.43:0/4074084491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c018020 con 0x7f827c0700f0 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f82737fe640 1 -- 172.21.15.43:0/4074084491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c014480 con 0x7f827c0700f0 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c159a90 con 0x7f827c0700f0 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f82737fe640 1 -- 172.21.15.43:0/4074084491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c014850 con 0x7f827c0700f0 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c15a050 con 0x7f827c0700f0 2024-03-30T13:12:03.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.807+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c068890 con 0x7f827c0700f0 2024-03-30T13:12:03.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.811+0000 7f82737fe640 1 -- 172.21.15.43:0/4074084491 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f826c01b020 con 0x7f827c0700f0 2024-03-30T13:12:03.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.811+0000 7f82737fe640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:03.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.811+0000 7f82737fe640 1 -- 172.21.15.43:0/4074084491 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f826c087330 con 0x7f827c0700f0 2024-03-30T13:12:03.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.811+0000 7f827a59c640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:03.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.811+0000 7f827a59c640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b1160 tx=0x7f8264001320 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.815+0000 7f82737fe640 1 -- 172.21.15.43:0/4074084491 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f826c022070 con 0x7f827c0700f0 2024-03-30T13:12:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.987+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f827c06f410 con 0x7f827c0700f0 2024-03-30T13:12:03.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.987+0000 7f82737fe640 1 -- 172.21.15.43:0/4074084491 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f826c010040 con 0x7f827c0700f0 2024-03-30T13:12:03.996 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:12:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 msgr2=0x7f825c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b1160 tx=0x7f8264001320 comp rx=0 tx=0).stop 2024-03-30T13:12:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 msgr2=0x7f827c1656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c1656a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f826c00efc0 tx=0x7f826c00c490 comp rx=0 tx=0).stop 2024-03-30T13:12:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 shutdown_connections 2024-03-30T13:12:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f825c067fc0 0x7f825c06a480 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f827c145d20 0x7f827c13dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f827c070a00 0x7f827c13d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 --2- 172.21.15.43:0/4074084491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f827c0700f0 0x7f827c1656a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 >> 172.21.15.43:0/4074084491 conn(0x7f827c06e410 msgr2=0x7f827c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 shutdown_connections 2024-03-30T13:12:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:03.991+0000 7f8281002640 1 -- 172.21.15.43:0/4074084491 wait complete. 2024-03-30T13:12:04.015 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi113.front.sepia.ceph.com... 2024-03-30T13:12:04.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:04.015 DEBUG:teuthology.orchestra.run.smithi113:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:04.046 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:12:04.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:12:04.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:04.047 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T13:12:04.097 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:12:04.098 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:04.103 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:12:04.104 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:04.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:04.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e8a29640 1 Processor -- start 2024-03-30T13:12:04.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e8a29640 1 -- start start 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e8a29640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6e4145d20 0x7fe6e40cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e8a29640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e8a29640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6e40cfc00 con 0x7fe6e406ff20 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e8a29640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe6e406e110 con 0x7fe6e4145d20 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e8a29640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e4070350 con 0x7fe6e4070a00 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6e4145d20 0x7fe6e40cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe6e406ff20 0x7fe6e415fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38986/0 (socket says 172.21.15.43:38986) 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e2d9d640 1 -- 172.21.15.43:0/3615251948 learned_addr learned my addr 172.21.15.43:0/3615251948 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6dbfff640 1 --2- 172.21.15.43:0/3615251948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e259c640 1 -- 172.21.15.43:0/3615251948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 msgr2=0x7fe6e406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e259c640 1 --2- 172.21.15.43:0/3615251948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e259c640 1 -- 172.21.15.43:0/3615251948 >> v1:172.21.15.43:6789/0 conn(0x7fe6e406ff20 legacy=0x7fe6e415fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e259c640 1 -- 172.21.15.43:0/3615251948 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e4146100 con 0x7fe6e4145d20 2024-03-30T13:12:04.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e1d9b640 1 -- 172.21.15.43:0/3615251948 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1785439337 0 0) 0x7fe6dc0014a0 con 0x7fe6e406ff20 2024-03-30T13:12:04.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6dbfff640 1 --2- 172.21.15.43:0/3615251948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:04.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e259c640 1 --2- 172.21.15.43:0/3615251948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e4145d20 0x7fe6e40cf6c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0097f0 tx=0x7fe6cc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf5b0e86e8fe0e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:04.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.311+0000 7fe6e1d9b640 1 -- 172.21.15.43:0/3615251948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0b0070 con 0x7fe6e4145d20 2024-03-30T13:12:04.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e1d9b640 1 -- 172.21.15.43:0/3615251948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6cc0ab600 con 0x7fe6e4145d20 2024-03-30T13:12:04.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e1d9b640 1 -- 172.21.15.43:0/3615251948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0ab9d0 con 0x7fe6e4145d20 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- 172.21.15.43:0/3615251948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e4145d20 msgr2=0x7fe6e40cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/3615251948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e4145d20 0x7fe6e40cf6c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0097f0 tx=0x7fe6cc01c410 comp rx=0 tx=0).stop 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- 172.21.15.43:0/3615251948 shutdown_connections 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/3615251948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/3615251948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e4145d20 0x7fe6e40cf6c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- 172.21.15.43:0/3615251948 >> 172.21.15.43:0/3615251948 conn(0x7fe6e406e410 msgr2=0x7fe6e406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- 172.21.15.43:0/3615251948 shutdown_connections 2024-03-30T13:12:04.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- 172.21.15.43:0/3615251948 wait complete. 2024-03-30T13:12:04.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 Processor -- start 2024-03-30T13:12:04.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- start start 2024-03-30T13:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e406ff20 0x7fe6e41413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e415af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e4145d20 0x7fe6e41629b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e4169180 con 0x7fe6e406ff20 2024-03-30T13:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e40d0720 con 0x7fe6e4145d20 2024-03-30T13:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e8a29640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e4169300 con 0x7fe6e4070a00 2024-03-30T13:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e406ff20 0x7fe6e41413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e406ff20 0x7fe6e41413a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57344/0 (socket says 172.21.15.43:57344) 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 -- 172.21.15.43:0/4165944197 learned_addr learned my addr 172.21.15.43:0/4165944197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 -- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 msgr2=0x7fe6e415af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 -- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e4145d20 msgr2=0x7fe6e41629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e4145d20 0x7fe6e41629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 -- 172.21.15.43:0/4165944197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e415d020 con 0x7fe6e406ff20 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.315+0000 7fe6e259c640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e406ff20 0x7fe6e41413a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0097f0 tx=0x7fe6cc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6daffd640 1 -- 172.21.15.43:0/4165944197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0b0050 con 0x7fe6e406ff20 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6daffd640 1 -- 172.21.15.43:0/4165944197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6cc0bc070 con 0x7fe6e406ff20 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e4163500 con 0x7fe6e406ff20 2024-03-30T13:12:04.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6daffd640 1 -- 172.21.15.43:0/4165944197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0b5a30 con 0x7fe6e406ff20 2024-03-30T13:12:04.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e4140ba0 con 0x7fe6e406ff20 2024-03-30T13:12:04.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e4068890 con 0x7fe6e406ff20 2024-03-30T13:12:04.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6daffd640 1 -- 172.21.15.43:0/4165944197 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe6cc0ab4c0 con 0x7fe6e406ff20 2024-03-30T13:12:04.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6daffd640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6c8067ff0 0x7fe6c806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6daffd640 1 -- 172.21.15.43:0/4165944197 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fe6cc1210d0 con 0x7fe6e406ff20 2024-03-30T13:12:04.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6dbfff640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6c8067ff0 0x7fe6c806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.319+0000 7fe6dbfff640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6c8067ff0 0x7fe6c806a4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe6e41f87d0 tx=0x7fe6d4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:04.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.323+0000 7fe6daffd640 1 -- 172.21.15.43:0/4165944197 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe6cc016090 con 0x7fe6e406ff20 2024-03-30T13:12:04.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.551+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6e406f410 con 0x7fe6e406ff20 2024-03-30T13:12:04.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6daffd640 1 -- 172.21.15.43:0/4165944197 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7fe6cc0f1510 con 0x7fe6e406ff20 2024-03-30T13:12:04.562 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:04.562 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6c8067ff0 msgr2=0x7fe6c806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6c8067ff0 0x7fe6c806a4b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe6e41f87d0 tx=0x7fe6d4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e406ff20 msgr2=0x7fe6e41413a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e406ff20 0x7fe6e41413a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc0097f0 tx=0x7fe6cc01ac50 comp rx=0 tx=0).stop 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 shutdown_connections 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe6c8067ff0 0x7fe6c806a4b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6e4145d20 0x7fe6e41629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe6e4070a00 0x7fe6e415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 --2- 172.21.15.43:0/4165944197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e406ff20 0x7fe6e41413a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 >> 172.21.15.43:0/4165944197 conn(0x7fe6e406e410 msgr2=0x7fe6e4072e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:04.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 shutdown_connections 2024-03-30T13:12:04.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.555+0000 7fe6e8a29640 1 -- 172.21.15.43:0/4165944197 wait complete. 2024-03-30T13:12:04.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:12:04.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:04.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 Processor -- start 2024-03-30T13:12:04.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 -- start start 2024-03-30T13:12:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87d406c790 0x7f87d406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f87d406f9e0 0x7f87d406fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f87d4070370 con 0x7f87d406c790 2024-03-30T13:12:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f87d415f8b0 con 0x7f87d406f9e0 2024-03-30T13:12:04.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d8c57640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f87d415e9e0 con 0x7f87d4161fd0 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87d406c790 0x7f87d406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87d406c790 0x7f87d406b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57348/0 (socket says 172.21.15.43:57348) 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 -- 172.21.15.43:0/2661776843 learned_addr learned my addr 172.21.15.43:0/2661776843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d1d9b640 1 --2- 172.21.15.43:0/2661776843 >> v2:172.21.15.113:3300/0 conn(0x7f87d406f9e0 0x7f87d406fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d2d9d640 1 --2- 172.21.15.43:0/2661776843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 -- 172.21.15.43:0/2661776843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 msgr2=0x7f87d4162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 --2- 172.21.15.43:0/2661776843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 -- 172.21.15.43:0/2661776843 >> v2:172.21.15.113:3300/0 conn(0x7f87d406f9e0 msgr2=0x7f87d406fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:04.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 --2- 172.21.15.43:0/2661776843 >> v2:172.21.15.113:3300/0 conn(0x7f87d406f9e0 0x7f87d406fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.699+0000 7f87d259c640 1 -- 172.21.15.43:0/2661776843 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d415eca0 con 0x7f87d406c790 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d259c640 1 --2- 172.21.15.43:0/2661776843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d406b660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f87d40d2060 tx=0x7f87c401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb632704c7a45862 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d159a640 1 -- 172.21.15.43:0/2661776843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40aa610 con 0x7f87d406c790 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d159a640 1 -- 172.21.15.43:0/2661776843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c40af070 con 0x7f87d406c790 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d159a640 1 -- 172.21.15.43:0/2661776843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40aa9d0 con 0x7f87d406c790 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- 172.21.15.43:0/2661776843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 msgr2=0x7f87d406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 --2- 172.21.15.43:0/2661776843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d406b660 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f87d40d2060 tx=0x7f87c401c4f0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- 172.21.15.43:0/2661776843 shutdown_connections 2024-03-30T13:12:04.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 --2- 172.21.15.43:0/2661776843 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4162420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 --2- 172.21.15.43:0/2661776843 >> v2:172.21.15.113:3300/0 conn(0x7f87d406f9e0 0x7f87d406fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 --2- 172.21.15.43:0/2661776843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d406b660 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- 172.21.15.43:0/2661776843 >> 172.21.15.43:0/2661776843 conn(0x7f87d40d1070 msgr2=0x7f87d40d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- 172.21.15.43:0/2661776843 shutdown_connections 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- 172.21.15.43:0/2661776843 wait complete. 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 Processor -- start 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- start start 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d41496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87d406f9e0 0x7f87d4156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4070b20 con 0x7f87d406c790 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4162d70 con 0x7f87d406f9e0 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d8c57640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f87d406cb70 con 0x7f87d4161fd0 2024-03-30T13:12:04.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d41496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d41496a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57364/0 (socket says 172.21.15.43:57364) 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d259c640 1 -- 172.21.15.43:0/1620565957 learned_addr learned my addr 172.21.15.43:0/1620565957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.703+0000 7f87d2d9d640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d1d9b640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87d406f9e0 0x7f87d4156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d259c640 1 -- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 msgr2=0x7f87d4157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d259c640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d259c640 1 -- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87d406f9e0 msgr2=0x7f87d4156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d259c640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87d406f9e0 0x7f87d4156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d259c640 1 -- 172.21.15.43:0/1620565957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d414d6b0 con 0x7f87d406c790 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d259c640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d41496a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f87c401aba0 tx=0x7f87c401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87c2ffd640 1 -- 172.21.15.43:0/1620565957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40bd5f0 con 0x7f87d406c790 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87c2ffd640 1 -- 172.21.15.43:0/1620565957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c40bd790 con 0x7f87d406c790 2024-03-30T13:12:04.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d414d990 con 0x7f87d406c790 2024-03-30T13:12:04.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87c2ffd640 1 -- 172.21.15.43:0/1620565957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c40bdb80 con 0x7f87d406c790 2024-03-30T13:12:04.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d414df50 con 0x7f87d406c790 2024-03-30T13:12:04.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d4068900 con 0x7f87d406c790 2024-03-30T13:12:04.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87c2ffd640 1 -- 172.21.15.43:0/1620565957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f87c40b33f0 con 0x7f87d406c790 2024-03-30T13:12:04.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87c2ffd640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f87b4067fc0 0x7f87b406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:04.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87c2ffd640 1 -- 172.21.15.43:0/1620565957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f87c41214f0 con 0x7f87d406c790 2024-03-30T13:12:04.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d1d9b640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f87b4067fc0 0x7f87b406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:04.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.707+0000 7f87d1d9b640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f87b4067fc0 0x7f87b406a480 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f87d41faa90 tx=0x7f87c8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:04.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.711+0000 7f87c2ffd640 1 -- 172.21.15.43:0/1620565957 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f87c40c3060 con 0x7f87d406c790 2024-03-30T13:12:04.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87d415e7c0 con 0x7f87d406c790 2024-03-30T13:12:04.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87c2ffd640 1 -- 172.21.15.43:0/1620565957 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f87c40a7070 con 0x7f87d406c790 2024-03-30T13:12:04.882 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f87b4067fc0 msgr2=0x7f87b406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f87b4067fc0 0x7f87b406a480 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f87d41faa90 tx=0x7f87c8008040 comp rx=0 tx=0).stop 2024-03-30T13:12:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 msgr2=0x7f87d41496a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d41496a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f87c401aba0 tx=0x7f87c401ad30 comp rx=0 tx=0).stop 2024-03-30T13:12:04.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 shutdown_connections 2024-03-30T13:12:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f87b4067fc0 0x7f87b406a480 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f87d4161fd0 0x7f87d4157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f87d406f9e0 0x7f87d4156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 --2- 172.21.15.43:0/1620565957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87d406c790 0x7f87d41496a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 >> 172.21.15.43:0/1620565957 conn(0x7f87d40d1070 msgr2=0x7f87d4161240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 shutdown_connections 2024-03-30T13:12:04.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:04.875+0000 7f87d8c57640 1 -- 172.21.15.43:0/1620565957 wait complete. 2024-03-30T13:12:04.901 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:05.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.019+0000 7fdda1e14640 1 Processor -- start 2024-03-30T13:12:05.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.019+0000 7fdda1e14640 1 -- start start 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cf2b0 0x7fdd9c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd9c0cfe10 0x7fdd9c06f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd9c06ebc0 con 0x7fdd9c0cd870 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdd9c0cf690 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9c06c470 con 0x7fdd9c0cf2b0 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd9c0cfe10 0x7fdd9c06f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd9c0cfe10 0x7fdd9c06f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57372/0 (socket says 172.21.15.43:57372) 2024-03-30T13:12:05.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 -- 172.21.15.43:0/1372355562 learned_addr learned my addr 172.21.15.43:0/1372355562 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:05.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9b7fe640 1 --2- 172.21.15.43:0/1372355562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cf2b0 0x7fdd9c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 -- 172.21.15.43:0/1372355562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cf2b0 msgr2=0x7fdd9c0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:05.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 --2- 172.21.15.43:0/1372355562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cf2b0 0x7fdd9c0cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 -- 172.21.15.43:0/1372355562 >> v1:172.21.15.113:6789/0 conn(0x7fdd9c0cd870 legacy=0x7fdd9c072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 -- 172.21.15.43:0/1372355562 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c06c750 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9affd640 1 --2- 172.21.15.43:0/1372355562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c06f070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009870 tx=0x7fdd9001d670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1416872c55af2e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9a7fc640 1 -- 172.21.15.43:0/1372355562 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd900aa610 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9a7fc640 1 -- 172.21.15.43:0/1372355562 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd900aa7b0 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdd9a7fc640 1 -- 172.21.15.43:0/1372355562 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd900aab80 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- 172.21.15.43:0/1372355562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 msgr2=0x7fdd9c06f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 --2- 172.21.15.43:0/1372355562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c06f070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009870 tx=0x7fdd9001d670 comp rx=0 tx=0).stop 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- 172.21.15.43:0/1372355562 shutdown_connections 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 --2- 172.21.15.43:0/1372355562 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c06f070 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 --2- 172.21.15.43:0/1372355562 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cf2b0 0x7fdd9c0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- 172.21.15.43:0/1372355562 >> 172.21.15.43:0/1372355562 conn(0x7fdd9c0b1e90 msgr2=0x7fdd9c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- 172.21.15.43:0/1372355562 shutdown_connections 2024-03-30T13:12:05.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.023+0000 7fdda1e14640 1 -- 172.21.15.43:0/1372355562 wait complete. 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 Processor -- start 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 -- start start 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cd870 0x7fdd9c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c0cf2b0 0x7fdd9c1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c0734d0 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c073350 con 0x7fdd9c0cf2b0 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9c0ce3d0 con 0x7fdd9c0cd870 2024-03-30T13:12:05.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c15ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57382/0 (socket says 172.21.15.43:57382) 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 -- 172.21.15.43:0/2795849005 learned_addr learned my addr 172.21.15.43:0/2795849005 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9affd640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c0cf2b0 0x7fdd9c1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9b7fe640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cd870 0x7fdd9c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 -- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cd870 msgr2=0x7fdd9c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cd870 0x7fdd9c15f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 -- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c0cf2b0 msgr2=0x7fdd9c1610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c0cf2b0 0x7fdd9c1610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 -- 172.21.15.43:0/2795849005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c15c770 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9affd640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c0cf2b0 0x7fdd9c1610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd9bfff640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c15ffb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c730 tx=0x7fdd9400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd83fff640 1 -- 172.21.15.43:0/2795849005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94003b80 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd83fff640 1 -- 172.21.15.43:0/2795849005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd94003d20 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c0cab40 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd83fff640 1 -- 172.21.15.43:0/2795849005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94005310 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c0cb100 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd9c068900 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.027+0000 7fdd83fff640 1 -- 172.21.15.43:0/2795849005 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdd940056c0 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.031+0000 7fdd83fff640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd88067fc0 0x7fdd8806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.031+0000 7fdd83fff640 1 -- 172.21.15.43:0/2795849005 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fdd94086940 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.031+0000 7fdd9b7fe640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd88067fc0 0x7fdd8806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.031+0000 7fdd9b7fe640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd88067fc0 0x7fdd8806a480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009790 tx=0x7fdd8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.035+0000 7fdd83fff640 1 -- 172.21.15.43:0/2795849005 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdd94056d80 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.207+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd9c0c3bd0 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.207+0000 7fdd83fff640 1 -- 172.21.15.43:0/2795849005 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fdd94056680 con 0x7fdd9c0cfe10 2024-03-30T13:12:05.216 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd88067fc0 msgr2=0x7fdd8806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd88067fc0 0x7fdd8806a480 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009790 tx=0x7fdd8c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 msgr2=0x7fdd9c15ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c15ffb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd9400c730 tx=0x7fdd9400cbf0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 shutdown_connections 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd88067fc0 0x7fdd8806a480 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd9c0cfe10 0x7fdd9c15ffb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd9c0cf2b0 0x7fdd9c1610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 --2- 172.21.15.43:0/2795849005 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd9c0cd870 0x7fdd9c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 >> 172.21.15.43:0/2795849005 conn(0x7fdd9c0b1e90 msgr2=0x7fdd9c072c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:05.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 shutdown_connections 2024-03-30T13:12:05.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.211+0000 7fdda1e14640 1 -- 172.21.15.43:0/2795849005 wait complete. 2024-03-30T13:12:05.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:05.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e04309640 1 Processor -- start 2024-03-30T13:12:05.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e04309640 1 -- start start 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e04309640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3dfc147e00 0x7f3dfc0b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e04309640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc06c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e04309640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dfc0b5470 con 0x7f3dfc06c5b0 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e04309640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3dfc0b5750 con 0x7f3dfc147e00 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e04309640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3dfc0b5a30 con 0x7f3dfc0b4f30 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e020a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3dfc147e00 0x7f3dfc0b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e028a6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3dfc06c5b0 0x7f3dfc06f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38994/0 (socket says 172.21.15.43:38994) 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e028a6640 1 -- 172.21.15.43:0/436485281 learned_addr learned my addr 172.21.15.43:0/436485281 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:05.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e018a4640 1 --2- 172.21.15.43:0/436485281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc06c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e020a5640 1 -- 172.21.15.43:0/436485281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 msgr2=0x7f3dfc06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e020a5640 1 --2- 172.21.15.43:0/436485281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc06c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e020a5640 1 -- 172.21.15.43:0/436485281 >> v1:172.21.15.43:6789/0 conn(0x7f3dfc06c5b0 legacy=0x7f3dfc06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e020a5640 1 -- 172.21.15.43:0/436485281 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc0b5d10 con 0x7f3dfc147e00 2024-03-30T13:12:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e010a3640 1 -- 172.21.15.43:0/436485281 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3085249406 0 0) 0x7f3dfc0b5470 con 0x7f3dfc06c5b0 2024-03-30T13:12:05.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.367+0000 7f3e018a4640 1 --2- 172.21.15.43:0/436485281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e020a5640 1 --2- 172.21.15.43:0/436485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc0b49f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009dc0 tx=0x7f3df801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a329d5fb929a41c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e010a3640 1 -- 172.21.15.43:0/436485281 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df80b2070 con 0x7f3dfc147e00 2024-03-30T13:12:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e010a3640 1 -- 172.21.15.43:0/436485281 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df80a5810 con 0x7f3dfc147e00 2024-03-30T13:12:05.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e010a3640 1 -- 172.21.15.43:0/436485281 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df80a5be0 con 0x7f3dfc147e00 2024-03-30T13:12:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- 172.21.15.43:0/436485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 msgr2=0x7f3dfc0b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 --2- 172.21.15.43:0/436485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc0b49f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009dc0 tx=0x7f3df801c950 comp rx=0 tx=0).stop 2024-03-30T13:12:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- 172.21.15.43:0/436485281 shutdown_connections 2024-03-30T13:12:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 --2- 172.21.15.43:0/436485281 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc06c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 --2- 172.21.15.43:0/436485281 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc0b49f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- 172.21.15.43:0/436485281 >> 172.21.15.43:0/436485281 conn(0x7f3dfc0d1070 msgr2=0x7f3dfc0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- 172.21.15.43:0/436485281 shutdown_connections 2024-03-30T13:12:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- 172.21.15.43:0/436485281 wait complete. 2024-03-30T13:12:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 Processor -- start 2024-03-30T13:12:05.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- start start 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc06c5b0 0x7f3dfc165060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc1655a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc16b330 con 0x7f3dfc147e00 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3dfc06ca00 con 0x7f3dfc06c5b0 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e04309640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3dfc16b4b0 con 0x7f3dfc0b4f30 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc145f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57398/0 (socket says 172.21.15.43:57398) 2024-03-30T13:12:05.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 -- 172.21.15.43:0/866445903 learned_addr learned my addr 172.21.15.43:0/866445903 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 -- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 msgr2=0x7f3dfc1655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc1655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 -- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc06c5b0 msgr2=0x7f3dfc165060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc06c5b0 0x7f3dfc165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.371+0000 7f3e028a6640 1 -- 172.21.15.43:0/866445903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dfc0af070 con 0x7f3dfc147e00 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3e028a6640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc145f00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c480 tx=0x7f3df400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3df2ffd640 1 -- 172.21.15.43:0/866445903 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4018070 con 0x7f3dfc147e00 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3df2ffd640 1 -- 172.21.15.43:0/866445903 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df40074e0 con 0x7f3dfc147e00 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dfc0b2500 con 0x7f3dfc147e00 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3df2ffd640 1 -- 172.21.15.43:0/866445903 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4004a30 con 0x7f3dfc147e00 2024-03-30T13:12:05.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dfc1630d0 con 0x7f3dfc147e00 2024-03-30T13:12:05.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dfc068900 con 0x7f3dfc147e00 2024-03-30T13:12:05.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3df2ffd640 1 -- 172.21.15.43:0/866445903 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3df401b070 con 0x7f3dfc147e00 2024-03-30T13:12:05.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3df2ffd640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3de0067ff0 0x7f3de006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3df2ffd640 1 -- 172.21.15.43:0/866445903 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f3df4085a80 con 0x7f3dfc147e00 2024-03-30T13:12:05.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3e020a5640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3de0067ff0 0x7f3de006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.375+0000 7f3e020a5640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3de0067ff0 0x7f3de006a4b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3df80af000 tx=0x7f3df8004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.379+0000 7f3df2ffd640 1 -- 172.21.15.43:0/866445903 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3dfc0af070 con 0x7f3dfc147e00 2024-03-30T13:12:05.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.535+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3dfc070a90 con 0x7f3dfc147e00 2024-03-30T13:12:05.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.535+0000 7f3df2ffd640 1 -- 172.21.15.43:0/866445903 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f3df4055f50 con 0x7f3dfc147e00 2024-03-30T13:12:05.543 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3de0067ff0 msgr2=0x7f3de006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3de0067ff0 0x7f3de006a4b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3df80af000 tx=0x7f3df8004ac0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 msgr2=0x7f3dfc145f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc145f00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c480 tx=0x7f3df400c940 comp rx=0 tx=0).stop 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 shutdown_connections 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3de0067ff0 0x7f3de006a4b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3dfc147e00 0x7f3dfc145f00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3dfc0b4f30 0x7f3dfc1655a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 --2- 172.21.15.43:0/866445903 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3dfc06c5b0 0x7f3dfc165060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 >> 172.21.15.43:0/866445903 conn(0x7f3dfc0d1070 msgr2=0x7f3dfc0b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 shutdown_connections 2024-03-30T13:12:05.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.539+0000 7f3e04309640 1 -- 172.21.15.43:0/866445903 wait complete. 2024-03-30T13:12:05.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:12:05.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ffdb5b640 1 Processor -- start 2024-03-30T13:12:05.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ffdb5b640 1 -- start start 2024-03-30T13:12:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ffdb5b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ff8161c50 0x7f3ff80ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ffdb5b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff80ce660 con 0x7f3ff80ccdc0 2024-03-30T13:12:05.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ffdb5b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff80b50f0 con 0x7f3ff80b7d40 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ffdb5b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff80b78b0 con 0x7f3ff8161c50 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ff8161c50 0x7f3ff80ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ff8161c50 0x7f3ff80ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57402/0 (socket says 172.21.15.43:57402) 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff77fe640 1 -- 172.21.15.43:0/1501835115 learned_addr learned my addr 172.21.15.43:0/1501835115 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff7fff640 1 --1- 172.21.15.43:0/1501835115 >> v1:172.21.15.43:6789/0 conn(0x7f3ff80ccdc0 0x7f3ff80cff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39010/0 (socket says 172.21.15.43:39010) 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff77fe640 1 -- 172.21.15.43:0/1501835115 >> v1:172.21.15.113:6789/0 conn(0x7f3ff80b7d40 legacy=0x7f3ff80cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff77fe640 1 -- 172.21.15.43:0/1501835115 >> v1:172.21.15.43:6789/0 conn(0x7f3ff80ccdc0 legacy=0x7f3ff80cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff77fe640 1 -- 172.21.15.43:0/1501835115 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff80b53d0 con 0x7f3ff8161c50 2024-03-30T13:12:05.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.679+0000 7f3ff67fc640 1 -- 172.21.15.43:0/1501835115 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1092768411 0 0) 0x7f3ff80ce660 con 0x7f3ff80ccdc0 2024-03-30T13:12:05.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 --2- 172.21.15.43:0/1501835115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8161c50 0x7f3ff80ce120 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009870 tx=0x7f3fe001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457a682e38b2f529 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff67fc640 1 -- 172.21.15.43:0/1501835115 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00b0070 con 0x7f3ff8161c50 2024-03-30T13:12:05.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff67fc640 1 -- 172.21.15.43:0/1501835115 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe00ab600 con 0x7f3ff8161c50 2024-03-30T13:12:05.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff67fc640 1 -- 172.21.15.43:0/1501835115 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00ab9d0 con 0x7f3ff8161c50 2024-03-30T13:12:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/1501835115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8161c50 msgr2=0x7f3ff80ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/1501835115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8161c50 0x7f3ff80ce120 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009870 tx=0x7f3fe001c440 comp rx=0 tx=0).stop 2024-03-30T13:12:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/1501835115 shutdown_connections 2024-03-30T13:12:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/1501835115 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff8161c50 0x7f3ff80ce120 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/1501835115 >> 172.21.15.43:0/1501835115 conn(0x7f3ff80ceae0 msgr2=0x7f3ff80ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/1501835115 shutdown_connections 2024-03-30T13:12:05.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/1501835115 wait complete. 2024-03-30T13:12:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 Processor -- start 2024-03-30T13:12:05.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- start start 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff80b7d40 0x7f3ff81551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ff80ccdc0 0x7f3ff8070270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ff8161c50 0x7f3ff8152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff816c950 con 0x7f3ff80b7d40 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff816c7d0 con 0x7f3ff80ccdc0 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ffdb5b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff816cad0 con 0x7f3ff8161c50 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff80b7d40 0x7f3ff81551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff80b7d40 0x7f3ff81551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57416/0 (socket says 172.21.15.43:57416) 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 -- 172.21.15.43:0/424738901 learned_addr learned my addr 172.21.15.43:0/424738901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:05.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff6ffd640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ff80ccdc0 0x7f3ff8070270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff7fff640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ff8161c50 0x7f3ff8152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 -- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ff8161c50 msgr2=0x7f3ff8152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ff8161c50 0x7f3ff8152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 -- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ff80ccdc0 msgr2=0x7f3ff8070270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ff80ccdc0 0x7f3ff8070270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.683+0000 7f3ff77fe640 1 -- 172.21.15.43:0/424738901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff80cb4e0 con 0x7f3ff80b7d40 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3ff77fe640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff80b7d40 0x7f3ff81551e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0019050 tx=0x7f3fe001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3feffff640 1 -- 172.21.15.43:0/424738901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00ba020 con 0x7f3ff80b7d40 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3feffff640 1 -- 172.21.15.43:0/424738901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe00aa7d0 con 0x7f3ff80b7d40 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3feffff640 1 -- 172.21.15.43:0/424738901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00b0040 con 0x7f3ff80b7d40 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff806f9c0 con 0x7f3ff80b7d40 2024-03-30T13:12:05.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff80cba30 con 0x7f3ff80b7d40 2024-03-30T13:12:05.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff8068900 con 0x7f3ff80b7d40 2024-03-30T13:12:05.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3feffff640 1 -- 172.21.15.43:0/424738901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3fe00bd020 con 0x7f3ff80b7d40 2024-03-30T13:12:05.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3feffff640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fd8067fc0 0x7f3fd806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:05.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3feffff640 1 -- 172.21.15.43:0/424738901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f3fe0120b60 con 0x7f3ff80b7d40 2024-03-30T13:12:05.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3ff6ffd640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fd8067fc0 0x7f3fd806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:05.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.687+0000 7f3ff6ffd640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fd8067fc0 0x7f3fd806a480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8160ad0 tx=0x7f3fe8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:05.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.691+0000 7f3feffff640 1 -- 172.21.15.43:0/424738901 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3fe00c4030 con 0x7f3ff80b7d40 2024-03-30T13:12:05.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.983+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3ff815a060 con 0x7f3ff80b7d40 2024-03-30T13:12:05.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.983+0000 7f3feffff640 1 -- 172.21.15.43:0/424738901 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f3fe0016150 con 0x7f3ff80b7d40 2024-03-30T13:12:05.989 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.983+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fd8067fc0 msgr2=0x7f3fd806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.983+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fd8067fc0 0x7f3fd806a480 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8160ad0 tx=0x7f3fe8009040 comp rx=0 tx=0).stop 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.983+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff80b7d40 msgr2=0x7f3ff81551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.983+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff80b7d40 0x7f3ff81551e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0019050 tx=0x7f3fe001ac80 comp rx=0 tx=0).stop 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 shutdown_connections 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fd8067fc0 0x7f3fd806a480 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3ff8161c50 0x7f3ff8152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3ff80ccdc0 0x7f3ff8070270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 --2- 172.21.15.43:0/424738901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ff80b7d40 0x7f3ff81551e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 >> 172.21.15.43:0/424738901 conn(0x7f3ff80ceae0 msgr2=0x7f3ff80b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:05.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 shutdown_connections 2024-03-30T13:12:05.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:05.987+0000 7f3ffdb5b640 1 -- 172.21.15.43:0/424738901 wait complete. 2024-03-30T13:12:06.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:06.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f1499b07640 1 Processor -- start 2024-03-30T13:12:06.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f1499b07640 1 -- start start 2024-03-30T13:12:06.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f1499b07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 0x7f1494169120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f1499b07640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149406bc40 con 0x7f149406b660 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f1499b07640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1494070600 con 0x7f1494147e80 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f1499b07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1494070c50 con 0x7f14941626d0 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f148bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f149406b660 0x7f1494161fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39022/0 (socket says 172.21.15.43:39022) 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.131+0000 7f148bfff640 1 -- 172.21.15.43:0/1153721725 learned_addr learned my addr 172.21.15.43:0/1153721725 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1493fff640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 0x7f1494169120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268920472 0 0) 0x7f149406bc40 con 0x7f149406b660 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1480003530 con 0x7f149406b660 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3126715209 0 0) 0x7f1480003530 con 0x7f149406b660 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 msgr2=0x7f1494169120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 0x7f1494169120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 >> v1:172.21.15.113:6789/0 conn(0x7f1494147e80 legacy=0x7f149406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14940708e0 con 0x7f149406b660 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f147c001680 con 0x7f149406b660 2024-03-30T13:12:06.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 >> v1:172.21.15.43:6789/0 conn(0x7f149406b660 legacy=0x7f1494161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800039f0 0x7f1480003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 0x7f1480004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480004d50 0x7f14800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1494070c50 con 0x7f14800039f0 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f148000bae0 con 0x7f1480004d50 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f148000bce0 con 0x7f14941626d0 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f14937fe640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800039f0 0x7f1480003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f148bfff640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 0x7f1480004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f14937fe640 1 -- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 msgr2=0x7f1480004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f14937fe640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14941626d0 0x7f1480004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f14937fe640 1 -- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480004d50 msgr2=0x7f14800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f14937fe640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1480004d50 0x7f14800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f14937fe640 1 -- 172.21.15.43:0/1153721725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1480003530 con 0x7f14800039f0 2024-03-30T13:12:06.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f14937fe640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800039f0 0x7f1480003e10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1484001750 tx=0x7f148401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806f2800819c20ee server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14840013a0 con 0x7f14800039f0 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14840052f0 con 0x7f14800039f0 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1492ffd640 1 -- 172.21.15.43:0/1153721725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484003840 con 0x7f14800039f0 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1499b07640 1 -- 172.21.15.43:0/1153721725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800039f0 msgr2=0x7f1480003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1499b07640 1 --2- 172.21.15.43:0/1153721725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14800039f0 0x7f1480003e10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f1484001750 tx=0x7f148401c3e0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1493fff640 1 -- 172.21.15.43:0/1153721725 reap_dead start 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1499b07640 1 -- 172.21.15.43:0/1153721725 shutdown_connections 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1499b07640 1 -- 172.21.15.43:0/1153721725 >> 172.21.15.43:0/1153721725 conn(0x7f149406e410 msgr2=0x7f149406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1499b07640 1 -- 172.21.15.43:0/1153721725 shutdown_connections 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.135+0000 7f1499b07640 1 -- 172.21.15.43:0/1153721725 wait complete. 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 Processor -- start 2024-03-30T13:12:06.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 -- start start 2024-03-30T13:12:06.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14800039f0 0x7f149415d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14940cd540 0x7f1494140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14940cd9c0 0x7f14941412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1494169cc0 con 0x7f14940cd540 2024-03-30T13:12:06.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1494169b40 con 0x7f14800039f0 2024-03-30T13:12:06.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1494169e40 con 0x7f14940cd9c0 2024-03-30T13:12:06.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14940cd540 0x7f1494140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14940cd540 0x7f1494140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57426/0 (socket says 172.21.15.43:57426) 2024-03-30T13:12:06.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 -- 172.21.15.43:0/2498726105 learned_addr learned my addr 172.21.15.43:0/2498726105 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:06.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1493fff640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14940cd9c0 0x7f14941412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f14937fe640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14800039f0 0x7f149415d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 -- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14940cd9c0 msgr2=0x7f14941412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14940cd9c0 0x7f14941412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 -- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14800039f0 msgr2=0x7f149415d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14800039f0 0x7f149415d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 -- 172.21.15.43:0/2498726105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14940adb30 con 0x7f14940cd540 2024-03-30T13:12:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1493fff640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14940cd9c0 0x7f14941412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f14937fe640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14800039f0 0x7f149415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:06.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f148bfff640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14940cd540 0x7f1494140da0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e780 tx=0x7f147c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1490ff9640 1 -- 172.21.15.43:0/2498726105 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00c6c0 con 0x7f14940cd540 2024-03-30T13:12:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1490ff9640 1 -- 172.21.15.43:0/2498726105 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f147c018070 con 0x7f14940cd540 2024-03-30T13:12:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14940ade10 con 0x7f14940cd540 2024-03-30T13:12:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1490ff9640 1 -- 172.21.15.43:0/2498726105 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00cac0 con 0x7f14940cd540 2024-03-30T13:12:06.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14940ae3d0 con 0x7f14940cd540 2024-03-30T13:12:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.139+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1494068890 con 0x7f14940cd540 2024-03-30T13:12:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.143+0000 7f1490ff9640 1 -- 172.21.15.43:0/2498726105 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f147c00cc60 con 0x7f14940cd540 2024-03-30T13:12:06.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.143+0000 7f1490ff9640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1474067fc0 0x7f147406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.143+0000 7f1490ff9640 1 -- 172.21.15.43:0/2498726105 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f147c087050 con 0x7f14940cd540 2024-03-30T13:12:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.143+0000 7f14937fe640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1474067fc0 0x7f147406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.143+0000 7f14937fe640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1474067fc0 0x7f147406a480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f14800057f0 tx=0x7f14840011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.147+0000 7f1490ff9640 1 -- 172.21.15.43:0/2498726105 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f147c057600 con 0x7f14940cd540 2024-03-30T13:12:06.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1494142000 con 0x7f14940cd540 2024-03-30T13:12:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1490ff9640 1 -- 172.21.15.43:0/2498726105 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f147c056f00 con 0x7f14940cd540 2024-03-30T13:12:06.322 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1474067fc0 msgr2=0x7f147406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1474067fc0 0x7f147406a480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f14800057f0 tx=0x7f14840011a0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14940cd540 msgr2=0x7f1494140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14940cd540 0x7f1494140da0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f147c00e780 tx=0x7f147c00eb80 comp rx=0 tx=0).stop 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 shutdown_connections 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1474067fc0 0x7f147406a480 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14940cd9c0 0x7f14941412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14940cd540 0x7f1494140da0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 --2- 172.21.15.43:0/2498726105 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14800039f0 0x7f149415d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 >> 172.21.15.43:0/2498726105 conn(0x7f149406e410 msgr2=0x7f1494142520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:06.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 shutdown_connections 2024-03-30T13:12:06.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.315+0000 7f1499b07640 1 -- 172.21.15.43:0/2498726105 wait complete. 2024-03-30T13:12:06.339 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:12:06.339 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:06.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.515+0000 7f3d3821a640 1 Processor -- start 2024-03-30T13:12:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.515+0000 7f3d3821a640 1 -- start start 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d3821a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3d30144270 0x7f3d30144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d3821a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d300b2ef0 0x7f3d3015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d3821a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d300b33f0 con 0x7f3d300b4490 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d3821a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3d30144fa0 con 0x7f3d300b2ef0 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d3821a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3d3015fce0 con 0x7f3d30144270 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d300b2ef0 0x7f3d3015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3d300b2ef0 0x7f3d3015f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57434/0 (socket says 172.21.15.43:57434) 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 -- 172.21.15.43:0/3034340655 learned_addr learned my addr 172.21.15.43:0/3034340655 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d35fb6640 1 --2- 172.21.15.43:0/3034340655 >> v2:172.21.15.113:3300/0 conn(0x7f3d30144270 0x7f3d30144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 -- 172.21.15.43:0/3034340655 >> v2:172.21.15.113:3300/0 conn(0x7f3d30144270 msgr2=0x7f3d30144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:06.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 --2- 172.21.15.43:0/3034340655 >> v2:172.21.15.113:3300/0 conn(0x7f3d30144270 0x7f3d30144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 -- 172.21.15.43:0/3034340655 >> v1:172.21.15.113:6789/0 conn(0x7f3d300b4490 legacy=0x7f3d3006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:06.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 -- 172.21.15.43:0/3034340655 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d300b36d0 con 0x7f3d300b2ef0 2024-03-30T13:12:06.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d357b5640 1 --2- 172.21.15.43:0/3034340655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b2ef0 0x7f3d3015f7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3d30072000 tx=0x7f3d2001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1001646cd6cb2094 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d34fb4640 1 -- 172.21.15.43:0/3034340655 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200b4070 con 0x7f3d300b2ef0 2024-03-30T13:12:06.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d34fb4640 1 -- 172.21.15.43:0/3034340655 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d200a7c50 con 0x7f3d300b2ef0 2024-03-30T13:12:06.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.519+0000 7f3d34fb4640 1 -- 172.21.15.43:0/3034340655 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200af440 con 0x7f3d300b2ef0 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- 172.21.15.43:0/3034340655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b2ef0 msgr2=0x7f3d3015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3034340655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b2ef0 0x7f3d3015f7a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3d30072000 tx=0x7f3d2001eb80 comp rx=0 tx=0).stop 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- 172.21.15.43:0/3034340655 shutdown_connections 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3034340655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b2ef0 0x7f3d3015f7a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3034340655 >> v2:172.21.15.113:3300/0 conn(0x7f3d30144270 0x7f3d30144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- 172.21.15.43:0/3034340655 >> 172.21.15.43:0/3034340655 conn(0x7f3d300cfee0 msgr2=0x7f3d300d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- 172.21.15.43:0/3034340655 shutdown_connections 2024-03-30T13:12:06.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- 172.21.15.43:0/3034340655 wait complete. 2024-03-30T13:12:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 Processor -- start 2024-03-30T13:12:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- start start 2024-03-30T13:12:06.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d300b2ef0 0x7f3d3015aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b4490 0x7f3d3015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d30144270 0x7f3d3015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3d300cdd10 con 0x7f3d300b4490 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3006cf90 con 0x7f3d30144270 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3d300cde90 con 0x7f3d300b2ef0 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b4490 0x7f3d3015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b4490 0x7f3d3015d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57442/0 (socket says 172.21.15.43:57442) 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 -- 172.21.15.43:0/3730460914 learned_addr learned my addr 172.21.15.43:0/3730460914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 -- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d300b2ef0 msgr2=0x7f3d3015aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d35fb6640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d300b2ef0 0x7f3d3015aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d367b7640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d30144270 0x7f3d3015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d300b2ef0 0x7f3d3015aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 -- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d30144270 msgr2=0x7f3d3015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d30144270 0x7f3d3015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 -- 172.21.15.43:0/3730460914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3013dab0 con 0x7f3d300b4490 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d357b5640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b4490 0x7f3d3015d9e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3d2001c7b0 tx=0x7f3d2001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d267fc640 1 -- 172.21.15.43:0/3730460914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200b4030 con 0x7f3d300b4490 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d267fc640 1 -- 172.21.15.43:0/3730460914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d200b6070 con 0x7f3d300b4490 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3013dd90 con 0x7f3d300b4490 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d267fc640 1 -- 172.21.15.43:0/3730460914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d200ad550 con 0x7f3d300b4490 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.523+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3013e350 con 0x7f3d300b4490 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.527+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d30068890 con 0x7f3d300b4490 2024-03-30T13:12:06.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.527+0000 7f3d267fc640 1 -- 172.21.15.43:0/3730460914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3d200c1060 con 0x7f3d300b4490 2024-03-30T13:12:06.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.527+0000 7f3d267fc640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3d14067fc0 0x7f3d1406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.527+0000 7f3d267fc640 1 -- 172.21.15.43:0/3730460914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f3d20120b60 con 0x7f3d300b4490 2024-03-30T13:12:06.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.527+0000 7f3d35fb6640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3d14067fc0 0x7f3d1406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.527+0000 7f3d35fb6640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3d14067fc0 0x7f3d1406a480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009790 tx=0x7f3d2c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.535+0000 7f3d267fc640 1 -- 172.21.15.43:0/3730460914 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3d200187a0 con 0x7f3d300b4490 2024-03-30T13:12:06.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.695+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d3015f210 con 0x7f3d300b4490 2024-03-30T13:12:06.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.695+0000 7f3d267fc640 1 -- 172.21.15.43:0/3730460914 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7f3d200f0f10 con 0x7f3d300b4490 2024-03-30T13:12:06.703 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:06.703 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:06.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3d14067fc0 msgr2=0x7f3d1406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3d14067fc0 0x7f3d1406a480 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009790 tx=0x7f3d2c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:06.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b4490 msgr2=0x7f3d3015d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b4490 0x7f3d3015d9e0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3d2001c7b0 tx=0x7f3d2001b920 comp rx=0 tx=0).stop 2024-03-30T13:12:06.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 shutdown_connections 2024-03-30T13:12:06.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3d14067fc0 0x7f3d1406a480 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3d30144270 0x7f3d3015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3d300b4490 0x7f3d3015d9e0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 --2- 172.21.15.43:0/3730460914 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3d300b2ef0 0x7f3d3015aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 >> 172.21.15.43:0/3730460914 conn(0x7f3d300cfee0 msgr2=0x7f3d3006d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 shutdown_connections 2024-03-30T13:12:06.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.699+0000 7f3d3821a640 1 -- 172.21.15.43:0/3730460914 wait complete. 2024-03-30T13:12:06.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:12:06.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a82f22640 1 Processor -- start 2024-03-30T13:12:06.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a82f22640 1 -- start start 2024-03-30T13:12:06.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a82f22640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a82f22640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a7c0700f0 0x7f5a7c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a82f22640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a7c0b4ba0 con 0x7f5a7c0b4520 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a82f22640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5a7c070c30 con 0x7f5a7c0700f0 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a82f22640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7c0b4960 con 0x7f5a7c15afc0 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a814bf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a7c0700f0 0x7f5a7c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a7bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5a7c0b4520 0x7f5a7c06f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39036/0 (socket says 172.21.15.43:39036) 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a7bfff640 1 -- 172.21.15.43:0/1149203692 learned_addr learned my addr 172.21.15.43:0/1149203692 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a80cbe640 1 --2- 172.21.15.43:0/1149203692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a814bf640 1 -- 172.21.15.43:0/1149203692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 msgr2=0x7f5a7c0b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a814bf640 1 --2- 172.21.15.43:0/1149203692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c0b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a814bf640 1 -- 172.21.15.43:0/1149203692 >> v1:172.21.15.43:6789/0 conn(0x7f5a7c0b4520 legacy=0x7f5a7c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a814bf640 1 -- 172.21.15.43:0/1149203692 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c15ec60 con 0x7f5a7c0700f0 2024-03-30T13:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.847+0000 7f5a7b7fe640 1 -- 172.21.15.43:0/1149203692 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1126216992 0 0) 0x7f5a7c0b4ba0 con 0x7f5a7c0b4520 2024-03-30T13:12:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a814bf640 1 --2- 172.21.15.43:0/1149203692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c0ce7b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5a740097c0 tx=0x7f5a7401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f0526247a1f82b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a7b7fe640 1 -- 172.21.15.43:0/1149203692 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740a7070 con 0x7f5a7c0700f0 2024-03-30T13:12:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a7b7fe640 1 -- 172.21.15.43:0/1149203692 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a740b0070 con 0x7f5a7c0700f0 2024-03-30T13:12:06.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a7b7fe640 1 -- 172.21.15.43:0/1149203692 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a740ab890 con 0x7f5a7c0700f0 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 -- 172.21.15.43:0/1149203692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0700f0 msgr2=0x7f5a7c0ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 --2- 172.21.15.43:0/1149203692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c0ce7b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5a740097c0 tx=0x7f5a7401c470 comp rx=0 tx=0).stop 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 -- 172.21.15.43:0/1149203692 shutdown_connections 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 --2- 172.21.15.43:0/1149203692 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c0ce7b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 --2- 172.21.15.43:0/1149203692 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c0b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 -- 172.21.15.43:0/1149203692 >> 172.21.15.43:0/1149203692 conn(0x7f5a7c0d1070 msgr2=0x7f5a7c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 -- 172.21.15.43:0/1149203692 shutdown_connections 2024-03-30T13:12:06.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 -- 172.21.15.43:0/1149203692 wait complete. 2024-03-30T13:12:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 Processor -- start 2024-03-30T13:12:06.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.851+0000 7f5a82f22640 1 -- start start 2024-03-30T13:12:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c072780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0b4520 0x7f5a7c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c15b3a0 con 0x7f5a7c0b4520 2024-03-30T13:12:06.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a7c15b520 con 0x7f5a7c0700f0 2024-03-30T13:12:06.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5a7c0cf500 con 0x7f5a7c15afc0 2024-03-30T13:12:06.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0b4520 0x7f5a7c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0b4520 0x7f5a7c06b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57462/0 (socket says 172.21.15.43:57462) 2024-03-30T13:12:06.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 -- 172.21.15.43:0/297453416 learned_addr learned my addr 172.21.15.43:0/297453416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:06.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a80cbe640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c072780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a814bf640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 -- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 msgr2=0x7f5a7c14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 -- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7c0700f0 msgr2=0x7f5a7c072780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:06.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c072780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:06.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 -- 172.21.15.43:0/297453416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7c06c0e0 con 0x7f5a7c0b4520 2024-03-30T13:12:06.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a80cbe640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:06.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a814bf640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:06.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a7bfff640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0b4520 0x7f5a7c06b660 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5a700011f0 tx=0x7f5a7000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a797fa640 1 -- 172.21.15.43:0/297453416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70018070 con 0x7f5a7c0b4520 2024-03-30T13:12:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a797fa640 1 -- 172.21.15.43:0/297453416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a700074e0 con 0x7f5a7c0b4520 2024-03-30T13:12:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7c06c3c0 con 0x7f5a7c0b4520 2024-03-30T13:12:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a797fa640 1 -- 172.21.15.43:0/297453416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a70004a10 con 0x7f5a7c0b4520 2024-03-30T13:12:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a7c06c980 con 0x7f5a7c0b4520 2024-03-30T13:12:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a7c068900 con 0x7f5a7c0b4520 2024-03-30T13:12:06.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a797fa640 1 -- 172.21.15.43:0/297453416 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5a7001b070 con 0x7f5a7c0b4520 2024-03-30T13:12:06.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a797fa640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a68067fc0 0x7f5a6806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:06.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a797fa640 1 -- 172.21.15.43:0/297453416 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f5a70085b90 con 0x7f5a7c0b4520 2024-03-30T13:12:06.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.855+0000 7f5a80cbe640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a68067fc0 0x7f5a6806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:06.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.859+0000 7f5a80cbe640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a68067fc0 0x7f5a6806a480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009f50 tx=0x7f5a6c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:06.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:06.863+0000 7f5a797fa640 1 -- 172.21.15.43:0/297453416 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5a70056060 con 0x7f5a7c0b4520 2024-03-30T13:12:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.059+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5a7c0706c0 con 0x7f5a7c0b4520 2024-03-30T13:12:07.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.059+0000 7f5a797fa640 1 -- 172.21.15.43:0/297453416 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f5a70055960 con 0x7f5a7c0b4520 2024-03-30T13:12:07.068 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a68067fc0 msgr2=0x7f5a6806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a68067fc0 0x7f5a6806a480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009f50 tx=0x7f5a6c00b040 comp rx=0 tx=0).stop 2024-03-30T13:12:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0b4520 msgr2=0x7f5a7c06b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0b4520 0x7f5a7c06b660 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5a700011f0 tx=0x7f5a7000c9b0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 shutdown_connections 2024-03-30T13:12:07.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a68067fc0 0x7f5a6806a480 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a7c15afc0 0x7f5a7c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a7c0b4520 0x7f5a7c06b660 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 --2- 172.21.15.43:0/297453416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a7c0700f0 0x7f5a7c072780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 >> 172.21.15.43:0/297453416 conn(0x7f5a7c0d1070 msgr2=0x7f5a7c06f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 shutdown_connections 2024-03-30T13:12:07.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.063+0000 7f5a82f22640 1 -- 172.21.15.43:0/297453416 wait complete. 2024-03-30T13:12:07.086 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:07.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.195+0000 7fd9fcdcd640 1 Processor -- start 2024-03-30T13:12:07.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.195+0000 7fd9fcdcd640 1 -- start start 2024-03-30T13:12:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9fcdcd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9f815f880 0x7fd9f80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9fcdcd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f806f090 0x7fd9f80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9fcdcd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f80ce4d0 con 0x7fd9f8070640 2024-03-30T13:12:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9fcdcd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f8070ad0 con 0x7fd9f815f880 2024-03-30T13:12:07.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9fcdcd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f80cc7f0 con 0x7fd9f806f090 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9f815f880 0x7fd9f80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd9f815f880 0x7fd9f80b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57478/0 (socket says 172.21.15.43:57478) 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f659c640 1 -- 172.21.15.43:0/307354249 learned_addr learned my addr 172.21.15.43:0/307354249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f6d9d640 1 --2- 172.21.15.43:0/307354249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f806f090 0x7fd9f80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f659c640 1 -- 172.21.15.43:0/307354249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f806f090 msgr2=0x7fd9f80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f659c640 1 --2- 172.21.15.43:0/307354249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f806f090 0x7fd9f80ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f659c640 1 -- 172.21.15.43:0/307354249 >> v1:172.21.15.113:6789/0 conn(0x7fd9f8070640 legacy=0x7fd9f806e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.199+0000 7fd9f659c640 1 -- 172.21.15.43:0/307354249 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f80ccad0 con 0x7fd9f815f880 2024-03-30T13:12:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f659c640 1 --2- 172.21.15.43:0/307354249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80b4ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009e50 tx=0x7fd9f001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a3723d02b70737 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f559a640 1 -- 172.21.15.43:0/307354249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00a56e0 con 0x7fd9f815f880 2024-03-30T13:12:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f559a640 1 -- 172.21.15.43:0/307354249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f00a5880 con 0x7fd9f815f880 2024-03-30T13:12:07.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f559a640 1 -- 172.21.15.43:0/307354249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00a5c50 con 0x7fd9f815f880 2024-03-30T13:12:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/307354249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 msgr2=0x7fd9f80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/307354249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80b4ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009e50 tx=0x7fd9f001c910 comp rx=0 tx=0).stop 2024-03-30T13:12:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/307354249 shutdown_connections 2024-03-30T13:12:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/307354249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f806f090 0x7fd9f80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/307354249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80b4ff0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/307354249 >> 172.21.15.43:0/307354249 conn(0x7fd9f80b3cf0 msgr2=0x7fd9f80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/307354249 shutdown_connections 2024-03-30T13:12:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/307354249 wait complete. 2024-03-30T13:12:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 Processor -- start 2024-03-30T13:12:07.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- start start 2024-03-30T13:12:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f806f090 0x7fd9f80c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f8070640 0x7fd9f80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f81609d0 con 0x7fd9f815f880 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f8160850 con 0x7fd9f806f090 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9fcdcd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f81691e0 con 0x7fd9f8070640 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57480/0 (socket says 172.21.15.43:57480) 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 -- 172.21.15.43:0/2621321392 learned_addr learned my addr 172.21.15.43:0/2621321392 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f659c640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f806f090 0x7fd9f80c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f5d9b640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f8070640 0x7fd9f80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 -- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f8070640 msgr2=0x7fd9f80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f8070640 0x7fd9f80cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 -- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f806f090 msgr2=0x7fd9f80c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f806f090 0x7fd9f80c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 -- 172.21.15.43:0/2621321392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f80c9270 con 0x7fd9f815f880 2024-03-30T13:12:07.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.203+0000 7fd9f6d9d640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80c8320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800d530 tx=0x7fd9e800d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9e6ffd640 1 -- 172.21.15.43:0/2621321392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e80125f0 con 0x7fd9f815f880 2024-03-30T13:12:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9e6ffd640 1 -- 172.21.15.43:0/2621321392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e8019070 con 0x7fd9f815f880 2024-03-30T13:12:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f80c9550 con 0x7fd9f815f880 2024-03-30T13:12:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9e6ffd640 1 -- 172.21.15.43:0/2621321392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e80129d0 con 0x7fd9f815f880 2024-03-30T13:12:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f80c9b10 con 0x7fd9f815f880 2024-03-30T13:12:07.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f8068900 con 0x7fd9f815f880 2024-03-30T13:12:07.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9e6ffd640 1 -- 172.21.15.43:0/2621321392 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd9e8012b70 con 0x7fd9f815f880 2024-03-30T13:12:07.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9e6ffd640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9d8067fc0 0x7fd9d806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9e6ffd640 1 -- 172.21.15.43:0/2621321392 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fd9e8086b20 con 0x7fd9f815f880 2024-03-30T13:12:07.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9f659c640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9d8067fc0 0x7fd9d806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.207+0000 7fd9f659c640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9d8067fc0 0x7fd9d806a480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00ae000 tx=0x7fd9f00a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.211+0000 7fd9e6ffd640 1 -- 172.21.15.43:0/2621321392 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd9e8056f60 con 0x7fd9f815f880 2024-03-30T13:12:07.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd9f806feb0 con 0x7fd9f815f880 2024-03-30T13:12:07.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9e6ffd640 1 -- 172.21.15.43:0/2621321392 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fd9e8056860 con 0x7fd9f815f880 2024-03-30T13:12:07.374 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9d8067fc0 msgr2=0x7fd9d806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9d8067fc0 0x7fd9d806a480 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00ae000 tx=0x7fd9f00a7070 comp rx=0 tx=0).stop 2024-03-30T13:12:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 msgr2=0x7fd9f80c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80c8320 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800d530 tx=0x7fd9e800d9f0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 shutdown_connections 2024-03-30T13:12:07.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd9d8067fc0 0x7fd9d806a480 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9f815f880 0x7fd9f80c8320 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd9f8070640 0x7fd9f80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 --2- 172.21.15.43:0/2621321392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9f806f090 0x7fd9f80c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 >> 172.21.15.43:0/2621321392 conn(0x7fd9f80b3cf0 msgr2=0x7fd9f815ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 shutdown_connections 2024-03-30T13:12:07.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.367+0000 7fd9fcdcd640 1 -- 172.21.15.43:0/2621321392 wait complete. 2024-03-30T13:12:07.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:07.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8cab3dc640 1 Processor -- start 2024-03-30T13:12:07.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8cab3dc640 1 -- start start 2024-03-30T13:12:07.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8cab3dc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8ca415f880 0x7f8ca40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8cab3dc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406f090 0x7f8ca40ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8cab3dc640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca40ce4d0 con 0x7f8ca4070640 2024-03-30T13:12:07.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8cab3dc640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca4070ad0 con 0x7f8ca415f880 2024-03-30T13:12:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8cab3dc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca40cc7f0 con 0x7f8ca406f090 2024-03-30T13:12:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9979640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406f090 0x7f8ca40ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9979640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406f090 0x7f8ca40ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:48214/0 (socket says 172.21.15.43:48214) 2024-03-30T13:12:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9979640 1 -- 172.21.15.43:0/3039402056 learned_addr learned my addr 172.21.15.43:0/3039402056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9178640 1 --2- 172.21.15.43:0/3039402056 >> v2:172.21.15.113:3300/0 conn(0x7f8ca415f880 0x7f8ca40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9979640 1 -- 172.21.15.43:0/3039402056 >> v2:172.21.15.113:3300/0 conn(0x7f8ca415f880 msgr2=0x7f8ca40b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9979640 1 --2- 172.21.15.43:0/3039402056 >> v2:172.21.15.113:3300/0 conn(0x7f8ca415f880 0x7f8ca40b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9979640 1 -- 172.21.15.43:0/3039402056 >> v1:172.21.15.113:6789/0 conn(0x7f8ca4070640 legacy=0x7f8ca406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.499+0000 7f8ca9979640 1 -- 172.21.15.43:0/3039402056 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca40ccad0 con 0x7f8ca406f090 2024-03-30T13:12:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8ca9979640 1 --2- 172.21.15.43:0/3039402056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406f090 0x7f8ca40ce9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009e50 tx=0x7f8c9c01c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9e533c1a5c05a04 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8c9bfff640 1 -- 172.21.15.43:0/3039402056 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0a56e0 con 0x7f8ca406f090 2024-03-30T13:12:07.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8c9bfff640 1 -- 172.21.15.43:0/3039402056 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c0a5880 con 0x7f8ca406f090 2024-03-30T13:12:07.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8c9bfff640 1 -- 172.21.15.43:0/3039402056 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0a5c50 con 0x7f8ca406f090 2024-03-30T13:12:07.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 -- 172.21.15.43:0/3039402056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406f090 msgr2=0x7f8ca40ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/3039402056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406f090 0x7f8ca40ce9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009e50 tx=0x7f8c9c01c910 comp rx=0 tx=0).stop 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 -- 172.21.15.43:0/3039402056 shutdown_connections 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/3039402056 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca406f090 0x7f8ca40ce9b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/3039402056 >> v2:172.21.15.113:3300/0 conn(0x7f8ca415f880 0x7f8ca40b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 -- 172.21.15.43:0/3039402056 >> 172.21.15.43:0/3039402056 conn(0x7f8ca40b3cf0 msgr2=0x7f8ca40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 -- 172.21.15.43:0/3039402056 shutdown_connections 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 -- 172.21.15.43:0/3039402056 wait complete. 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 Processor -- start 2024-03-30T13:12:07.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.503+0000 7f8cab3dc640 1 -- start start 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca406f090 0x7f8ca4153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca4070640 0x7f8ca4157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca415f880 0x7f8ca406e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca41609d0 con 0x7f8ca415f880 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca4160850 con 0x7f8ca406f090 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca41691e0 con 0x7f8ca4070640 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca415f880 0x7f8ca406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca415f880 0x7f8ca406e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57494/0 (socket says 172.21.15.43:57494) 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 -- 172.21.15.43:0/2776515496 learned_addr learned my addr 172.21.15.43:0/2776515496 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca8977640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca4070640 0x7f8ca4157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9178640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca406f090 0x7f8ca4153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 -- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca4070640 msgr2=0x7f8ca4157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca4070640 0x7f8ca4157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 -- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca406f090 msgr2=0x7f8ca4153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca406f090 0x7f8ca4153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 -- 172.21.15.43:0/2776515496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca4157510 con 0x7f8ca415f880 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8ca9979640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca415f880 0x7f8ca406e170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c01a540 tx=0x7f8c9c0196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8c99ffb640 1 -- 172.21.15.43:0/2776515496 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0bc020 con 0x7f8ca415f880 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8c99ffb640 1 -- 172.21.15.43:0/2776515496 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c0b8670 con 0x7f8ca415f880 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca406e9c0 con 0x7f8ca415f880 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8c99ffb640 1 -- 172.21.15.43:0/2776515496 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c0b8a60 con 0x7f8ca415f880 2024-03-30T13:12:07.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca406e6b0 con 0x7f8ca415f880 2024-03-30T13:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca4068900 con 0x7f8ca415f880 2024-03-30T13:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.507+0000 7f8c99ffb640 1 -- 172.21.15.43:0/2776515496 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8c9c0bf020 con 0x7f8ca415f880 2024-03-30T13:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.511+0000 7f8c99ffb640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.511+0000 7f8c99ffb640 1 -- 172.21.15.43:0/2776515496 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f8c9c121550 con 0x7f8ca415f880 2024-03-30T13:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.511+0000 7f8ca9178640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.511+0000 7f8ca9178640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ca406d9a0 tx=0x7f8c94009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.515+0000 7f8c99ffb640 1 -- 172.21.15.43:0/2776515496 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8c9c0c6030 con 0x7f8ca415f880 2024-03-30T13:12:07.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.675+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8ca40d1790 con 0x7f8ca415f880 2024-03-30T13:12:07.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.675+0000 7f8c99ffb640 1 -- 172.21.15.43:0/2776515496 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f8c9c016620 con 0x7f8ca415f880 2024-03-30T13:12:07.683 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 msgr2=0x7f8c8806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8ca406d9a0 tx=0x7f8c94009190 comp rx=0 tx=0).stop 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca415f880 msgr2=0x7f8ca406e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca415f880 0x7f8ca406e170 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c01a540 tx=0x7f8c9c0196b0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 shutdown_connections 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8c88067fc0 0x7f8c8806a480 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ca415f880 0x7f8ca406e170 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8ca4070640 0x7f8ca4157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 --2- 172.21.15.43:0/2776515496 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8ca406f090 0x7f8ca4153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 >> 172.21.15.43:0/2776515496 conn(0x7f8ca40b3cf0 msgr2=0x7f8ca40b6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 shutdown_connections 2024-03-30T13:12:07.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.679+0000 7f8cab3dc640 1 -- 172.21.15.43:0/2776515496 wait complete. 2024-03-30T13:12:07.703 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:12:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f2234b9f640 1 Processor -- start 2024-03-30T13:12:07.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f2234b9f640 1 -- start start 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f2234b9f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22300afb90 0x7f223006eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f2234b9f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f223006dc00 con 0x7f223015bd70 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f2234b9f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f223006dee0 con 0x7f223006f610 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f2234b9f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f223006e1c0 con 0x7f22300afb90 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22300afb90 0x7f223006eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f223015bd70 0x7f223015c150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39044/0 (socket says 172.21.15.43:39044) 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222e59c640 1 -- 172.21.15.43:0/3990512576 learned_addr learned my addr 172.21.15.43:0/3990512576 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222d59a640 1 -- 172.21.15.43:0/3990512576 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 250820310 0 0) 0x7f223006dc00 con 0x7f223015bd70 2024-03-30T13:12:07.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222d59a640 1 -- 172.21.15.43:0/3990512576 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2218003530 con 0x7f223015bd70 2024-03-30T13:12:07.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222dd9b640 1 -- 172.21.15.43:0/3990512576 >> v1:172.21.15.113:6789/0 conn(0x7f223006f610 legacy=0x7f223006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:07.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222dd9b640 1 -- 172.21.15.43:0/3990512576 >> v1:172.21.15.43:6789/0 conn(0x7f223015bd70 legacy=0x7f223015c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.823+0000 7f222dd9b640 1 -- 172.21.15.43:0/3990512576 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223006e4a0 con 0x7f22300afb90 2024-03-30T13:12:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 --2- 172.21.15.43:0/3990512576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223006eca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f22240097f0 tx=0x7f222401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b6a144b151a651 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222d59a640 1 -- 172.21.15.43:0/3990512576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22240b1070 con 0x7f22300afb90 2024-03-30T13:12:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222d59a640 1 -- 172.21.15.43:0/3990512576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22240ac410 con 0x7f22300afb90 2024-03-30T13:12:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222d59a640 1 -- 172.21.15.43:0/3990512576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22240ac7e0 con 0x7f22300afb90 2024-03-30T13:12:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- 172.21.15.43:0/3990512576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 msgr2=0x7f223006eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3990512576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223006eca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f22240097f0 tx=0x7f222401c4a0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- 172.21.15.43:0/3990512576 shutdown_connections 2024-03-30T13:12:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3990512576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223006eca0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- 172.21.15.43:0/3990512576 >> 172.21.15.43:0/3990512576 conn(0x7f22300d0c90 msgr2=0x7f22300d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- 172.21.15.43:0/3990512576 shutdown_connections 2024-03-30T13:12:07.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- 172.21.15.43:0/3990512576 wait complete. 2024-03-30T13:12:07.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 Processor -- start 2024-03-30T13:12:07.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- start start 2024-03-30T13:12:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223006f610 0x7f22300aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223014e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223015bd70 0x7f223014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22300b43c0 con 0x7f22300afb90 2024-03-30T13:12:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22300b4240 con 0x7f223006f610 2024-03-30T13:12:07.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f2234b9f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f22300b4540 con 0x7f223015bd70 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223014e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223014e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57520/0 (socket says 172.21.15.43:57520) 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 -- 172.21.15.43:0/3483490926 learned_addr learned my addr 172.21.15.43:0/3483490926 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222e59c640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223006f610 0x7f22300aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 -- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223015bd70 msgr2=0x7f223014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223015bd70 0x7f223014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 -- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223006f610 msgr2=0x7f22300aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223006f610 0x7f22300aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.827+0000 7f222dd9b640 1 -- 172.21.15.43:0/3483490926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223013d260 con 0x7f22300afb90 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f222dd9b640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223014e5f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f22240190b0 tx=0x7f2224019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f221effd640 1 -- 172.21.15.43:0/3483490926 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22240b9020 con 0x7f22300afb90 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f221effd640 1 -- 172.21.15.43:0/3483490926 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22240b5410 con 0x7f22300afb90 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223013d540 con 0x7f22300afb90 2024-03-30T13:12:07.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f221effd640 1 -- 172.21.15.43:0/3483490926 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22240b1040 con 0x7f22300afb90 2024-03-30T13:12:07.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223013db00 con 0x7f22300afb90 2024-03-30T13:12:07.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2230068900 con 0x7f22300afb90 2024-03-30T13:12:07.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f221effd640 1 -- 172.21.15.43:0/3483490926 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f22240bc020 con 0x7f22300afb90 2024-03-30T13:12:07.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f221effd640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2210067ff0 0x7f221006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:07.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f221effd640 1 -- 172.21.15.43:0/3483490926 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f2224120620 con 0x7f22300afb90 2024-03-30T13:12:07.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f222e59c640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2210067ff0 0x7f221006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:07.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.831+0000 7f222e59c640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2210067ff0 0x7f221006a4b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22280011f0 tx=0x7f2228009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:07.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:07.835+0000 7f221effd640 1 -- 172.21.15.43:0/3483490926 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f22240c3030 con 0x7f22300afb90 2024-03-30T13:12:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.003+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f223006fe70 con 0x7f22300afb90 2024-03-30T13:12:08.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.003+0000 7f221effd640 1 -- 172.21.15.43:0/3483490926 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f2224016090 con 0x7f22300afb90 2024-03-30T13:12:08.012 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2210067ff0 msgr2=0x7f221006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2210067ff0 0x7f221006a4b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f22280011f0 tx=0x7f2228009040 comp rx=0 tx=0).stop 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 msgr2=0x7f223014e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223014e5f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f22240190b0 tx=0x7f2224019240 comp rx=0 tx=0).stop 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 shutdown_connections 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2210067ff0 0x7f221006a4b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f223015bd70 0x7f223014eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22300afb90 0x7f223014e5f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 --2- 172.21.15.43:0/3483490926 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f223006f610 0x7f22300aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 >> 172.21.15.43:0/3483490926 conn(0x7f22300d0c90 msgr2=0x7f223006bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 shutdown_connections 2024-03-30T13:12:08.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.007+0000 7f2234b9f640 1 -- 172.21.15.43:0/3483490926 wait complete. 2024-03-30T13:12:08.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7670a5b640 1 Processor -- start 2024-03-30T13:12:08.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7670a5b640 1 -- start start 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7670a5b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f766c15c310 0x7f766c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7670a5b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c0d1aa0 0x7f766c1607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7670a5b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f766c15c6f0 con 0x7f766c070ed0 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7670a5b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f766c0ce990 con 0x7f766c15c310 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7670a5b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f766c0ce6b0 con 0x7f766c0d1aa0 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7669d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c0d1aa0 0x7f766c1607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7669d9b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c0d1aa0 0x7f766c1607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:48248/0 (socket says 172.21.15.43:48248) 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f766a59c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f766c15c310 0x7f766c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.151+0000 7f7669d9b640 1 -- 172.21.15.43:0/4213583757 learned_addr learned my addr 172.21.15.43:0/4213583757 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7669d9b640 1 -- 172.21.15.43:0/4213583757 >> v2:172.21.15.113:3300/0 conn(0x7f766c15c310 msgr2=0x7f766c0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7669d9b640 1 --2- 172.21.15.43:0/4213583757 >> v2:172.21.15.113:3300/0 conn(0x7f766c15c310 0x7f766c0b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7669d9b640 1 -- 172.21.15.43:0/4213583757 >> v1:172.21.15.113:6789/0 conn(0x7f766c070ed0 legacy=0x7f766c0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7669d9b640 1 -- 172.21.15.43:0/4213583757 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766c0cc890 con 0x7f766c0d1aa0 2024-03-30T13:12:08.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7669d9b640 1 --2- 172.21.15.43:0/4213583757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c0d1aa0 0x7f766c1607f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f765c009870 tx=0x7f765c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b5e5446ae650f40 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f766959a640 1 -- 172.21.15.43:0/4213583757 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c0b0070 con 0x7f766c0d1aa0 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f766959a640 1 -- 172.21.15.43:0/4213583757 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765c0ab600 con 0x7f766c0d1aa0 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f766959a640 1 -- 172.21.15.43:0/4213583757 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c0ab9d0 con 0x7f766c0d1aa0 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 -- 172.21.15.43:0/4213583757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c0d1aa0 msgr2=0x7f766c1607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 --2- 172.21.15.43:0/4213583757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c0d1aa0 0x7f766c1607f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f765c009870 tx=0x7f765c01c440 comp rx=0 tx=0).stop 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 -- 172.21.15.43:0/4213583757 shutdown_connections 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 --2- 172.21.15.43:0/4213583757 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c0d1aa0 0x7f766c1607f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 --2- 172.21.15.43:0/4213583757 >> v2:172.21.15.113:3300/0 conn(0x7f766c15c310 0x7f766c0b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 -- 172.21.15.43:0/4213583757 >> 172.21.15.43:0/4213583757 conn(0x7f766c0d2110 msgr2=0x7f766c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 -- 172.21.15.43:0/4213583757 shutdown_connections 2024-03-30T13:12:08.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.155+0000 7f7670a5b640 1 -- 172.21.15.43:0/4213583757 wait complete. 2024-03-30T13:12:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 Processor -- start 2024-03-30T13:12:08.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 -- start start 2024-03-30T13:12:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f766c070ed0 0x7f766c1fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f766c0d1aa0 0x7f766c0cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c15c310 0x7f766c0d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f766c071860 con 0x7f766c070ed0 2024-03-30T13:12:08.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f766c0716e0 con 0x7f766c0d1aa0 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f766c169120 con 0x7f766c15c310 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f766c070ed0 0x7f766c1fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f766c070ed0 0x7f766c1fa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57530/0 (socket says 172.21.15.43:57530) 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 -- 172.21.15.43:0/3930260980 learned_addr learned my addr 172.21.15.43:0/3930260980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7669d9b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f766c0d1aa0 0x7f766c0cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766ad9d640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c15c310 0x7f766c0d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 -- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c15c310 msgr2=0x7f766c0d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c15c310 0x7f766c0d01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 -- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f766c0d1aa0 msgr2=0x7f766c0cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f766c0d1aa0 0x7f766c0cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 -- 172.21.15.43:0/3930260980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766c156250 con 0x7f766c070ed0 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7669d9b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f766c0d1aa0 0x7f766c0cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766ad9d640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c15c310 0x7f766c0d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:08.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f766a59c640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f766c070ed0 0x7f766c1fa3d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f766000e930 tx=0x7f766000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f765affd640 1 -- 172.21.15.43:0/3930260980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660017070 con 0x7f766c070ed0 2024-03-30T13:12:08.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f765affd640 1 -- 172.21.15.43:0/3930260980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f766000c9a0 con 0x7f766c070ed0 2024-03-30T13:12:08.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f765affd640 1 -- 172.21.15.43:0/3930260980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766000cd50 con 0x7f766c070ed0 2024-03-30T13:12:08.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766c1fa910 con 0x7f766c070ed0 2024-03-30T13:12:08.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766c155f70 con 0x7f766c070ed0 2024-03-30T13:12:08.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.159+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f766c068890 con 0x7f766c070ed0 2024-03-30T13:12:08.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.163+0000 7f765affd640 1 -- 172.21.15.43:0/3930260980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f766001a070 con 0x7f766c070ed0 2024-03-30T13:12:08.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.163+0000 7f765affd640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f764c067fc0 0x7f764c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.163+0000 7f765affd640 1 -- 172.21.15.43:0/3930260980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f76600867f0 con 0x7f766c070ed0 2024-03-30T13:12:08.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.163+0000 7f7669d9b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f764c067fc0 0x7f764c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.163+0000 7f7669d9b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f764c067fc0 0x7f764c06a480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f765c0ad000 tx=0x7f765c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.167+0000 7f765affd640 1 -- 172.21.15.43:0/3930260980 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f766c156250 con 0x7f766c070ed0 2024-03-30T13:12:08.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.339+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f766c0b3250 con 0x7f766c070ed0 2024-03-30T13:12:08.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.339+0000 7f765affd640 1 -- 172.21.15.43:0/3930260980 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f7660056d80 con 0x7f766c070ed0 2024-03-30T13:12:08.348 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:12:08.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f764c067fc0 msgr2=0x7f764c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f764c067fc0 0x7f764c06a480 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f765c0ad000 tx=0x7f765c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f766c070ed0 msgr2=0x7f766c1fa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f766c070ed0 0x7f766c1fa3d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f766000e930 tx=0x7f766000edf0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 shutdown_connections 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f764c067fc0 0x7f764c06a480 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f766c15c310 0x7f766c0d01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f766c0d1aa0 0x7f766c0cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 --2- 172.21.15.43:0/3930260980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f766c070ed0 0x7f766c1fa3d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 >> 172.21.15.43:0/3930260980 conn(0x7f766c0d2110 msgr2=0x7f766c0b5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 shutdown_connections 2024-03-30T13:12:08.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.343+0000 7f7670a5b640 1 -- 172.21.15.43:0/3930260980 wait complete. 2024-03-30T13:12:08.367 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-03-30T13:12:08.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:08.368 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-30T13:12:08.381 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:12:08.381 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:12:08.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:08.382 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T13:12:08.430 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:12:08.430 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:08.436 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:12:08.436 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:08.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 Processor -- start 2024-03-30T13:12:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 -- start start 2024-03-30T13:12:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e600afd70 0x7f6e600cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6e600d01b0 0x7f6e60160760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600cd140 0x7f6e600cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6e600b0150 con 0x7f6e600afd70 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6e600cf3f0 con 0x7f6e600d01b0 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e64b6b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6e600cf6d0 con 0x7f6e600cd140 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e600afd70 0x7f6e600cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6e600afd70 0x7f6e600cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57536/0 (socket says 172.21.15.43:57536) 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 -- 172.21.15.43:0/128525386 learned_addr learned my addr 172.21.15.43:0/128525386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5ed9d640 1 --2- 172.21.15.43:0/128525386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600cd140 0x7f6e600cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 -- 172.21.15.43:0/128525386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600cd140 msgr2=0x7f6e600cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 --2- 172.21.15.43:0/128525386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600cd140 0x7f6e600cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 -- 172.21.15.43:0/128525386 >> v2:172.21.15.113:3300/0 conn(0x7f6e600d01b0 msgr2=0x7f6e60160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 --2- 172.21.15.43:0/128525386 >> v2:172.21.15.113:3300/0 conn(0x7f6e600d01b0 0x7f6e60160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 -- 172.21.15.43:0/128525386 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e600cf9b0 con 0x7f6e600afd70 2024-03-30T13:12:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5e59c640 1 --2- 172.21.15.43:0/128525386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e600cc910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6e6006fdc0 tx=0x7f6e5001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6822267b93920542 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5d59a640 1 -- 172.21.15.43:0/128525386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500b0070 con 0x7f6e600afd70 2024-03-30T13:12:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.595+0000 7f6e5d59a640 1 -- 172.21.15.43:0/128525386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e500ab600 con 0x7f6e600afd70 2024-03-30T13:12:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5d59a640 1 -- 172.21.15.43:0/128525386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500ab9d0 con 0x7f6e600afd70 2024-03-30T13:12:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- 172.21.15.43:0/128525386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 msgr2=0x7f6e600cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/128525386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e600cc910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6e6006fdc0 tx=0x7f6e5001c520 comp rx=0 tx=0).stop 2024-03-30T13:12:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- 172.21.15.43:0/128525386 shutdown_connections 2024-03-30T13:12:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/128525386 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600cd140 0x7f6e600cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/128525386 >> v2:172.21.15.113:3300/0 conn(0x7f6e600d01b0 0x7f6e60160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/128525386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e600cc910 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- 172.21.15.43:0/128525386 >> 172.21.15.43:0/128525386 conn(0x7f6e600b2b30 msgr2=0x7f6e600d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- 172.21.15.43:0/128525386 shutdown_connections 2024-03-30T13:12:08.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- 172.21.15.43:0/128525386 wait complete. 2024-03-30T13:12:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 Processor -- start 2024-03-30T13:12:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- start start 2024-03-30T13:12:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e60132d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e600cd140 0x7f6e6013d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600d01b0 0x7f6e6013da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6e601425b0 con 0x7f6e600afd70 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6e60142430 con 0x7f6e600cd140 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6e60142730 con 0x7f6e600d01b0 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e60132d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e60132d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57544/0 (socket says 172.21.15.43:57544) 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 -- 172.21.15.43:0/2405818428 learned_addr learned my addr 172.21.15.43:0/2405818428 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5ed9d640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600d01b0 0x7f6e6013da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5dd9b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e600cd140 0x7f6e6013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 -- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600d01b0 msgr2=0x7f6e6013da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600d01b0 0x7f6e6013da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 -- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e600cd140 msgr2=0x7f6e6013d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e600cd140 0x7f6e6013d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 -- 172.21.15.43:0/2405818428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e60133790 con 0x7f6e600afd70 2024-03-30T13:12:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e5e59c640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e60132d20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e500ae040 tx=0x7f6e5001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e4effd640 1 -- 172.21.15.43:0/2405818428 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500ba020 con 0x7f6e600afd70 2024-03-30T13:12:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e4effd640 1 -- 172.21.15.43:0/2405818428 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e500b6420 con 0x7f6e600afd70 2024-03-30T13:12:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e60133a70 con 0x7f6e600afd70 2024-03-30T13:12:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e4effd640 1 -- 172.21.15.43:0/2405818428 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e500b0040 con 0x7f6e600afd70 2024-03-30T13:12:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.599+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e60134030 con 0x7f6e600afd70 2024-03-30T13:12:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.603+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e600ce710 con 0x7f6e600afd70 2024-03-30T13:12:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.603+0000 7f6e4effd640 1 -- 172.21.15.43:0/2405818428 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6e500bd020 con 0x7f6e600afd70 2024-03-30T13:12:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.603+0000 7f6e4effd640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6e40067fc0 0x7f6e4006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.603+0000 7f6e4effd640 1 -- 172.21.15.43:0/2405818428 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f6e50120a90 con 0x7f6e600afd70 2024-03-30T13:12:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.603+0000 7f6e5dd9b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6e40067fc0 0x7f6e4006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.603+0000 7f6e5dd9b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6e40067fc0 0x7f6e4006a480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e601f8c00 tx=0x7f6e54009110 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.607+0000 7f6e4effd640 1 -- 172.21.15.43:0/2405818428 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6e500c4030 con 0x7f6e600afd70 2024-03-30T13:12:08.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.827+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e600d1470 con 0x7f6e600afd70 2024-03-30T13:12:08.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.827+0000 7f6e4effd640 1 -- 172.21.15.43:0/2405818428 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7f6e50016110 con 0x7f6e600afd70 2024-03-30T13:12:08.837 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:08.837 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:08.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6e40067fc0 msgr2=0x7f6e4006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6e40067fc0 0x7f6e4006a480 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6e601f8c00 tx=0x7f6e54009110 comp rx=0 tx=0).stop 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 msgr2=0x7f6e60132d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e60132d20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6e500ae040 tx=0x7f6e5001ad60 comp rx=0 tx=0).stop 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 shutdown_connections 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6e40067fc0 0x7f6e4006a480 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6e600d01b0 0x7f6e6013da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6e600cd140 0x7f6e6013d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 --2- 172.21.15.43:0/2405818428 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e600afd70 0x7f6e60132d20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 >> 172.21.15.43:0/2405818428 conn(0x7f6e600b2b30 msgr2=0x7f6e6006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 shutdown_connections 2024-03-30T13:12:08.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.831+0000 7f6e64b6b640 1 -- 172.21.15.43:0/2405818428 wait complete. 2024-03-30T13:12:08.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:12:08.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd946cb4640 1 Processor -- start 2024-03-30T13:12:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd946cb4640 1 -- start start 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd946cb4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd94006f9e0 0x7fd94006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd946cb4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94015d190 con 0x7fd9400d0bc0 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd946cb4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd94006bf40 con 0x7fd94006cfe0 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd946cb4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd94015cf70 con 0x7fd94006f9e0 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd944a50640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd94006f9e0 0x7fd94006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd944a50640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd94006f9e0 0x7fd94006b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57546/0 (socket says 172.21.15.43:57546) 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd944a50640 1 -- 172.21.15.43:0/1890981424 learned_addr learned my addr 172.21.15.43:0/1890981424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd944a50640 1 -- 172.21.15.43:0/1890981424 >> v1:172.21.15.113:6789/0 conn(0x7fd94006cfe0 legacy=0x7fd94006fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd944a50640 1 -- 172.21.15.43:0/1890981424 >> v1:172.21.15.43:6789/0 conn(0x7fd9400d0bc0 legacy=0x7fd94015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.967+0000 7fd944a50640 1 -- 172.21.15.43:0/1890981424 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94006c220 con 0x7fd94006f9e0 2024-03-30T13:12:08.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd944a50640 1 --2- 172.21.15.43:0/1890981424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94006b660 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd938009e50 tx=0x7fd93801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8462ca84d232ab3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd93f7fe640 1 -- 172.21.15.43:0/1890981424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380a56c0 con 0x7fd94006f9e0 2024-03-30T13:12:08.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd93f7fe640 1 -- 172.21.15.43:0/1890981424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9380a5860 con 0x7fd94006f9e0 2024-03-30T13:12:08.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd93f7fe640 1 -- 172.21.15.43:0/1890981424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9380a5c30 con 0x7fd94006f9e0 2024-03-30T13:12:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 -- 172.21.15.43:0/1890981424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 msgr2=0x7fd94006b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 --2- 172.21.15.43:0/1890981424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94006b660 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd938009e50 tx=0x7fd93801c9f0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 -- 172.21.15.43:0/1890981424 shutdown_connections 2024-03-30T13:12:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 --2- 172.21.15.43:0/1890981424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94006b660 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 -- 172.21.15.43:0/1890981424 >> 172.21.15.43:0/1890981424 conn(0x7fd9400d1070 msgr2=0x7fd9400d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 -- 172.21.15.43:0/1890981424 shutdown_connections 2024-03-30T13:12:08.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 -- 172.21.15.43:0/1890981424 wait complete. 2024-03-30T13:12:08.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 Processor -- start 2024-03-30T13:12:08.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.971+0000 7fd946cb4640 1 -- start start 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd94006cfe0 0x7fd9400affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9400d0bc0 0x7fd9400b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd94016a840 con 0x7fd94006f9e0 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd94016a6c0 con 0x7fd9400d0bc0 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd94016a9c0 con 0x7fd94006cfe0 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94014d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57560/0 (socket says 172.21.15.43:57560) 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 -- 172.21.15.43:0/422357119 learned_addr learned my addr 172.21.15.43:0/422357119 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd944a50640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd94006cfe0 0x7fd9400affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd945251640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9400d0bc0 0x7fd9400b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 -- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd94006cfe0 msgr2=0x7fd9400affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd94006cfe0 0x7fd9400affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 -- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9400d0bc0 msgr2=0x7fd9400b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9400d0bc0 0x7fd9400b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 -- 172.21.15.43:0/422357119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd940070300 con 0x7fd94006f9e0 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd945251640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9400d0bc0 0x7fd9400b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93ffff640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94014d380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c420 tx=0x7fd93400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93d7fa640 1 -- 172.21.15.43:0/422357119 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9340052d0 con 0x7fd94006f9e0 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93d7fa640 1 -- 172.21.15.43:0/422357119 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd934005470 con 0x7fd94006f9e0 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd94014fb30 con 0x7fd94006f9e0 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93d7fa640 1 -- 172.21.15.43:0/422357119 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd934007260 con 0x7fd94006f9e0 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9401500f0 con 0x7fd94006f9e0 2024-03-30T13:12:08.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd940068900 con 0x7fd94006f9e0 2024-03-30T13:12:08.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93d7fa640 1 -- 172.21.15.43:0/422357119 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd934007400 con 0x7fd94006f9e0 2024-03-30T13:12:08.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93d7fa640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd92c067ff0 0x7fd92c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:08.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.975+0000 7fd93d7fa640 1 -- 172.21.15.43:0/422357119 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fd934085d40 con 0x7fd94006f9e0 2024-03-30T13:12:08.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.979+0000 7fd944a50640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd92c067ff0 0x7fd92c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:08.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.979+0000 7fd944a50640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd92c067ff0 0x7fd92c06a4b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9380ae000 tx=0x7fd9380a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:08.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:08.983+0000 7fd93d7fa640 1 -- 172.21.15.43:0/422357119 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd934056180 con 0x7fd94006f9e0 2024-03-30T13:12:09.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.143+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd94014d8c0 con 0x7fd94006f9e0 2024-03-30T13:12:09.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd93d7fa640 1 -- 172.21.15.43:0/422357119 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fd934055a80 con 0x7fd94006f9e0 2024-03-30T13:12:09.153 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:09.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd92c067ff0 msgr2=0x7fd92c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd92c067ff0 0x7fd92c06a4b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd9380ae000 tx=0x7fd9380a7070 comp rx=0 tx=0).stop 2024-03-30T13:12:09.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 msgr2=0x7fd94014d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94014d380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c420 tx=0x7fd93400c8e0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 shutdown_connections 2024-03-30T13:12:09.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd92c067ff0 0x7fd92c06a4b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd9400d0bc0 0x7fd9400b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd94006f9e0 0x7fd94014d380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 --2- 172.21.15.43:0/422357119 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd94006cfe0 0x7fd9400affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 >> 172.21.15.43:0/422357119 conn(0x7fd9400d1070 msgr2=0x7fd9400d1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 shutdown_connections 2024-03-30T13:12:09.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.147+0000 7fd946cb4640 1 -- 172.21.15.43:0/422357119 wait complete. 2024-03-30T13:12:09.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:09.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f6405638640 1 Processor -- start 2024-03-30T13:12:09.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f6405638640 1 -- start start 2024-03-30T13:12:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f6405638640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f640015e2f0 0x7f64000b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f6405638640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f640015e6d0 0x7f64000ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f6405638640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64000b58c0 con 0x7f64000ad8e0 2024-03-30T13:12:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f6405638640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f64000cd060 con 0x7f640015e2f0 2024-03-30T13:12:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f6405638640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f640006b660 con 0x7f640015e6d0 2024-03-30T13:12:09.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63feffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f640015e2f0 0x7f64000b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63feffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f640015e2f0 0x7f64000b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57570/0 (socket says 172.21.15.43:57570) 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63feffd640 1 -- 172.21.15.43:0/1043865812 learned_addr learned my addr 172.21.15.43:0/1043865812 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63feffd640 1 -- 172.21.15.43:0/1043865812 >> v2:172.21.15.113:3300/0 conn(0x7f640015e6d0 msgr2=0x7f64000ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63fe7fc640 1 --2- 172.21.15.43:0/1043865812 >> v2:172.21.15.113:3300/0 conn(0x7f640015e6d0 0x7f64000ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63feffd640 1 --2- 172.21.15.43:0/1043865812 >> v2:172.21.15.113:3300/0 conn(0x7f640015e6d0 0x7f64000ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63feffd640 1 -- 172.21.15.43:0/1043865812 >> v1:172.21.15.113:6789/0 conn(0x7f64000ad8e0 legacy=0x7f64001620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63feffd640 1 -- 172.21.15.43:0/1043865812 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64000b5ba0 con 0x7f640015e2f0 2024-03-30T13:12:09.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.299+0000 7f63fe7fc640 1 --2- 172.21.15.43:0/1043865812 >> v2:172.21.15.113:3300/0 conn(0x7f640015e6d0 0x7f64000ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f63feffd640 1 --2- 172.21.15.43:0/1043865812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f64000b51e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6400068380 tx=0x7f63f001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb2e6e54ddff5f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f63fdffb640 1 -- 172.21.15.43:0/1043865812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b0070 con 0x7f640015e2f0 2024-03-30T13:12:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f63fdffb640 1 -- 172.21.15.43:0/1043865812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f00ab600 con 0x7f640015e2f0 2024-03-30T13:12:09.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f63fdffb640 1 -- 172.21.15.43:0/1043865812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f00ab9d0 con 0x7f640015e2f0 2024-03-30T13:12:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 -- 172.21.15.43:0/1043865812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 msgr2=0x7f64000b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 --2- 172.21.15.43:0/1043865812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f64000b51e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6400068380 tx=0x7f63f001c390 comp rx=0 tx=0).stop 2024-03-30T13:12:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 -- 172.21.15.43:0/1043865812 shutdown_connections 2024-03-30T13:12:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 --2- 172.21.15.43:0/1043865812 >> v2:172.21.15.113:3300/0 conn(0x7f640015e6d0 0x7f64000ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 --2- 172.21.15.43:0/1043865812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f64000b51e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 -- 172.21.15.43:0/1043865812 >> 172.21.15.43:0/1043865812 conn(0x7f64000d0c90 msgr2=0x7f64000d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 -- 172.21.15.43:0/1043865812 shutdown_connections 2024-03-30T13:12:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 -- 172.21.15.43:0/1043865812 wait complete. 2024-03-30T13:12:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 Processor -- start 2024-03-30T13:12:09.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.303+0000 7f6405638640 1 -- start start 2024-03-30T13:12:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64000ad8e0 0x7f6400152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f6400152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f640015e6d0 0x7f640014b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f640016b480 con 0x7f640015e2f0 2024-03-30T13:12:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f640016b600 con 0x7f64000ad8e0 2024-03-30T13:12:09.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f640016b780 con 0x7f640015e6d0 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f6400152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f6400152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57586/0 (socket says 172.21.15.43:57586) 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 -- 172.21.15.43:0/1917663045 learned_addr learned my addr 172.21.15.43:0/1917663045 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63feffd640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64000ad8e0 0x7f6400152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63ff7fe640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f640015e6d0 0x7f640014b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 -- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f640015e6d0 msgr2=0x7f640014b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f640015e6d0 0x7f640014b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 -- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64000ad8e0 msgr2=0x7f6400152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64000ad8e0 0x7f6400152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 -- 172.21.15.43:0/1917663045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640014c420 con 0x7f640015e2f0 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63feffd640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64000ad8e0 0x7f6400152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63fe7fc640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f6400152db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f63f400c7c0 tx=0x7f63f400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63e77fe640 1 -- 172.21.15.43:0/1917663045 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4007a30 con 0x7f640015e2f0 2024-03-30T13:12:09.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63e77fe640 1 -- 172.21.15.43:0/1917663045 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f4007bd0 con 0x7f640015e2f0 2024-03-30T13:12:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640014c700 con 0x7f640015e2f0 2024-03-30T13:12:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63e77fe640 1 -- 172.21.15.43:0/1917663045 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f40113f0 con 0x7f640015e2f0 2024-03-30T13:12:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640014ccc0 con 0x7f640015e2f0 2024-03-30T13:12:09.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f640014cec0 con 0x7f640015e2f0 2024-03-30T13:12:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63e77fe640 1 -- 172.21.15.43:0/1917663045 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f63f4011590 con 0x7f640015e2f0 2024-03-30T13:12:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63e77fe640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f63ec067fc0 0x7f63ec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63e77fe640 1 -- 172.21.15.43:0/1917663045 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f63f4086ab0 con 0x7f640015e2f0 2024-03-30T13:12:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.307+0000 7f63feffd640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f63ec067fc0 0x7f63ec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.311+0000 7f63feffd640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f63ec067fc0 0x7f63ec06a480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ad000 tx=0x7f63f0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.311+0000 7f63e77fe640 1 -- 172.21.15.43:0/1917663045 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f63f4015020 con 0x7f640015e2f0 2024-03-30T13:12:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6400068900 con 0x7f640015e2f0 2024-03-30T13:12:09.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f63e77fe640 1 -- 172.21.15.43:0/1917663045 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f63f4056ef0 con 0x7f640015e2f0 2024-03-30T13:12:09.534 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f63ec067fc0 msgr2=0x7f63ec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f63ec067fc0 0x7f63ec06a480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f63f00ad000 tx=0x7f63f0008040 comp rx=0 tx=0).stop 2024-03-30T13:12:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 msgr2=0x7f6400152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f6400152db0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f63f400c7c0 tx=0x7f63f400cc80 comp rx=0 tx=0).stop 2024-03-30T13:12:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 shutdown_connections 2024-03-30T13:12:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f63ec067fc0 0x7f63ec06a480 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f640015e6d0 0x7f640014b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f640015e2f0 0x7f6400152db0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 --2- 172.21.15.43:0/1917663045 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f64000ad8e0 0x7f6400152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 >> 172.21.15.43:0/1917663045 conn(0x7f64000d0c90 msgr2=0x7f64000b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 shutdown_connections 2024-03-30T13:12:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.527+0000 7f6405638640 1 -- 172.21.15.43:0/1917663045 wait complete. 2024-03-30T13:12:09.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.667+0000 7f1aa562c640 1 Processor -- start 2024-03-30T13:12:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.667+0000 7f1aa562c640 1 -- start start 2024-03-30T13:12:09.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1aa562c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1aa015afc0 0x7f1aa00b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1aa562c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1aa00b4520 0x7f1aa006f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1aa562c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa00b4ba0 con 0x7f1aa006ff20 2024-03-30T13:12:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1aa562c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa0070c30 con 0x7f1aa00b4520 2024-03-30T13:12:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1aa562c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa00b4960 con 0x7f1aa015afc0 2024-03-30T13:12:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a97fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1aa00b4520 0x7f1aa006f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a9f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1aa006ff20 0x7f1aa006c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39048/0 (socket says 172.21.15.43:39048) 2024-03-30T13:12:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a9f7fe640 1 -- 172.21.15.43:0/2153434795 learned_addr learned my addr 172.21.15.43:0/2153434795 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a9effd640 1 --2- 172.21.15.43:0/2153434795 >> v2:172.21.15.113:3300/0 conn(0x7f1aa015afc0 0x7f1aa00b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a97fff640 1 -- 172.21.15.43:0/2153434795 >> v2:172.21.15.113:3300/0 conn(0x7f1aa015afc0 msgr2=0x7f1aa00b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a97fff640 1 --2- 172.21.15.43:0/2153434795 >> v2:172.21.15.113:3300/0 conn(0x7f1aa015afc0 0x7f1aa00b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a97fff640 1 -- 172.21.15.43:0/2153434795 >> v1:172.21.15.43:6789/0 conn(0x7f1aa006ff20 legacy=0x7f1aa006c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a9e7fc640 1 -- 172.21.15.43:0/2153434795 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189916770 0 0) 0x7f1aa00b4ba0 con 0x7f1aa006ff20 2024-03-30T13:12:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a97fff640 1 -- 172.21.15.43:0/2153434795 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa015ec60 con 0x7f1aa00b4520 2024-03-30T13:12:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.671+0000 7f1a97fff640 1 --2- 172.21.15.43:0/2153434795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa006f9e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0073e60 tx=0x7f1a8801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d87e3aeb2cacdf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a9e7fc640 1 -- 172.21.15.43:0/2153434795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880b0070 con 0x7f1aa00b4520 2024-03-30T13:12:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a9e7fc640 1 -- 172.21.15.43:0/2153434795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a880ab600 con 0x7f1aa00b4520 2024-03-30T13:12:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a9e7fc640 1 -- 172.21.15.43:0/2153434795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880ab9d0 con 0x7f1aa00b4520 2024-03-30T13:12:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- 172.21.15.43:0/2153434795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 msgr2=0x7f1aa006f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 --2- 172.21.15.43:0/2153434795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa006f9e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0073e60 tx=0x7f1a8801c480 comp rx=0 tx=0).stop 2024-03-30T13:12:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- 172.21.15.43:0/2153434795 shutdown_connections 2024-03-30T13:12:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 --2- 172.21.15.43:0/2153434795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa006f9e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 --2- 172.21.15.43:0/2153434795 >> v2:172.21.15.113:3300/0 conn(0x7f1aa015afc0 0x7f1aa00b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- 172.21.15.43:0/2153434795 >> 172.21.15.43:0/2153434795 conn(0x7f1aa00d1070 msgr2=0x7f1aa00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:09.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- 172.21.15.43:0/2153434795 shutdown_connections 2024-03-30T13:12:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- 172.21.15.43:0/2153434795 wait complete. 2024-03-30T13:12:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 Processor -- start 2024-03-30T13:12:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- start start 2024-03-30T13:12:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aa006ff20 0x7f1aa015fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa015afc0 0x7f1aa0150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa015fa20 con 0x7f1aa00b4520 2024-03-30T13:12:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa00cf7c0 con 0x7f1aa015afc0 2024-03-30T13:12:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa0163180 con 0x7f1aa006ff20 2024-03-30T13:12:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa0154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57608/0 (socket says 172.21.15.43:57608) 2024-03-30T13:12:09.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 -- 172.21.15.43:0/3319831710 learned_addr learned my addr 172.21.15.43:0/3319831710 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a9effd640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aa006ff20 0x7f1aa015fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 -- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aa006ff20 msgr2=0x7f1aa015fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aa006ff20 0x7f1aa015fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a9f7fe640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa015afc0 0x7f1aa0150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 -- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa015afc0 msgr2=0x7f1aa0150d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa015afc0 0x7f1aa0150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 -- 172.21.15.43:0/3319831710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa01545d0 con 0x7f1aa00b4520 2024-03-30T13:12:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a97fff640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa0154090 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1a88019090 tx=0x7f1a88019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a977fe640 1 -- 172.21.15.43:0/3319831710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880b0040 con 0x7f1aa00b4520 2024-03-30T13:12:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a977fe640 1 -- 172.21.15.43:0/3319831710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a880b9070 con 0x7f1aa00b4520 2024-03-30T13:12:09.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa0160ce0 con 0x7f1aa00b4520 2024-03-30T13:12:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1a977fe640 1 -- 172.21.15.43:0/3319831710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880b4610 con 0x7f1aa00b4520 2024-03-30T13:12:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.675+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa0160720 con 0x7f1aa00b4520 2024-03-30T13:12:09.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.679+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa0068900 con 0x7f1aa00b4520 2024-03-30T13:12:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.679+0000 7f1a977fe640 1 -- 172.21.15.43:0/3319831710 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1a880b47b0 con 0x7f1aa00b4520 2024-03-30T13:12:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.679+0000 7f1a977fe640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a80067ff0 0x7f1a8006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.679+0000 7f1a977fe640 1 -- 172.21.15.43:0/3319831710 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f1a88121230 con 0x7f1aa00b4520 2024-03-30T13:12:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.679+0000 7f1a9effd640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a80067ff0 0x7f1a8006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.679+0000 7f1a9effd640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a80067ff0 0x7f1a8006a4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009790 tx=0x7f1a90008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:09.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.683+0000 7f1a977fe640 1 -- 172.21.15.43:0/3319831710 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1aa01545d0 con 0x7f1aa00b4520 2024-03-30T13:12:09.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1aa015b4c0 con 0x7f1aa00b4520 2024-03-30T13:12:09.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1a977fe640 1 -- 172.21.15.43:0/3319831710 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f1a88016070 con 0x7f1aa00b4520 2024-03-30T13:12:09.854 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a80067ff0 msgr2=0x7f1a8006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a80067ff0 0x7f1a8006a4b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a90009790 tx=0x7f1a90008040 comp rx=0 tx=0).stop 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 msgr2=0x7f1aa0154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa0154090 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1a88019090 tx=0x7f1a88019220 comp rx=0 tx=0).stop 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 shutdown_connections 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a80067ff0 0x7f1a8006a4b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa015afc0 0x7f1aa0150d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa00b4520 0x7f1aa0154090 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.847+0000 7f1aa562c640 1 --2- 172.21.15.43:0/3319831710 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1aa006ff20 0x7f1aa015fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.851+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 >> 172.21.15.43:0/3319831710 conn(0x7f1aa00d1070 msgr2=0x7f1aa00730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.851+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 shutdown_connections 2024-03-30T13:12:09.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:09.851+0000 7f1aa562c640 1 -- 172.21.15.43:0/3319831710 wait complete. 2024-03-30T13:12:09.874 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:12:10.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d68665640 1 Processor -- start 2024-03-30T13:12:10.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d68665640 1 -- start start 2024-03-30T13:12:10.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d68665640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600cc750 0x7f0d600ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d68665640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d600b6c50 0x7f0d600b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d68665640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d60158f20 con 0x7f0d60158b40 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d68665640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0d60162d00 con 0x7f0d600b6c50 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d68665640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d601620f0 con 0x7f0d600cc750 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d66c02640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d600b6c50 0x7f0d600b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d66c02640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d600b6c50 0x7f0d600b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57616/0 (socket says 172.21.15.43:57616) 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d66c02640 1 -- 172.21.15.43:0/2895469340 learned_addr learned my addr 172.21.15.43:0/2895469340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d65c00640 1 --2- 172.21.15.43:0/2895469340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600cc750 0x7f0d600ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d66c02640 1 -- 172.21.15.43:0/2895469340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600cc750 msgr2=0x7f0d600ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d66c02640 1 --2- 172.21.15.43:0/2895469340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600cc750 0x7f0d600ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d66c02640 1 -- 172.21.15.43:0/2895469340 >> v1:172.21.15.43:6789/0 conn(0x7f0d60158b40 legacy=0x7f0d600b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.003+0000 7f0d66c02640 1 -- 172.21.15.43:0/2895469340 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d601623d0 con 0x7f0d600b6c50 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d66c02640 1 --2- 172.21.15.43:0/2895469340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d600b6c50 0x7f0d600b70a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009e50 tx=0x7f0d5801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71892a528250300d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d653ff640 1 -- 172.21.15.43:0/2895469340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580a5750 con 0x7f0d600b6c50 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d653ff640 1 -- 172.21.15.43:0/2895469340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d580a58f0 con 0x7f0d600b6c50 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d653ff640 1 -- 172.21.15.43:0/2895469340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580a5cc0 con 0x7f0d600b6c50 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 -- 172.21.15.43:0/2895469340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d600b6c50 msgr2=0x7f0d600b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 --2- 172.21.15.43:0/2895469340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d600b6c50 0x7f0d600b70a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009e50 tx=0x7f0d5801c9a0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 -- 172.21.15.43:0/2895469340 shutdown_connections 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 --2- 172.21.15.43:0/2895469340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d600b6c50 0x7f0d600b70a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 --2- 172.21.15.43:0/2895469340 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600cc750 0x7f0d600ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 -- 172.21.15.43:0/2895469340 >> 172.21.15.43:0/2895469340 conn(0x7f0d600b3cf0 msgr2=0x7f0d600b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 -- 172.21.15.43:0/2895469340 shutdown_connections 2024-03-30T13:12:10.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 -- 172.21.15.43:0/2895469340 wait complete. 2024-03-30T13:12:10.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 Processor -- start 2024-03-30T13:12:10.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.007+0000 7f0d68665640 1 -- start start 2024-03-30T13:12:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600b6c50 0x7f0d600b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d600cc750 0x7f0d60147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d60158b40 0x7f0d60160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600d1eb0 con 0x7f0d60158b40 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d600d1d30 con 0x7f0d600cc750 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d600b7df0 con 0x7f0d600b6c50 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d60158b40 0x7f0d60160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d60158b40 0x7f0d60160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57622/0 (socket says 172.21.15.43:57622) 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 -- 172.21.15.43:0/1098862303 learned_addr learned my addr 172.21.15.43:0/1098862303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d65c00640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d600cc750 0x7f0d60147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66401640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600b6c50 0x7f0d600b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 -- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600b6c50 msgr2=0x7f0d600b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600b6c50 0x7f0d600b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 -- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d600cc750 msgr2=0x7f0d60147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d600cc750 0x7f0d60147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 -- 172.21.15.43:0/1098862303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d60161be0 con 0x7f0d60158b40 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66c02640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d60158b40 0x7f0d60160ba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0d580af040 tx=0x7f0d58019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d56ffd640 1 -- 172.21.15.43:0/1098862303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580bc020 con 0x7f0d60158b40 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d56ffd640 1 -- 172.21.15.43:0/1098862303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d580b8670 con 0x7f0d60158b40 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d601568d0 con 0x7f0d60158b40 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d56ffd640 1 -- 172.21.15.43:0/1098862303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d580b8a60 con 0x7f0d60158b40 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d600b26a0 con 0x7f0d60158b40 2024-03-30T13:12:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d60068900 con 0x7f0d60158b40 2024-03-30T13:12:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d56ffd640 1 -- 172.21.15.43:0/1098862303 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0d580bf020 con 0x7f0d60158b40 2024-03-30T13:12:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d56ffd640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d44067fc0 0x7f0d4406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d56ffd640 1 -- 172.21.15.43:0/1098862303 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0d58121850 con 0x7f0d60158b40 2024-03-30T13:12:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.011+0000 7f0d66401640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d44067fc0 0x7f0d4406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.015+0000 7f0d66401640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d44067fc0 0x7f0d4406a480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c000f70 tx=0x7f0d5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.015+0000 7f0d56ffd640 1 -- 172.21.15.43:0/1098862303 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0d580c6030 con 0x7f0d60158b40 2024-03-30T13:12:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.179+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0d600cd540 con 0x7f0d60158b40 2024-03-30T13:12:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.179+0000 7f0d56ffd640 1 -- 172.21.15.43:0/1098862303 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f0d58016620 con 0x7f0d60158b40 2024-03-30T13:12:10.185 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.179+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d44067fc0 msgr2=0x7f0d4406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.179+0000 7f0d68665640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d44067fc0 0x7f0d4406a480 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c000f70 tx=0x7f0d5c009040 comp rx=0 tx=0).stop 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.179+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d60158b40 msgr2=0x7f0d60160ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.179+0000 7f0d68665640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d60158b40 0x7f0d60160ba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0d580af040 tx=0x7f0d58019740 comp rx=0 tx=0).stop 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 shutdown_connections 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d44067fc0 0x7f0d4406a480 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d60158b40 0x7f0d60160ba0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d600cc750 0x7f0d60147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 --2- 172.21.15.43:0/1098862303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d600b6c50 0x7f0d600b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 >> 172.21.15.43:0/1098862303 conn(0x7f0d600b3cf0 msgr2=0x7f0d600ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 shutdown_connections 2024-03-30T13:12:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.183+0000 7f0d68665640 1 -- 172.21.15.43:0/1098862303 wait complete. 2024-03-30T13:12:10.206 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:10.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cc735640 1 Processor -- start 2024-03-30T13:12:10.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cc735640 1 -- start start 2024-03-30T13:12:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cc735640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa8c40b5b20 0x7fa8c406d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cc735640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c415edd0 con 0x7fa8c406dcc0 2024-03-30T13:12:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cc735640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8c415f0b0 con 0x7fa8c4070490 2024-03-30T13:12:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cc735640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa8c415f390 con 0x7fa8c40b5b20 2024-03-30T13:12:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cacd2640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa8c406dcc0 0x7fa8c406e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39060/0 (socket says 172.21.15.43:39060) 2024-03-30T13:12:10.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cacd2640 1 -- 172.21.15.43:0/265900628 learned_addr learned my addr 172.21.15.43:0/265900628 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c9cd0640 1 --2- 172.21.15.43:0/265900628 >> v2:172.21.15.113:3300/0 conn(0x7fa8c40b5b20 0x7fa8c406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331900153 0 0) 0x7fa8c415edd0 con 0x7fa8c406dcc0 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8b0003530 con 0x7fa8c406dcc0 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3023407163 0 0) 0x7fa8b0003530 con 0x7fa8c406dcc0 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 >> v2:172.21.15.113:3300/0 conn(0x7fa8c40b5b20 msgr2=0x7fa8c406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 --2- 172.21.15.43:0/265900628 >> v2:172.21.15.113:3300/0 conn(0x7fa8c40b5b20 0x7fa8c406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 >> v1:172.21.15.113:6789/0 conn(0x7fa8c4070490 legacy=0x7fa8c415c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c415f670 con 0x7fa8c406dcc0 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fa8bc001a20 con 0x7fa8c406dcc0 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 >> v1:172.21.15.43:6789/0 conn(0x7fa8c406dcc0 legacy=0x7fa8c406e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8b0004810 0x7fa8b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b00052f0 0x7fa8b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c415f390 con 0x7fa8b00039c0 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b000c1c0 con 0x7fa8b00052f0 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b000c3c0 con 0x7fa8b0004810 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8ca4d1640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cacd2640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b00052f0 0x7fa8b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8ca4d1640 1 -- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8b0004810 msgr2=0x7fa8b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8ca4d1640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8b0004810 0x7fa8b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8ca4d1640 1 -- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b00052f0 msgr2=0x7fa8b000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8ca4d1640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b00052f0 0x7fa8b000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8ca4d1640 1 -- 172.21.15.43:0/265900628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b0003530 con 0x7fa8b00039c0 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cacd2640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8b00052f0 0x7fa8b000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:10.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.315+0000 7fa8cacd2640 1 -- 172.21.15.43:0/265900628 reap_dead start 2024-03-30T13:12:10.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8ca4d1640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8b0003de0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0001750 tx=0x7fa8c001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d57c26cc3dd0ecb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00013a0 con 0x7fa8b00039c0 2024-03-30T13:12:10.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c00052f0 con 0x7fa8b00039c0 2024-03-30T13:12:10.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8c94cf640 1 -- 172.21.15.43:0/265900628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0003840 con 0x7fa8b00039c0 2024-03-30T13:12:10.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8cc735640 1 -- 172.21.15.43:0/265900628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 msgr2=0x7fa8b0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8cc735640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8b0003de0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0001750 tx=0x7fa8c001c3c0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8cc735640 1 -- 172.21.15.43:0/265900628 shutdown_connections 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8cc735640 1 --2- 172.21.15.43:0/265900628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8b0003de0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8cc735640 1 -- 172.21.15.43:0/265900628 >> 172.21.15.43:0/265900628 conn(0x7fa8c4072500 msgr2=0x7fa8c4072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8cc735640 1 -- 172.21.15.43:0/265900628 shutdown_connections 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.319+0000 7fa8cc735640 1 -- 172.21.15.43:0/265900628 wait complete. 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 Processor -- start 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 -- start start 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8c4147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8c415d7a0 0x7fa8c415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c40ceae0 0x7fa8c40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c40b4820 con 0x7fa8b00039c0 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c40b46a0 con 0x7fa8c40ceae0 2024-03-30T13:12:10.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa8c40b49a0 con 0x7fa8c415d7a0 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8c4147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8c4147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57638/0 (socket says 172.21.15.43:57638) 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 -- 172.21.15.43:0/2535299931 learned_addr learned my addr 172.21.15.43:0/2535299931 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8c9cd0640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8c415d7a0 0x7fa8c415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cacd2640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c40ceae0 0x7fa8c40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 -- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8c415d7a0 msgr2=0x7fa8c415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8c415d7a0 0x7fa8c415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 -- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c40ceae0 msgr2=0x7fa8c40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c40ceae0 0x7fa8c40cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 -- 172.21.15.43:0/2535299931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c41e6ec0 con 0x7fa8b00039c0 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8c9cd0640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8c415d7a0 0x7fa8c415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8ca4d1640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8c4147a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa8c001c8a0 tx=0x7fa8b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8baffd640 1 -- 172.21.15.43:0/2535299931 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0007a40 con 0x7fa8b00039c0 2024-03-30T13:12:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8baffd640 1 -- 172.21.15.43:0/2535299931 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c00ba070 con 0x7fa8b00039c0 2024-03-30T13:12:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c41e71a0 con 0x7fa8b00039c0 2024-03-30T13:12:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8baffd640 1 -- 172.21.15.43:0/2535299931 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00054a0 con 0x7fa8b00039c0 2024-03-30T13:12:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c40cf4a0 con 0x7fa8b00039c0 2024-03-30T13:12:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c4068890 con 0x7fa8b00039c0 2024-03-30T13:12:10.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.323+0000 7fa8baffd640 1 -- 172.21.15.43:0/2535299931 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa8c0007030 con 0x7fa8b00039c0 2024-03-30T13:12:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.327+0000 7fa8baffd640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa8a8067ff0 0x7fa8a806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.327+0000 7fa8baffd640 1 -- 172.21.15.43:0/2535299931 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fa8c0121220 con 0x7fa8b00039c0 2024-03-30T13:12:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.327+0000 7fa8c9cd0640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa8a8067ff0 0x7fa8a806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.327+0000 7fa8c9cd0640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa8a8067ff0 0x7fa8a806a4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40cd170 tx=0x7fa8b4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.331+0000 7fa8baffd640 1 -- 172.21.15.43:0/2535299931 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fa8c00160d0 con 0x7fa8b00039c0 2024-03-30T13:12:10.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.503+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8c40b53f0 con 0x7fa8b00039c0 2024-03-30T13:12:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.503+0000 7fa8baffd640 1 -- 172.21.15.43:0/2535299931 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fa8c00f1860 con 0x7fa8b00039c0 2024-03-30T13:12:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:12:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa8a8067ff0 msgr2=0x7fa8a806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa8a8067ff0 0x7fa8a806a4b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8c40cd170 tx=0x7fa8b4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 msgr2=0x7fa8c4147a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8c4147a50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa8c001c8a0 tx=0x7fa8b0005840 comp rx=0 tx=0).stop 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 shutdown_connections 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa8a8067ff0 0x7fa8a806a4b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa8c40ceae0 0x7fa8c40cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa8c415d7a0 0x7fa8c415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 --2- 172.21.15.43:0/2535299931 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8b00039c0 0x7fa8c4147a50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 >> 172.21.15.43:0/2535299931 conn(0x7fa8c4072500 msgr2=0x7fa8c4157930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 shutdown_connections 2024-03-30T13:12:10.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.507+0000 7fa8cc735640 1 -- 172.21.15.43:0/2535299931 wait complete. 2024-03-30T13:12:10.530 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:12:10.531 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:12:10.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.655+0000 7f5b7aa9e640 1 Processor -- start 2024-03-30T13:12:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.655+0000 7f5b7aa9e640 1 -- start start 2024-03-30T13:12:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b7aa9e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5b741420a0 0x7f5b740cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b7aa9e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b740cdc10 con 0x7f5b740cfb00 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b7aa9e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b740cf060 con 0x7f5b7415c0a0 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b7aa9e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5b740cf670 con 0x7f5b741420a0 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5b740cfb00 0x7f5b74141990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39074/0 (socket says 172.21.15.43:39074) 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 -- 172.21.15.43:0/971302574 learned_addr learned my addr 172.21.15.43:0/971302574 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b7903b640 1 --2- 172.21.15.43:0/971302574 >> v2:172.21.15.113:3300/0 conn(0x7f5b741420a0 0x7f5b740cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3180813560 0 0) 0x7f5b740cdc10 con 0x7f5b740cfb00 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b60003530 con 0x7f5b740cfb00 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2753021810 0 0) 0x7f5b60003530 con 0x7f5b740cfb00 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 >> v2:172.21.15.113:3300/0 conn(0x7f5b741420a0 msgr2=0x7f5b740cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 --2- 172.21.15.43:0/971302574 >> v2:172.21.15.113:3300/0 conn(0x7f5b741420a0 0x7f5b740cff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 >> v1:172.21.15.113:6789/0 conn(0x7f5b7415c0a0 legacy=0x7f5b7406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b740cd650 con 0x7f5b740cfb00 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f5b5c001680 con 0x7f5b740cfb00 2024-03-30T13:12:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 >> v1:172.21.15.43:6789/0 conn(0x7f5b740cfb00 legacy=0x7f5b74141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b600039c0 0x7f5b60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b600051d0 0x7f5b6000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740cf670 con 0x7f5b60004810 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6000c070 con 0x7f5b600051d0 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6000c270 con 0x7f5b600039c0 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 -- 172.21.15.43:0/971302574 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b600039c0 msgr2=0x7f5b60003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b600039c0 0x7f5b60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 -- 172.21.15.43:0/971302574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b600051d0 msgr2=0x7f5b6000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b7903b640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b600051d0 0x7f5b6000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b600051d0 0x7f5b6000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b7903b640 1 -- 172.21.15.43:0/971302574 reap_dead start 2024-03-30T13:12:10.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.659+0000 7f5b73fff640 1 -- 172.21.15.43:0/971302574 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b60003530 con 0x7f5b60004810 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b73fff640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b60004c90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c210 tx=0x7f5b5c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae8b6105c717cb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c007d20 con 0x7f5b60004810 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b5c005b00 con 0x7f5b60004810 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b737fe640 1 -- 172.21.15.43:0/971302574 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c007070 con 0x7f5b60004810 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/971302574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 msgr2=0x7f5b60004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b60004c90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00c210 tx=0x7f5b5c01f330 comp rx=0 tx=0).stop 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/971302574 shutdown_connections 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/971302574 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b60004c90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/971302574 >> 172.21.15.43:0/971302574 conn(0x7f5b740b2980 msgr2=0x7f5b740d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/971302574 shutdown_connections 2024-03-30T13:12:10.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/971302574 wait complete. 2024-03-30T13:12:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 Processor -- start 2024-03-30T13:12:10.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.663+0000 7f5b7aa9e640 1 -- start start 2024-03-30T13:12:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b74144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b7415c780 0x7f5b7413a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b741409b0 0x7f5b74140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7415c480 con 0x7f5b60004810 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5b740cf300 con 0x7f5b741409b0 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5b7415c600 con 0x7f5b7415c780 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b74144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b74144750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57662/0 (socket says 172.21.15.43:57662) 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 -- 172.21.15.43:0/1284428212 learned_addr learned my addr 172.21.15.43:0/1284428212 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7903b640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b741409b0 0x7f5b74140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b73fff640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b7415c780 0x7f5b7413a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 -- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b7415c780 msgr2=0x7f5b7413a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b7415c780 0x7f5b7413a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 -- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b741409b0 msgr2=0x7f5b74140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b741409b0 0x7f5b74140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 -- 172.21.15.43:0/1284428212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b741436b0 con 0x7f5b60004810 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b73fff640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b7415c780 0x7f5b7413a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:10.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7903b640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b741409b0 0x7f5b74140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7883a640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b74144750 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c480 tx=0x7f5b6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b717fa640 1 -- 172.21.15.43:0/1284428212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0052d0 con 0x7f5b60004810 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b717fa640 1 -- 172.21.15.43:0/1284428212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c005470 con 0x7f5b60004810 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b74143990 con 0x7f5b60004810 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b717fa640 1 -- 172.21.15.43:0/1284428212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c012870 con 0x7f5b60004810 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b74143f50 con 0x7f5b60004810 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b74068890 con 0x7f5b60004810 2024-03-30T13:12:10.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.667+0000 7f5b717fa640 1 -- 172.21.15.43:0/1284428212 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5b6c012a10 con 0x7f5b60004810 2024-03-30T13:12:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.671+0000 7f5b717fa640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b54067ff0 0x7f5b5406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.671+0000 7f5b717fa640 1 -- 172.21.15.43:0/1284428212 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f5b6c086ef0 con 0x7f5b60004810 2024-03-30T13:12:10.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.671+0000 7f5b73fff640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b54067ff0 0x7f5b5406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.671+0000 7f5b73fff640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b54067ff0 0x7f5b5406a4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0ae000 tx=0x7f5b5c007040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.671+0000 7f5b717fa640 1 -- 172.21.15.43:0/1284428212 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5b6c057560 con 0x7f5b60004810 2024-03-30T13:12:10.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.831+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b740d0f20 con 0x7f5b60004810 2024-03-30T13:12:10.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b717fa640 1 -- 172.21.15.43:0/1284428212 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7f5b6c056e60 con 0x7f5b60004810 2024-03-30T13:12:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:10.842 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:10.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b54067ff0 msgr2=0x7f5b5406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b54067ff0 0x7f5b5406a4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c0ae000 tx=0x7f5b5c007040 comp rx=0 tx=0).stop 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 msgr2=0x7f5b74144750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b74144750 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c00c480 tx=0x7f5b6c00c940 comp rx=0 tx=0).stop 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 shutdown_connections 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5b54067ff0 0x7f5b5406a4b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5b741409b0 0x7f5b74140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5b7415c780 0x7f5b7413a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 --2- 172.21.15.43:0/1284428212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b60004810 0x7f5b74144750 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 >> 172.21.15.43:0/1284428212 conn(0x7f5b740b2980 msgr2=0x7f5b7406dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 shutdown_connections 2024-03-30T13:12:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.835+0000 7f5b7aa9e640 1 -- 172.21.15.43:0/1284428212 wait complete. 2024-03-30T13:12:10.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:12:10.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:10.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d7c59640 1 Processor -- start 2024-03-30T13:12:10.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d7c59640 1 -- start start 2024-03-30T13:12:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d7c59640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 0x7f04d00ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d7c59640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d00cd080 con 0x7f04d006b970 2024-03-30T13:12:10.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d7c59640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d00cdbc0 con 0x7f04d01600f0 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d7c59640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f04d00b5120 con 0x7f04d015c650 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d61f6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f04d006b970 0x7f04d00b55b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39090/0 (socket says 172.21.15.43:39090) 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d61f6640 1 -- 172.21.15.43:0/1190378754 learned_addr learned my addr 172.21.15.43:0/1190378754 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d59f5640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 0x7f04d00ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3556837268 0 0) 0x7f04d00cd080 con 0x7f04d006b970 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04bc003530 con 0x7f04d006b970 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3632168657 0 0) 0x7f04bc003530 con 0x7f04d006b970 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 msgr2=0x7f04d00ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 0x7f04d00ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 >> v1:172.21.15.113:6789/0 conn(0x7f04d01600f0 legacy=0x7f04d00ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d00cd320 con 0x7f04d006b970 2024-03-30T13:12:10.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f04c8001a00 con 0x7f04d006b970 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 >> v1:172.21.15.43:6789/0 conn(0x7f04d006b970 legacy=0x7f04d00b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 0x7f04bc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04bc003f30 0x7f04bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04bc004d50 0x7f04bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04d00b5120 con 0x7f04bc004d50 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc00ba30 con 0x7f04bc003f30 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.967+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc00bc30 con 0x7f04d015c650 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04bc004d50 0x7f04bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d59f5640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 0x7f04bc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d51f4640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04bc003f30 0x7f04bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 -- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 msgr2=0x7f04bc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015c650 0x7f04bc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 -- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04bc003f30 msgr2=0x7f04bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:10.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04bc003f30 0x7f04bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 -- 172.21.15.43:0/1190378754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc003530 con 0x7f04bc004d50 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04bc004d50 0x7f04bc0051d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f04c800a920 tx=0x7f04c801fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5151400218adf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800a4c0 con 0x7f04bc004d50 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c80ad070 con 0x7f04bc004d50 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d49f3640 1 -- 172.21.15.43:0/1190378754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80a99b0 con 0x7f04bc004d50 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d7c59640 1 -- 172.21.15.43:0/1190378754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04bc004d50 msgr2=0x7f04bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d7c59640 1 --2- 172.21.15.43:0/1190378754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04bc004d50 0x7f04bc0051d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f04c800a920 tx=0x7f04c801fb80 comp rx=0 tx=0).stop 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d61f6640 1 -- 172.21.15.43:0/1190378754 reap_dead start 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d7c59640 1 -- 172.21.15.43:0/1190378754 shutdown_connections 2024-03-30T13:12:10.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d7c59640 1 -- 172.21.15.43:0/1190378754 >> 172.21.15.43:0/1190378754 conn(0x7f04d00b1e90 msgr2=0x7f04d00b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d7c59640 1 -- 172.21.15.43:0/1190378754 shutdown_connections 2024-03-30T13:12:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.971+0000 7f04d7c59640 1 -- 172.21.15.43:0/1190378754 wait complete. 2024-03-30T13:12:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 Processor -- start 2024-03-30T13:12:10.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 -- start start 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d01572d0 0x7f04d01595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015dbf0 0x7f04d00cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d00cb050 0x7f04d0157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04d00cd8e0 con 0x7f04d01572d0 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04d015ca30 con 0x7f04d00cb050 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f04d00ccb40 con 0x7f04d015dbf0 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d01572d0 0x7f04d01595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d01572d0 0x7f04d01595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57690/0 (socket says 172.21.15.43:57690) 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 -- 172.21.15.43:0/846621768 learned_addr learned my addr 172.21.15.43:0/846621768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d51f4640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015dbf0 0x7f04d00cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d61f6640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d00cb050 0x7f04d0157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 -- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015dbf0 msgr2=0x7f04d00cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015dbf0 0x7f04d00cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 -- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d00cb050 msgr2=0x7f04d0157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d00cb050 0x7f04d0157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 -- 172.21.15.43:0/846621768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d015e180 con 0x7f04d01572d0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d59f5640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d01572d0 0x7f04d01595a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f04c00115e0 tx=0x7f04c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04c67fc640 1 -- 172.21.15.43:0/846621768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c000f8f0 con 0x7f04d01572d0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04c67fc640 1 -- 172.21.15.43:0/846621768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c000fa90 con 0x7f04d01572d0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04c67fc640 1 -- 172.21.15.43:0/846621768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c000fe60 con 0x7f04d01572d0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d015e460 con 0x7f04d01572d0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d015ea20 con 0x7f04d01572d0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d0068900 con 0x7f04d01572d0 2024-03-30T13:12:10.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04c67fc640 1 -- 172.21.15.43:0/846621768 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f04c0015460 con 0x7f04d01572d0 2024-03-30T13:12:10.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04c67fc640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f04b4067fc0 0x7f04b406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:10.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.975+0000 7f04c67fc640 1 -- 172.21.15.43:0/846621768 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f04c0087280 con 0x7f04d01572d0 2024-03-30T13:12:10.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.979+0000 7f04d51f4640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f04b4067fc0 0x7f04b406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:10.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.979+0000 7f04d51f4640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f04b4067fc0 0x7f04b406a480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04d00ca400 tx=0x7f04cc007c60 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:10.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:10.983+0000 7f04c67fc640 1 -- 172.21.15.43:0/846621768 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f04c0019030 con 0x7f04d01572d0 2024-03-30T13:12:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.135+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04d0068d80 con 0x7f04d01572d0 2024-03-30T13:12:11.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.135+0000 7f04c67fc640 1 -- 172.21.15.43:0/846621768 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f04c00576c0 con 0x7f04d01572d0 2024-03-30T13:12:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f04b4067fc0 msgr2=0x7f04b406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f04b4067fc0 0x7f04b406a480 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f04d00ca400 tx=0x7f04cc007c60 comp rx=0 tx=0).stop 2024-03-30T13:12:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d01572d0 msgr2=0x7f04d01595a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d01572d0 0x7f04d01595a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f04c00115e0 tx=0x7f04c0011aa0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 shutdown_connections 2024-03-30T13:12:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f04b4067fc0 0x7f04b406a480 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f04d00cb050 0x7f04d0157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f04d015dbf0 0x7f04d00cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 --2- 172.21.15.43:0/846621768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d01572d0 0x7f04d01595a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 >> 172.21.15.43:0/846621768 conn(0x7f04d00b1e90 msgr2=0x7f04d00b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 shutdown_connections 2024-03-30T13:12:11.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.139+0000 7f04d7c59640 1 -- 172.21.15.43:0/846621768 wait complete. 2024-03-30T13:12:11.161 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:11.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.279+0000 7f0c18c68640 1 Processor -- start 2024-03-30T13:12:11.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.279+0000 7f0c18c68640 1 -- start start 2024-03-30T13:12:11.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 0x7f0c1406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c14141890 0x7f0c14141d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c140cfea0 con 0x7f0c141414b0 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0c140d0180 con 0x7f0c14141890 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140d0460 con 0x7f0c140cf9b0 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c1259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0c141414b0 0x7f0c140cf2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33980/0 (socket says 172.21.15.43:33980) 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c1259c640 1 -- 172.21.15.43:0/1000918888 learned_addr learned my addr 172.21.15.43:0/1000918888 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c12d9d640 1 --2- 172.21.15.43:0/1000918888 >> v2:172.21.15.43:3300/0 conn(0x7f0c14141890 0x7f0c14141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c11d9b640 1 --2- 172.21.15.43:0/1000918888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 0x7f0c1406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c12d9d640 1 -- 172.21.15.43:0/1000918888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 msgr2=0x7f0c1406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c12d9d640 1 --2- 172.21.15.43:0/1000918888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 0x7f0c1406b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c12d9d640 1 -- 172.21.15.43:0/1000918888 >> v1:172.21.15.43:6789/0 conn(0x7f0c141414b0 legacy=0x7f0c140cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c12d9d640 1 -- 172.21.15.43:0/1000918888 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c140ccbb0 con 0x7f0c14141890 2024-03-30T13:12:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c12d9d640 1 --2- 172.21.15.43:0/1000918888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c14141d10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009d90 tx=0x7f0c0c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c4d4bf83c5e306 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c1159a640 1 -- 172.21.15.43:0/1000918888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0b2070 con 0x7f0c14141890 2024-03-30T13:12:11.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c1159a640 1 -- 172.21.15.43:0/1000918888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0a5940 con 0x7f0c14141890 2024-03-30T13:12:11.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c1159a640 1 -- 172.21.15.43:0/1000918888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0a5d10 con 0x7f0c14141890 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- 172.21.15.43:0/1000918888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 msgr2=0x7f0c14141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 --2- 172.21.15.43:0/1000918888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c14141d10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009d90 tx=0x7f0c0c01c970 comp rx=0 tx=0).stop 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- 172.21.15.43:0/1000918888 shutdown_connections 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 --2- 172.21.15.43:0/1000918888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c14141d10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 --2- 172.21.15.43:0/1000918888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 0x7f0c1406b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- 172.21.15.43:0/1000918888 >> 172.21.15.43:0/1000918888 conn(0x7f0c140d0c90 msgr2=0x7f0c140d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- 172.21.15.43:0/1000918888 shutdown_connections 2024-03-30T13:12:11.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.283+0000 7f0c18c68640 1 -- 172.21.15.43:0/1000918888 wait complete. 2024-03-30T13:12:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 Processor -- start 2024-03-30T13:12:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 -- start start 2024-03-30T13:12:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 0x7f0c140b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c141414b0 0x7f0c1413de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c141fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140cdc70 con 0x7f0c14141890 2024-03-30T13:12:11.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140cdaf0 con 0x7f0c141414b0 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140cddf0 con 0x7f0c140cf9b0 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c141fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c141fb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52558/0 (socket says 172.21.15.43:52558) 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 -- 172.21.15.43:0/3369610416 learned_addr learned my addr 172.21.15.43:0/3369610416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 -- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 msgr2=0x7f0c140b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 0x7f0c140b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 -- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c141414b0 msgr2=0x7f0c1413de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c141414b0 0x7f0c1413de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 -- 172.21.15.43:0/3369610416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c140b49f0 con 0x7f0c14141890 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c12d9d640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c141fb0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c01b020 tx=0x7f0c0c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c02ffd640 1 -- 172.21.15.43:0/3369610416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0b2040 con 0x7f0c14141890 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c02ffd640 1 -- 172.21.15.43:0/3369610416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0bf070 con 0x7f0c14141890 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c141fba90 con 0x7f0c14141890 2024-03-30T13:12:11.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c02ffd640 1 -- 172.21.15.43:0/3369610416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0b8890 con 0x7f0c14141890 2024-03-30T13:12:11.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c140b3cb0 con 0x7f0c14141890 2024-03-30T13:12:11.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c141fbfb0 con 0x7f0c14141890 2024-03-30T13:12:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.287+0000 7f0c02ffd640 1 -- 172.21.15.43:0/3369610416 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0c0c0b8a30 con 0x7f0c14141890 2024-03-30T13:12:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.291+0000 7f0c02ffd640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0bf4067fc0 0x7f0bf406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.291+0000 7f0c02ffd640 1 -- 172.21.15.43:0/3369610416 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f0c0c121950 con 0x7f0c14141890 2024-03-30T13:12:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.291+0000 7f0c1259c640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0bf4067fc0 0x7f0bf406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.291+0000 7f0c1259c640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0bf4067fc0 0x7f0bf406a480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0c08000f70 tx=0x7f0c08007a90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.299+0000 7f0c02ffd640 1 -- 172.21.15.43:0/3369610416 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0c0c016560 con 0x7f0c14141890 2024-03-30T13:12:11.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c14068900 con 0x7f0c14141890 2024-03-30T13:12:11.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c02ffd640 1 -- 172.21.15.43:0/3369610416 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f0c0c0f1d90 con 0x7f0c14141890 2024-03-30T13:12:11.485 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:11.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0bf4067fc0 msgr2=0x7f0bf406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0bf4067fc0 0x7f0bf406a480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0c08000f70 tx=0x7f0c08007a90 comp rx=0 tx=0).stop 2024-03-30T13:12:11.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 msgr2=0x7f0c141fb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c141fb0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c01b020 tx=0x7f0c0c019710 comp rx=0 tx=0).stop 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 shutdown_connections 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0bf4067fc0 0x7f0bf406a480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c14141890 0x7f0c141fb0f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0c141414b0 0x7f0c1413de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 --2- 172.21.15.43:0/3369610416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0c140cf9b0 0x7f0c140b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 >> 172.21.15.43:0/3369610416 conn(0x7f0c140d0c90 msgr2=0x7f0c1406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 shutdown_connections 2024-03-30T13:12:11.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.479+0000 7f0c18c68640 1 -- 172.21.15.43:0/3369610416 wait complete. 2024-03-30T13:12:11.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:11.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccee224640 1 Processor -- start 2024-03-30T13:12:11.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccee224640 1 -- start start 2024-03-30T13:12:11.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccee224640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcce815e2f0 0x7fcce80b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccee224640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fcce815e6d0 0x7fcce80ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccee224640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcce80b58c0 con 0x7fcce80ad8e0 2024-03-30T13:12:11.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccee224640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fcce80cd060 con 0x7fcce815e2f0 2024-03-30T13:12:11.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccee224640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fcce806b660 con 0x7fcce815e6d0 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcce815e2f0 0x7fcce80b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcce815e2f0 0x7fcce80b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52572/0 (socket says 172.21.15.43:52572) 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 -- 172.21.15.43:0/2566392305 learned_addr learned my addr 172.21.15.43:0/2566392305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fcce77fe640 1 --2- 172.21.15.43:0/2566392305 >> v2:172.21.15.113:3300/0 conn(0x7fcce815e6d0 0x7fcce80ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 -- 172.21.15.43:0/2566392305 >> v2:172.21.15.113:3300/0 conn(0x7fcce815e6d0 msgr2=0x7fcce80ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 --2- 172.21.15.43:0/2566392305 >> v2:172.21.15.113:3300/0 conn(0x7fcce815e6d0 0x7fcce80ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 -- 172.21.15.43:0/2566392305 >> v1:172.21.15.113:6789/0 conn(0x7fcce80ad8e0 legacy=0x7fcce81620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:11.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 -- 172.21.15.43:0/2566392305 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce80b5ba0 con 0x7fcce815e2f0 2024-03-30T13:12:11.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.607+0000 7fccdffff640 1 --2- 172.21.15.43:0/2566392305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e2f0 0x7fcce80b51e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcce8068380 tx=0x7fccd801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9c58bc909ccad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce6ffd640 1 -- 172.21.15.43:0/2566392305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b0070 con 0x7fcce815e2f0 2024-03-30T13:12:11.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce6ffd640 1 -- 172.21.15.43:0/2566392305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd80ab600 con 0x7fcce815e2f0 2024-03-30T13:12:11.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce6ffd640 1 -- 172.21.15.43:0/2566392305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80ab9d0 con 0x7fcce815e2f0 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- 172.21.15.43:0/2566392305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e2f0 msgr2=0x7fcce80b51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 --2- 172.21.15.43:0/2566392305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e2f0 0x7fcce80b51e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcce8068380 tx=0x7fccd801c390 comp rx=0 tx=0).stop 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- 172.21.15.43:0/2566392305 shutdown_connections 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 --2- 172.21.15.43:0/2566392305 >> v2:172.21.15.113:3300/0 conn(0x7fcce815e6d0 0x7fcce80ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 --2- 172.21.15.43:0/2566392305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e2f0 0x7fcce80b51e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- 172.21.15.43:0/2566392305 >> 172.21.15.43:0/2566392305 conn(0x7fcce80d0c90 msgr2=0x7fcce80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- 172.21.15.43:0/2566392305 shutdown_connections 2024-03-30T13:12:11.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- 172.21.15.43:0/2566392305 wait complete. 2024-03-30T13:12:11.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 Processor -- start 2024-03-30T13:12:11.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- start start 2024-03-30T13:12:11.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcce80ad8e0 0x7fcce80b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce815e2f0 0x7fcce80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e6d0 0x7fcce814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcce816b480 con 0x7fcce815e6d0 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcce816b600 con 0x7fcce815e2f0 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcce816b780 con 0x7fcce80ad8e0 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e6d0 0x7fcce814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e6d0 0x7fcce814cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52582/0 (socket says 172.21.15.43:52582) 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 -- 172.21.15.43:0/3025146975 learned_addr learned my addr 172.21.15.43:0/3025146975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccdffff640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcce80ad8e0 0x7fcce80b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce77fe640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce815e2f0 0x7fcce80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 -- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcce80ad8e0 msgr2=0x7fcce80b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcce80ad8e0 0x7fcce80b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 -- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce815e2f0 msgr2=0x7fcce80cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce815e2f0 0x7fcce80cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 -- 172.21.15.43:0/3025146975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce80cd680 con 0x7fcce815e6d0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce77fe640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce815e2f0 0x7fcce80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccdffff640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcce80ad8e0 0x7fcce80b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce7fff640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e6d0 0x7fcce814cc20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b970 tx=0x7fcce000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce4ff9640 1 -- 172.21.15.43:0/3025146975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0019070 con 0x7fcce815e6d0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce4ff9640 1 -- 172.21.15.43:0/3025146975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcce000cb40 con 0x7fcce815e6d0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fcce4ff9640 1 -- 172.21.15.43:0/3025146975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce0014440 con 0x7fcce815e6d0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce80cd960 con 0x7fcce815e6d0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.611+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce814a300 con 0x7fcce815e6d0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.615+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce8068900 con 0x7fcce815e6d0 2024-03-30T13:12:11.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.615+0000 7fcce4ff9640 1 -- 172.21.15.43:0/3025146975 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcce001c070 con 0x7fcce815e6d0 2024-03-30T13:12:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.615+0000 7fcce4ff9640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fccc8067fc0 0x7fccc806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.615+0000 7fcce4ff9640 1 -- 172.21.15.43:0/3025146975 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fcce0086550 con 0x7fcce815e6d0 2024-03-30T13:12:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.615+0000 7fccdffff640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fccc8067fc0 0x7fccc806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.615+0000 7fccdffff640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fccc8067fc0 0x7fccc806a480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ad000 tx=0x7fccd80a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.619+0000 7fcce4ff9640 1 -- 172.21.15.43:0/3025146975 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcce0056900 con 0x7fcce815e6d0 2024-03-30T13:12:11.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.775+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcce814a610 con 0x7fcce815e6d0 2024-03-30T13:12:11.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fcce4ff9640 1 -- 172.21.15.43:0/3025146975 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fcce0056200 con 0x7fcce815e6d0 2024-03-30T13:12:11.785 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fccc8067fc0 msgr2=0x7fccc806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fccc8067fc0 0x7fccc806a480 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fccd80ad000 tx=0x7fccd80a6000 comp rx=0 tx=0).stop 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e6d0 msgr2=0x7fcce814cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e6d0 0x7fcce814cc20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcce000b970 tx=0x7fcce000be30 comp rx=0 tx=0).stop 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 shutdown_connections 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fccc8067fc0 0x7fccc806a480 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce815e6d0 0x7fcce814cc20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcce815e2f0 0x7fcce80cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 --2- 172.21.15.43:0/3025146975 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcce80ad8e0 0x7fcce80b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 >> 172.21.15.43:0/3025146975 conn(0x7fcce80d0c90 msgr2=0x7fcce80b47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:11.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 shutdown_connections 2024-03-30T13:12:11.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.779+0000 7fccee224640 1 -- 172.21.15.43:0/3025146975 wait complete. 2024-03-30T13:12:11.803 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:12:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.915+0000 7f36491c6640 1 Processor -- start 2024-03-30T13:12:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f36491c6640 1 -- start start 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f36491c6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 0x7f3644070640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f36491c6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36440ce4d0 con 0x7f364415f880 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f36491c6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36440cc7f0 con 0x7f364406f090 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f36491c6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f36440b5440 con 0x7f36440b4ff0 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f364415f880 0x7f364406e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33996/0 (socket says 172.21.15.43:33996) 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 -- 172.21.15.43:0/1715099265 learned_addr learned my addr 172.21.15.43:0/1715099265 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f364359e640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 0x7f3644070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209798569 0 0) 0x7f36440ce4d0 con 0x7f364415f880 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f362c003530 con 0x7f364415f880 2024-03-30T13:12:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 344854219 0 0) 0x7f362c003530 con 0x7f364415f880 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 msgr2=0x7f3644070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 0x7f3644070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 >> v1:172.21.15.113:6789/0 conn(0x7f364406f090 legacy=0x7f36440ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440ccad0 con 0x7f364415f880 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f3634001680 con 0x7f364415f880 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 >> v1:172.21.15.43:6789/0 conn(0x7f364415f880 legacy=0x7f364406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0039f0 0x7f362c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c004810 0x7f362c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 0x7f362c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36440b5440 con 0x7f362c0039f0 2024-03-30T13:12:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f362c00bb00 con 0x7f362c004810 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f362c00bd00 con 0x7f36440b4ff0 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0039f0 0x7f362c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f364259c640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c004810 0x7f362c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f364359e640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 0x7f362c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 -- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 msgr2=0x7f362c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 0x7f362c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 -- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c004810 msgr2=0x7f362c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c004810 0x7f362c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f3642d9d640 1 -- 172.21.15.43:0/1715099265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c003530 con 0x7f362c0039f0 2024-03-30T13:12:11.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f364259c640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c004810 0x7f362c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:11.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.919+0000 7f364359e640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b4ff0 0x7f362c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:11.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f3642d9d640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0039f0 0x7f362c003e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f363400c240 tx=0x7f363401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e432f1ba086fc9fa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634007d20 con 0x7f362c0039f0 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36340a9070 con 0x7f362c0039f0 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f3641d9b640 1 -- 172.21.15.43:0/1715099265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634005d40 con 0x7f362c0039f0 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- 172.21.15.43:0/1715099265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0039f0 msgr2=0x7f362c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 --2- 172.21.15.43:0/1715099265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f362c0039f0 0x7f362c003e10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f363400c240 tx=0x7f363401cc80 comp rx=0 tx=0).stop 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364359e640 1 -- 172.21.15.43:0/1715099265 reap_dead start 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- 172.21.15.43:0/1715099265 shutdown_connections 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- 172.21.15.43:0/1715099265 >> 172.21.15.43:0/1715099265 conn(0x7f36440b3cf0 msgr2=0x7f36440b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- 172.21.15.43:0/1715099265 shutdown_connections 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- 172.21.15.43:0/1715099265 wait complete. 2024-03-30T13:12:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 Processor -- start 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- start start 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c0039f0 0x7f36440cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f364414a4d0 0x7f36440b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b2180 0x7f36441f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f36440ccf30 con 0x7f364414a4d0 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f36440ccdb0 con 0x7f362c0039f0 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f36491c6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f36440cf4d0 con 0x7f36440b2180 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f364414a4d0 0x7f36440b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f364414a4d0 0x7f36440b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52598/0 (socket says 172.21.15.43:52598) 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 -- 172.21.15.43:0/1989303828 learned_addr learned my addr 172.21.15.43:0/1989303828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364359e640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b2180 0x7f36441f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f3642d9d640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c0039f0 0x7f36440cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 -- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b2180 msgr2=0x7f36441f5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b2180 0x7f36441f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 -- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c0039f0 msgr2=0x7f36440cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c0039f0 0x7f36440cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.923+0000 7f364259c640 1 -- 172.21.15.43:0/1989303828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36440cac60 con 0x7f364414a4d0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f364259c640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f364414a4d0 0x7f36440b1c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f363800d7a0 tx=0x7f363800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f362b7fe640 1 -- 172.21.15.43:0/1989303828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638005270 con 0x7f364414a4d0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f362b7fe640 1 -- 172.21.15.43:0/1989303828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3638017070 con 0x7f364414a4d0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36440caf40 con 0x7f364414a4d0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f362b7fe640 1 -- 172.21.15.43:0/1989303828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800b650 con 0x7f364414a4d0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36440cb500 con 0x7f364414a4d0 2024-03-30T13:12:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644068900 con 0x7f364414a4d0 2024-03-30T13:12:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f362b7fe640 1 -- 172.21.15.43:0/1989303828 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f363800b7f0 con 0x7f364414a4d0 2024-03-30T13:12:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f362b7fe640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3630067ff0 0x7f363006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f362b7fe640 1 -- 172.21.15.43:0/1989303828 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f3638086da0 con 0x7f364414a4d0 2024-03-30T13:12:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f3642d9d640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3630067ff0 0x7f363006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.927+0000 7f3642d9d640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3630067ff0 0x7f363006a4b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36340ab000 tx=0x7f36340a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:11.931+0000 7f362b7fe640 1 -- 172.21.15.43:0/1989303828 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f36380572c0 con 0x7f364414a4d0 2024-03-30T13:12:12.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.095+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f36440d1790 con 0x7f364414a4d0 2024-03-30T13:12:12.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.095+0000 7f362b7fe640 1 -- 172.21.15.43:0/1989303828 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f3638056bc0 con 0x7f364414a4d0 2024-03-30T13:12:12.104 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:12.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3630067ff0 msgr2=0x7f363006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3630067ff0 0x7f363006a4b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f36340ab000 tx=0x7f36340a7070 comp rx=0 tx=0).stop 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f364414a4d0 msgr2=0x7f36440b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f364414a4d0 0x7f36440b1c40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f363800d7a0 tx=0x7f363800dc60 comp rx=0 tx=0).stop 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 shutdown_connections 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3630067ff0 0x7f363006a4b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f36440b2180 0x7f36441f5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f364414a4d0 0x7f36440b1c40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 --2- 172.21.15.43:0/1989303828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f362c0039f0 0x7f36440cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 >> 172.21.15.43:0/1989303828 conn(0x7f36440b3cf0 msgr2=0x7f364415ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 shutdown_connections 2024-03-30T13:12:12.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.099+0000 7f36491c6640 1 -- 172.21.15.43:0/1989303828 wait complete. 2024-03-30T13:12:12.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:12:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6878e640 1 Processor -- start 2024-03-30T13:12:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6878e640 1 -- start start 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6878e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 0x7f1a601607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6878e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1a600cd850 0x7f1a6006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6878e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a600b0150 con 0x7f1a600cc910 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6878e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1a600cf480 con 0x7f1a600cd850 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6878e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1a600cf760 con 0x7f1a600afd70 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a65d29640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1a600cc910 0x7f1a600cd140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34000/0 (socket says 172.21.15.43:34000) 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a65d29640 1 -- 172.21.15.43:0/524005211 learned_addr learned my addr 172.21.15.43:0/524005211 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a6652a640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 0x7f1a601607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a66d2b640 1 --2- 172.21.15.43:0/524005211 >> v2:172.21.15.113:3300/0 conn(0x7f1a600cd850 0x7f1a6006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3967422910 0 0) 0x7f1a600b0150 con 0x7f1a600cc910 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.231+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a4c003530 con 0x7f1a600cc910 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2157061431 0 0) 0x7f1a4c003530 con 0x7f1a600cc910 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 msgr2=0x7f1a601607d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 0x7f1a601607d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 >> v2:172.21.15.113:3300/0 conn(0x7f1a600cd850 msgr2=0x7f1a6006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 --2- 172.21.15.43:0/524005211 >> v2:172.21.15.113:3300/0 conn(0x7f1a600cd850 0x7f1a6006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a600cfa40 con 0x7f1a600cc910 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f1a5c001680 con 0x7f1a600cc910 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 >> v1:172.21.15.43:6789/0 conn(0x7f1a600cc910 legacy=0x7f1a600cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 0x7f1a4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4c004d50 0x7f1a4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600cf480 con 0x7f1a4c0039c0 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600cf760 con 0x7f1a4c004d50 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c00b9d0 con 0x7f1a600afd70 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6652a640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65d29640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 0x7f1a4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a66d2b640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4c004d50 0x7f1a4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6652a640 1 -- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 msgr2=0x7f1a4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6652a640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a600afd70 0x7f1a4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6652a640 1 -- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4c004d50 msgr2=0x7f1a4c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6652a640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a4c004d50 0x7f1a4c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6652a640 1 -- 172.21.15.43:0/524005211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c003530 con 0x7f1a4c0039c0 2024-03-30T13:12:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6652a640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a4c003de0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009ae0 tx=0x7f1a5001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171299172fb7a09d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500b2070 con 0x7f1a4c0039c0 2024-03-30T13:12:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a500a5630 con 0x7f1a4c0039c0 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a65528640 1 -- 172.21.15.43:0/524005211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500096b0 con 0x7f1a4c0039c0 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6878e640 1 -- 172.21.15.43:0/524005211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 msgr2=0x7f1a4c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6878e640 1 --2- 172.21.15.43:0/524005211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a4c003de0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009ae0 tx=0x7f1a5001ca40 comp rx=0 tx=0).stop 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a66d2b640 1 -- 172.21.15.43:0/524005211 reap_dead start 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6878e640 1 -- 172.21.15.43:0/524005211 shutdown_connections 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6878e640 1 -- 172.21.15.43:0/524005211 >> 172.21.15.43:0/524005211 conn(0x7f1a600b2b30 msgr2=0x7f1a600d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6878e640 1 -- 172.21.15.43:0/524005211 shutdown_connections 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.235+0000 7f1a6878e640 1 -- 172.21.15.43:0/524005211 wait complete. 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 Processor -- start 2024-03-30T13:12:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 -- start start 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a6006f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a6013f350 0x7f1a60143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a60143980 0x7f1a6013fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600cf220 con 0x7f1a4c0039c0 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1a600cf0a0 con 0x7f1a60143980 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1a600d0240 con 0x7f1a6013f350 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a6006f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a6006f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52616/0 (socket says 172.21.15.43:52616) 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 -- 172.21.15.43:0/1526982982 learned_addr learned my addr 172.21.15.43:0/1526982982 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:12.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a65d29640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a6013f350 0x7f1a60143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a66d2b640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a60143980 0x7f1a6013fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 -- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a6013f350 msgr2=0x7f1a60143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a6013f350 0x7f1a60143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 -- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a60143980 msgr2=0x7f1a6013fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a60143980 0x7f1a6013fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 -- 172.21.15.43:0/1526982982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6013e590 con 0x7f1a4c0039c0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a65d29640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a6013f350 0x7f1a60143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6652a640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a6006f270 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1a5001c870 tx=0x7f1a4c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a56ffd640 1 -- 172.21.15.43:0/1526982982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500a5e60 con 0x7f1a4c0039c0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a56ffd640 1 -- 172.21.15.43:0/1526982982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5000f000 con 0x7f1a4c0039c0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6013e870 con 0x7f1a4c0039c0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a56ffd640 1 -- 172.21.15.43:0/1526982982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a500b2040 con 0x7f1a4c0039c0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.239+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6013e260 con 0x7f1a4c0039c0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.243+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a60068890 con 0x7f1a4c0039c0 2024-03-30T13:12:12.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.243+0000 7f1a56ffd640 1 -- 172.21.15.43:0/1526982982 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1a500be030 con 0x7f1a4c0039c0 2024-03-30T13:12:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.243+0000 7f1a56ffd640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a44067fc0 0x7f1a4406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.243+0000 7f1a56ffd640 1 -- 172.21.15.43:0/1526982982 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f1a50120040 con 0x7f1a4c0039c0 2024-03-30T13:12:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.243+0000 7f1a65d29640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a44067fc0 0x7f1a4406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.243+0000 7f1a65d29640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a44067fc0 0x7f1a4406a480 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c001ec0 tx=0x7f1a5c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:12.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.247+0000 7f1a56ffd640 1 -- 172.21.15.43:0/1526982982 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1a500c5020 con 0x7f1a4c0039c0 2024-03-30T13:12:12.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.423+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1a600d1470 con 0x7f1a4c0039c0 2024-03-30T13:12:12.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.423+0000 7f1a56ffd640 1 -- 172.21.15.43:0/1526982982 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f1a50016780 con 0x7f1a4c0039c0 2024-03-30T13:12:12.430 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a44067fc0 msgr2=0x7f1a4406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a44067fc0 0x7f1a4406a480 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c001ec0 tx=0x7f1a5c009040 comp rx=0 tx=0).stop 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 msgr2=0x7f1a6006f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a6006f270 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1a5001c870 tx=0x7f1a4c005840 comp rx=0 tx=0).stop 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 shutdown_connections 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a44067fc0 0x7f1a4406a480 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1a60143980 0x7f1a6013fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1a6013f350 0x7f1a60143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 --2- 172.21.15.43:0/1526982982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a4c0039c0 0x7f1a6006f270 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 >> 172.21.15.43:0/1526982982 conn(0x7f1a600b2b30 msgr2=0x7f1a6015e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 shutdown_connections 2024-03-30T13:12:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.427+0000 7f1a6878e640 1 -- 172.21.15.43:0/1526982982 wait complete. 2024-03-30T13:12:12.450 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi113.front.sepia.ceph.com... 2024-03-30T13:12:12.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:12.450 DEBUG:teuthology.orchestra.run.smithi113:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-30T13:12:12.466 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:12:12.466 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:12:12.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:12.466 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T13:12:12.513 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-30T13:12:12.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f226723f640 1 Processor -- start 2024-03-30T13:12:12.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f226723f640 1 -- start start 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f226723f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 0x7f2260161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f226723f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2260162500 0x7f226015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f226723f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f226006bc40 con 0x7f2260147e80 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f226723f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2260070c50 con 0x7f2260162500 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f226723f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2260070600 con 0x7f226006b660 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2260147e80 0x7f226006c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34014/0 (socket says 172.21.15.43:34014) 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 -- 172.21.15.43:0/4232614590 learned_addr learned my addr 172.21.15.43:0/4232614590 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f22657dc640 1 --2- 172.21.15.43:0/4232614590 >> v2:172.21.15.113:3300/0 conn(0x7f2260162500 0x7f226015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225ffff640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 0x7f2260161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347393667 0 0) 0x7f226006bc40 con 0x7f2260147e80 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2250003530 con 0x7f2260147e80 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4189352507 0 0) 0x7f2250003530 con 0x7f2260147e80 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 msgr2=0x7f2260161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 0x7f2260161fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 >> v2:172.21.15.113:3300/0 conn(0x7f2260162500 msgr2=0x7f226015fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 --2- 172.21.15.43:0/4232614590 >> v2:172.21.15.113:3300/0 conn(0x7f2260162500 0x7f226015fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600708e0 con 0x7f2260147e80 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f2254001680 con 0x7f2260147e80 2024-03-30T13:12:12.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 >> v1:172.21.15.43:6789/0 conn(0x7f2260147e80 legacy=0x7f226006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f2250003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250004810 0x7f2250004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 0x7f22500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2260070c50 con 0x7f22500039c0 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2260070600 con 0x7f2250004810 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f225000b9d0 con 0x7f226006b660 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f2250003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f22657dc640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 0x7f22500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225ffff640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250004810 0x7f2250004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 -- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 msgr2=0x7f22500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f226006b660 0x7f22500051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 -- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250004810 msgr2=0x7f2250004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250004810 0x7f2250004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f2264fdb640 1 -- 172.21.15.43:0/4232614590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2250003530 con 0x7f22500039c0 2024-03-30T13:12:12.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.635+0000 7f225ffff640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2250004810 0x7f2250004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:12.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f2264fdb640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f2250003de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2254001060 tx=0x7f225401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6016d1df75740e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254007d80 con 0x7f22500039c0 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22540a9020 con 0x7f22500039c0 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f225f7fe640 1 -- 172.21.15.43:0/4232614590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254005b00 con 0x7f22500039c0 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 -- 172.21.15.43:0/4232614590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 msgr2=0x7f2250003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 --2- 172.21.15.43:0/4232614590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f2250003de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2254001060 tx=0x7f225401cca0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f22657dc640 1 -- 172.21.15.43:0/4232614590 reap_dead start 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 -- 172.21.15.43:0/4232614590 shutdown_connections 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 -- 172.21.15.43:0/4232614590 >> 172.21.15.43:0/4232614590 conn(0x7f226006e410 msgr2=0x7f226006e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 -- 172.21.15.43:0/4232614590 shutdown_connections 2024-03-30T13:12:12.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 -- 172.21.15.43:0/4232614590 wait complete. 2024-03-30T13:12:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 Processor -- start 2024-03-30T13:12:12.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.639+0000 7f226723f640 1 -- start start 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f22600cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2260152d70 0x7f22600cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22601f76b0 0x7f22601f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2260160c00 con 0x7f22500039c0 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2260160a80 con 0x7f22601f76b0 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2260160d80 con 0x7f2260152d70 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f22600cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f22600cd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52626/0 (socket says 172.21.15.43:52626) 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 -- 172.21.15.43:0/1964224503 learned_addr learned my addr 172.21.15.43:0/1964224503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f22657dc640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22601f76b0 0x7f22601f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f225ffff640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2260152d70 0x7f22600cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 -- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2260152d70 msgr2=0x7f22600cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2260152d70 0x7f22600cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 -- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22601f76b0 msgr2=0x7f22601f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22601f76b0 0x7f22601f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 -- 172.21.15.43:0/1964224503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600aee70 con 0x7f22500039c0 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f2264fdb640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f22600cd330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2254007040 tx=0x7f2250005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f225d7fa640 1 -- 172.21.15.43:0/1964224503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254001cd0 con 0x7f22500039c0 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f225d7fa640 1 -- 172.21.15.43:0/1964224503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22540072d0 con 0x7f22500039c0 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22600af150 con 0x7f22500039c0 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f225d7fa640 1 -- 172.21.15.43:0/1964224503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254007680 con 0x7f22500039c0 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22600af710 con 0x7f22500039c0 2024-03-30T13:12:12.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.643+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2260153150 con 0x7f22500039c0 2024-03-30T13:12:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.647+0000 7f225d7fa640 1 -- 172.21.15.43:0/1964224503 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f22540c1440 con 0x7f22500039c0 2024-03-30T13:12:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.647+0000 7f225d7fa640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2244067fc0 0x7f224406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:12.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.647+0000 7f225d7fa640 1 -- 172.21.15.43:0/1964224503 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f2254120670 con 0x7f22500039c0 2024-03-30T13:12:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.647+0000 7f225ffff640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2244067fc0 0x7f224406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:12.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.647+0000 7f225ffff640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2244067fc0 0x7f224406a480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22600cdad0 tx=0x7f224c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:12.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:12.651+0000 7f225d7fa640 1 -- 172.21.15.43:0/1964224503 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f22540169b0 con 0x7f22500039c0 2024-03-30T13:12:13.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.035+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2260068890 con 0x7f22500039c0 2024-03-30T13:12:13.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.035+0000 7f225d7fa640 1 -- 172.21.15.43:0/1964224503 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f22540f0b90 con 0x7f22500039c0 2024-03-30T13:12:13.044 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:13.044 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:12:13.044 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:12:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2244067fc0 msgr2=0x7f224406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2244067fc0 0x7f224406a480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f22600cdad0 tx=0x7f224c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 msgr2=0x7f22600cd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f22600cd330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2254007040 tx=0x7f2250005840 comp rx=0 tx=0).stop 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 shutdown_connections 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2244067fc0 0x7f224406a480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f22601f76b0 0x7f22601f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2260152d70 0x7f22600cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 --2- 172.21.15.43:0/1964224503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22500039c0 0x7f22600cd330 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 >> 172.21.15.43:0/1964224503 conn(0x7f226006e410 msgr2=0x7f22600b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 shutdown_connections 2024-03-30T13:12:13.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.039+0000 7f226723f640 1 -- 172.21.15.43:0/1964224503 wait complete. 2024-03-30T13:12:13.070 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:13.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.175+0000 7fac8887d640 1 Processor -- start 2024-03-30T13:12:13.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.175+0000 7fac8887d640 1 -- start start 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8887d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac84145d20 0x7fac840cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8887d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fac84070a00 0x7fac8406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8887d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac8406ff20 0x7fac8415fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8887d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fac8406e110 con 0x7fac84145d20 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8887d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fac840cfc00 con 0x7fac84070a00 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8887d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fac84070330 con 0x7fac8406ff20 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac84145d20 0x7fac840cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac84145d20 0x7fac840cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52642/0 (socket says 172.21.15.43:52642) 2024-03-30T13:12:13.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 -- 172.21.15.43:0/3632428241 learned_addr learned my addr 172.21.15.43:0/3632428241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac82d9d640 1 --2- 172.21.15.43:0/3632428241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac8406ff20 0x7fac8415fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac7bfff640 1 --2- 172.21.15.43:0/3632428241 >> v2:172.21.15.113:3300/0 conn(0x7fac84070a00 0x7fac8406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 -- 172.21.15.43:0/3632428241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac8406ff20 msgr2=0x7fac8415fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 --2- 172.21.15.43:0/3632428241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac8406ff20 0x7fac8415fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 -- 172.21.15.43:0/3632428241 >> v2:172.21.15.113:3300/0 conn(0x7fac84070a00 msgr2=0x7fac8406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 --2- 172.21.15.43:0/3632428241 >> v2:172.21.15.113:3300/0 conn(0x7fac84070a00 0x7fac8406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 -- 172.21.15.43:0/3632428241 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac84146100 con 0x7fac84145d20 2024-03-30T13:12:13.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac8259c640 1 --2- 172.21.15.43:0/3632428241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac840cf6c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fac84072540 tx=0x7fac7401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a345722bbb92a2f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac81d9b640 1 -- 172.21.15.43:0/3632428241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac740b0070 con 0x7fac84145d20 2024-03-30T13:12:13.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac81d9b640 1 -- 172.21.15.43:0/3632428241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac740ab600 con 0x7fac84145d20 2024-03-30T13:12:13.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.179+0000 7fac81d9b640 1 -- 172.21.15.43:0/3632428241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac740ab9d0 con 0x7fac84145d20 2024-03-30T13:12:13.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- 172.21.15.43:0/3632428241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 msgr2=0x7fac840cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 --2- 172.21.15.43:0/3632428241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac840cf6c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fac84072540 tx=0x7fac7401c1d0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- 172.21.15.43:0/3632428241 shutdown_connections 2024-03-30T13:12:13.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 --2- 172.21.15.43:0/3632428241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac8406ff20 0x7fac8415fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 --2- 172.21.15.43:0/3632428241 >> v2:172.21.15.113:3300/0 conn(0x7fac84070a00 0x7fac8406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 --2- 172.21.15.43:0/3632428241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac840cf6c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- 172.21.15.43:0/3632428241 >> 172.21.15.43:0/3632428241 conn(0x7fac8406e410 msgr2=0x7fac8406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- 172.21.15.43:0/3632428241 shutdown_connections 2024-03-30T13:12:13.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- 172.21.15.43:0/3632428241 wait complete. 2024-03-30T13:12:13.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 Processor -- start 2024-03-30T13:12:13.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- start start 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac8406ff20 0x7fac841fa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84070a00 0x7fac841fae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac8413f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fac8406b920 con 0x7fac84145d20 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fac840d0720 con 0x7fac8406ff20 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fac8406baa0 con 0x7fac84070a00 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac8413f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac8413f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52652/0 (socket says 172.21.15.43:52652) 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 -- 172.21.15.43:0/1841490584 learned_addr learned my addr 172.21.15.43:0/1841490584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8259c640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac8406ff20 0x7fac841fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac7bfff640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84070a00 0x7fac841fae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 -- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84070a00 msgr2=0x7fac841fae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84070a00 0x7fac841fae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 -- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac8406ff20 msgr2=0x7fac841fa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac8406ff20 0x7fac841fa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 -- 172.21.15.43:0/1841490584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac84143020 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8259c640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac8406ff20 0x7fac841fa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac82d9d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac8413f080 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac7c007e90 tx=0x7fac7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac7affd640 1 -- 172.21.15.43:0/1841490584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c014420 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac7affd640 1 -- 172.21.15.43:0/1841490584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c019070 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac841fb650 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac7affd640 1 -- 172.21.15.43:0/1841490584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c014820 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.183+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac841fbc10 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.187+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac84068890 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.187+0000 7fac7affd640 1 -- 172.21.15.43:0/1841490584 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fac7c0149c0 con 0x7fac84145d20 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.187+0000 7fac7affd640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fac64067fc0 0x7fac6406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.187+0000 7fac7affd640 1 -- 172.21.15.43:0/1841490584 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fac7c0870d0 con 0x7fac84145d20 2024-03-30T13:12:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.187+0000 7fac8259c640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fac64067fc0 0x7fac6406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.187+0000 7fac8259c640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fac64067fc0 0x7fac6406a480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac740ad000 tx=0x7fac74008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.191+0000 7fac7affd640 1 -- 172.21.15.43:0/1841490584 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fac7c057510 con 0x7fac84145d20 2024-03-30T13:12:13.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.351+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac8406db20 con 0x7fac84145d20 2024-03-30T13:12:13.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac7affd640 1 -- 172.21.15.43:0/1841490584 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7fac7c056e10 con 0x7fac84145d20 2024-03-30T13:12:13.362 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:13.362 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:13.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fac64067fc0 msgr2=0x7fac6406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fac64067fc0 0x7fac6406a480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac740ad000 tx=0x7fac74008040 comp rx=0 tx=0).stop 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 msgr2=0x7fac8413f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac8413f080 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac7c007e90 tx=0x7fac7c00e4b0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 shutdown_connections 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fac64067fc0 0x7fac6406a480 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac84145d20 0x7fac8413f080 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fac84070a00 0x7fac841fae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 --2- 172.21.15.43:0/1841490584 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fac8406ff20 0x7fac841fa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 >> 172.21.15.43:0/1841490584 conn(0x7fac8406e410 msgr2=0x7fac840b1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 shutdown_connections 2024-03-30T13:12:13.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.355+0000 7fac8887d640 1 -- 172.21.15.43:0/1841490584 wait complete. 2024-03-30T13:12:13.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:10:53.630387+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:10:54.248332+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-03-30T13:10:57.972722+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-03-30T13:12:13.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:13.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2def667640 1 Processor -- start 2024-03-30T13:12:13.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2def667640 1 -- start start 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2def667640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2de81450f0 0x7f2de815ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2def667640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2de8160440 0x7f2de80cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2def667640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de80d0b50 con 0x7f2de80d02d0 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2def667640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2de80d0e30 con 0x7f2de8160440 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2def667640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2de806b700 con 0x7f2de81450f0 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2decc02640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2de8160440 0x7f2de80cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2dedc04640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2de80d02d0 0x7f2de806f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34016/0 (socket says 172.21.15.43:34016) 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2dedc04640 1 -- 172.21.15.43:0/900325049 learned_addr learned my addr 172.21.15.43:0/900325049 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:13.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2ded403640 1 --2- 172.21.15.43:0/900325049 >> v2:172.21.15.113:3300/0 conn(0x7f2de81450f0 0x7f2de815ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2ddffff640 1 -- 172.21.15.43:0/900325049 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1438397998 0 0) 0x7f2de80d0b50 con 0x7f2de80d02d0 2024-03-30T13:12:13.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2ddffff640 1 -- 172.21.15.43:0/900325049 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dd4003530 con 0x7f2de80d02d0 2024-03-30T13:12:13.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2decc02640 1 -- 172.21.15.43:0/900325049 >> v2:172.21.15.113:3300/0 conn(0x7f2de81450f0 msgr2=0x7f2de815ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:13.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2decc02640 1 --2- 172.21.15.43:0/900325049 >> v2:172.21.15.113:3300/0 conn(0x7f2de81450f0 0x7f2de815ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2decc02640 1 -- 172.21.15.43:0/900325049 >> v1:172.21.15.43:6789/0 conn(0x7f2de80d02d0 legacy=0x7f2de806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.487+0000 7f2decc02640 1 -- 172.21.15.43:0/900325049 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de806b9e0 con 0x7f2de8160440 2024-03-30T13:12:13.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 --2- 172.21.15.43:0/900325049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de8160440 0x7f2de80cfd90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80097c0 tx=0x7f2dd801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f55dee89fe41c8f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2ddffff640 1 -- 172.21.15.43:0/900325049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80b0070 con 0x7f2de8160440 2024-03-30T13:12:13.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2ddffff640 1 -- 172.21.15.43:0/900325049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de81456e0 con 0x7f2de8160440 2024-03-30T13:12:13.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2ddffff640 1 -- 172.21.15.43:0/900325049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80ab820 con 0x7f2de8160440 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- 172.21.15.43:0/900325049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de8160440 msgr2=0x7f2de80cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 --2- 172.21.15.43:0/900325049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de8160440 0x7f2de80cfd90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80097c0 tx=0x7f2dd801c460 comp rx=0 tx=0).stop 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- 172.21.15.43:0/900325049 shutdown_connections 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 --2- 172.21.15.43:0/900325049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de8160440 0x7f2de80cfd90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 --2- 172.21.15.43:0/900325049 >> v2:172.21.15.113:3300/0 conn(0x7f2de81450f0 0x7f2de815ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- 172.21.15.43:0/900325049 >> 172.21.15.43:0/900325049 conn(0x7f2de80d1070 msgr2=0x7f2de80d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- 172.21.15.43:0/900325049 shutdown_connections 2024-03-30T13:12:13.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- 172.21.15.43:0/900325049 wait complete. 2024-03-30T13:12:13.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 Processor -- start 2024-03-30T13:12:13.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- start start 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de80d02d0 0x7f2de813ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de81450f0 0x7f2de813ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de8160440 0x7f2de8140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2de81692a0 con 0x7f2de81450f0 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2de80d0900 con 0x7f2de80d02d0 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2def667640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2de8169420 con 0x7f2de8160440 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de81450f0 0x7f2de813ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de81450f0 0x7f2de813ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52666/0 (socket says 172.21.15.43:52666) 2024-03-30T13:12:13.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 -- 172.21.15.43:0/421491231 learned_addr learned my addr 172.21.15.43:0/421491231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2ded403640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de80d02d0 0x7f2de813ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2dedc04640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de8160440 0x7f2de8140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 -- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de8160440 msgr2=0x7f2de8140760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de8160440 0x7f2de8140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 -- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de80d02d0 msgr2=0x7f2de813ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.491+0000 7f2decc02640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de80d02d0 0x7f2de813ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2decc02640 1 -- 172.21.15.43:0/421491231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de813c720 con 0x7f2de81450f0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2ded403640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de80d02d0 0x7f2de813ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2decc02640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de81450f0 0x7f2de813ce50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8019070 tx=0x7f2dd8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2dddffb640 1 -- 172.21.15.43:0/421491231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80b0040 con 0x7f2de81450f0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2dddffb640 1 -- 172.21.15.43:0/421491231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd80bb070 con 0x7f2de81450f0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de813ca00 con 0x7f2de81450f0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2dddffb640 1 -- 172.21.15.43:0/421491231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80b6410 con 0x7f2de81450f0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de813b250 con 0x7f2de81450f0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de8068900 con 0x7f2de81450f0 2024-03-30T13:12:13.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2dddffb640 1 -- 172.21.15.43:0/421491231 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2dd80b3070 con 0x7f2de81450f0 2024-03-30T13:12:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2dddffb640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2dcc067fc0 0x7f2dcc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2dddffb640 1 -- 172.21.15.43:0/421491231 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f2dd81204e0 con 0x7f2de81450f0 2024-03-30T13:12:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2ded403640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2dcc067fc0 0x7f2dcc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.495+0000 7f2ded403640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2dcc067fc0 0x7f2dcc06a480 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2de81445d0 tx=0x7f2de4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.499+0000 7f2dddffb640 1 -- 172.21.15.43:0/421491231 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2dd8016050 con 0x7f2de81450f0 2024-03-30T13:12:13.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2de806e340 con 0x7f2de81450f0 2024-03-30T13:12:13.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2dddffb640 1 -- 172.21.15.43:0/421491231 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f2dd80f09b0 con 0x7f2de81450f0 2024-03-30T13:12:13.666 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2dcc067fc0 msgr2=0x7f2dcc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2dcc067fc0 0x7f2dcc06a480 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2de81445d0 tx=0x7f2de4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de81450f0 msgr2=0x7f2de813ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de81450f0 0x7f2de813ce50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8019070 tx=0x7f2dd8019200 comp rx=0 tx=0).stop 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 shutdown_connections 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2dcc067fc0 0x7f2dcc06a480 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2de8160440 0x7f2de8140760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de81450f0 0x7f2de813ce50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 --2- 172.21.15.43:0/421491231 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2de80d02d0 0x7f2de813ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 >> 172.21.15.43:0/421491231 conn(0x7f2de80d1070 msgr2=0x7f2de80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:13.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 shutdown_connections 2024-03-30T13:12:13.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.659+0000 7f2def667640 1 -- 172.21.15.43:0/421491231 wait complete. 2024-03-30T13:12:13.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:12:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 Processor -- start 2024-03-30T13:12:13.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 -- start start 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e2f0 0x7f077c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f077c15e6d0 0x7f077c0ae250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f077c0ad8e0 0x7f077c0ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f077c0cd060 con 0x7f077c0ad8e0 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f077c06b660 con 0x7f077c15e6d0 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f0781169640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f077c0b58c0 con 0x7f077c15e2f0 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f077c0ad8e0 0x7f077c0ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f077c0ad8e0 0x7f077c0ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52676/0 (socket says 172.21.15.43:52676) 2024-03-30T13:12:13.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 -- 172.21.15.43:0/2724458922 learned_addr learned my addr 172.21.15.43:0/2724458922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077ad9d640 1 --2- 172.21.15.43:0/2724458922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e2f0 0x7f077c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077a59c640 1 --2- 172.21.15.43:0/2724458922 >> v2:172.21.15.113:3300/0 conn(0x7f077c15e6d0 0x7f077c0ae250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 -- 172.21.15.43:0/2724458922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e2f0 msgr2=0x7f077c0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 --2- 172.21.15.43:0/2724458922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e2f0 0x7f077c0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 -- 172.21.15.43:0/2724458922 >> v2:172.21.15.113:3300/0 conn(0x7f077c15e6d0 msgr2=0x7f077c0ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 --2- 172.21.15.43:0/2724458922 >> v2:172.21.15.113:3300/0 conn(0x7f077c15e6d0 0x7f077c0ae250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.791+0000 7f077b59e640 1 -- 172.21.15.43:0/2724458922 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c0b5ba0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f077b59e640 1 --2- 172.21.15.43:0/2724458922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c0ae790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f077c0b5ad0 tx=0x7f077401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb1c2f558274fb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0779d9b640 1 -- 172.21.15.43:0/2724458922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07740a5720 con 0x7f077c0ad8e0 2024-03-30T13:12:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0779d9b640 1 -- 172.21.15.43:0/2724458922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07740a58c0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0779d9b640 1 -- 172.21.15.43:0/2724458922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07740a5c90 con 0x7f077c0ad8e0 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 -- 172.21.15.43:0/2724458922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 msgr2=0x7f077c0ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 --2- 172.21.15.43:0/2724458922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c0ae790 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f077c0b5ad0 tx=0x7f077401c8a0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 -- 172.21.15.43:0/2724458922 shutdown_connections 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 --2- 172.21.15.43:0/2724458922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c0ae790 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 --2- 172.21.15.43:0/2724458922 >> v2:172.21.15.113:3300/0 conn(0x7f077c15e6d0 0x7f077c0ae250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 --2- 172.21.15.43:0/2724458922 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e2f0 0x7f077c0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 -- 172.21.15.43:0/2724458922 >> 172.21.15.43:0/2724458922 conn(0x7f077c0d0c90 msgr2=0x7f077c0d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 -- 172.21.15.43:0/2724458922 shutdown_connections 2024-03-30T13:12:13.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 -- 172.21.15.43:0/2724458922 wait complete. 2024-03-30T13:12:13.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 Processor -- start 2024-03-30T13:12:13.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.795+0000 7f0781169640 1 -- start start 2024-03-30T13:12:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c14bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077c15e2f0 0x7f077c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e6d0 0x7f077c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1628f0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f077c162a70 con 0x7f077c15e2f0 2024-03-30T13:12:13.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f077c162bf0 con 0x7f077c15e6d0 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c14bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c14bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52684/0 (socket says 172.21.15.43:52684) 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 -- 172.21.15.43:0/115389149 learned_addr learned my addr 172.21.15.43:0/115389149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077a59c640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077c15e2f0 0x7f077c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077b59e640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e6d0 0x7f077c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 -- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e6d0 msgr2=0x7f077c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e6d0 0x7f077c15c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 -- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077c15e2f0 msgr2=0x7f077c15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077c15e2f0 0x7f077c15bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 -- 172.21.15.43:0/115389149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c15d2d0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077b59e640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e6d0 0x7f077c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077ad9d640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c14bfc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d800 tx=0x7f076c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f07637fe640 1 -- 172.21.15.43:0/115389149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c018020 con 0x7f077c0ad8e0 2024-03-30T13:12:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f07637fe640 1 -- 172.21.15.43:0/115389149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f076c0096c0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c156ec0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f07637fe640 1 -- 172.21.15.43:0/115389149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c009ab0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c157480 con 0x7f077c0ad8e0 2024-03-30T13:12:13.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077c068900 con 0x7f077c0ad8e0 2024-03-30T13:12:13.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f07637fe640 1 -- 172.21.15.43:0/115389149 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f076c01b020 con 0x7f077c0ad8e0 2024-03-30T13:12:13.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f07637fe640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0768067fc0 0x7f076806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:13.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f07637fe640 1 -- 172.21.15.43:0/115389149 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f076c0873a0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.799+0000 7f077a59c640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0768067fc0 0x7f076806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:13.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.803+0000 7f077a59c640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0768067fc0 0x7f076806a480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f077c06c100 tx=0x7f0770008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:13.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.807+0000 7f07637fe640 1 -- 172.21.15.43:0/115389149 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f076c022020 con 0x7f077c0ad8e0 2024-03-30T13:12:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f077c070680 con 0x7f077c0ad8e0 2024-03-30T13:12:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f07637fe640 1 -- 172.21.15.43:0/115389149 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f076c0578c0 con 0x7f077c0ad8e0 2024-03-30T13:12:13.974 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0768067fc0 msgr2=0x7f076806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0768067fc0 0x7f076806a480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f077c06c100 tx=0x7f0770008040 comp rx=0 tx=0).stop 2024-03-30T13:12:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 msgr2=0x7f077c14bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c14bfc0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f076c00d800 tx=0x7f076c00dcc0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 shutdown_connections 2024-03-30T13:12:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0768067fc0 0x7f076806a480 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f077c15e6d0 0x7f077c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f077c15e2f0 0x7f077c15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 --2- 172.21.15.43:0/115389149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f077c0ad8e0 0x7f077c14bfc0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 >> 172.21.15.43:0/115389149 conn(0x7f077c0d0c90 msgr2=0x7f077c0b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 shutdown_connections 2024-03-30T13:12:13.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:13.967+0000 7f0781169640 1 -- 172.21.15.43:0/115389149 wait complete. 2024-03-30T13:12:13.993 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:12:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14e1ffa640 1 Processor -- start 2024-03-30T13:12:14.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14e1ffa640 1 -- start start 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14e1ffa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0b9e30 0x7f14dc0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14e1ffa640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f14dc0cf4f0 0x7f14dc0d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14e1ffa640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14dc06c850 con 0x7f14dc15e4c0 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14e1ffa640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f14dc0ba290 con 0x7f14dc0cf4f0 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14e1ffa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc15e8a0 con 0x7f14dc0b9e30 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14db7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f14dc15e4c0 0x7f14dc06c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34026/0 (socket says 172.21.15.43:34026) 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14db7fe640 1 -- 172.21.15.43:0/1157160415 learned_addr learned my addr 172.21.15.43:0/1157160415 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14dbfff640 1 --2- 172.21.15.43:0/1157160415 >> v2:172.21.15.43:3300/0 conn(0x7f14dc0cf4f0 0x7f14dc0d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14daffd640 1 --2- 172.21.15.43:0/1157160415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0b9e30 0x7f14dc0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14da7fc640 1 -- 172.21.15.43:0/1157160415 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965790597 0 0) 0x7f14dc06c850 con 0x7f14dc15e4c0 2024-03-30T13:12:14.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14da7fc640 1 -- 172.21.15.43:0/1157160415 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14c4003530 con 0x7f14dc15e4c0 2024-03-30T13:12:14.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14dbfff640 1 -- 172.21.15.43:0/1157160415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0b9e30 msgr2=0x7f14dc0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:14.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14dbfff640 1 --2- 172.21.15.43:0/1157160415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0b9e30 0x7f14dc0cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14dbfff640 1 -- 172.21.15.43:0/1157160415 >> v1:172.21.15.43:6789/0 conn(0x7f14dc15e4c0 legacy=0x7f14dc06c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.107+0000 7f14dbfff640 1 -- 172.21.15.43:0/1157160415 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc06cb30 con 0x7f14dc0cf4f0 2024-03-30T13:12:14.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14dbfff640 1 --2- 172.21.15.43:0/1157160415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc0cf4f0 0x7f14dc0d2d70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009d90 tx=0x7f14d401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21dee282d8909a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14da7fc640 1 -- 172.21.15.43:0/1157160415 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40b2070 con 0x7f14dc0cf4f0 2024-03-30T13:12:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14da7fc640 1 -- 172.21.15.43:0/1157160415 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d40a5830 con 0x7f14dc0cf4f0 2024-03-30T13:12:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14da7fc640 1 -- 172.21.15.43:0/1157160415 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40a5c00 con 0x7f14dc0cf4f0 2024-03-30T13:12:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 -- 172.21.15.43:0/1157160415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc0cf4f0 msgr2=0x7f14dc0d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/1157160415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc0cf4f0 0x7f14dc0d2d70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009d90 tx=0x7f14d401c970 comp rx=0 tx=0).stop 2024-03-30T13:12:14.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 -- 172.21.15.43:0/1157160415 shutdown_connections 2024-03-30T13:12:14.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/1157160415 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc0cf4f0 0x7f14dc0d2d70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/1157160415 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0b9e30 0x7f14dc0cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 -- 172.21.15.43:0/1157160415 >> 172.21.15.43:0/1157160415 conn(0x7f14dc0b6410 msgr2=0x7f14dc0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:14.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 -- 172.21.15.43:0/1157160415 shutdown_connections 2024-03-30T13:12:14.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.111+0000 7f14e1ffa640 1 -- 172.21.15.43:0/1157160415 wait complete. 2024-03-30T13:12:14.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 Processor -- start 2024-03-30T13:12:14.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 -- start start 2024-03-30T13:12:14.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc0b9e30 0x7f14dc1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0cf4f0 0x7f14dc0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc15e4c0 0x7f14dc14e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0d3e90 con 0x7f14dc15e4c0 2024-03-30T13:12:14.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0d3d10 con 0x7f14dc0b9e30 2024-03-30T13:12:14.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc0d4010 con 0x7f14dc0cf4f0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc15e4c0 0x7f14dc14e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc15e4c0 0x7f14dc14e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52692/0 (socket says 172.21.15.43:52692) 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 -- 172.21.15.43:0/3642567609 learned_addr learned my addr 172.21.15.43:0/3642567609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14db7fe640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc0b9e30 0x7f14dc1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14daffd640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0cf4f0 0x7f14dc0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 -- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0cf4f0 msgr2=0x7f14dc0c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0cf4f0 0x7f14dc0c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 -- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc0b9e30 msgr2=0x7f14dc1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc0b9e30 0x7f14dc1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 -- 172.21.15.43:0/3642567609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc15c380 con 0x7f14dc15e4c0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14dbfff640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc15e4c0 0x7f14dc14e060 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009d90 tx=0x7f14d4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14c3fff640 1 -- 172.21.15.43:0/3642567609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40bd020 con 0x7f14dc15e4c0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14c3fff640 1 -- 172.21.15.43:0/3642567609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d40ad7f0 con 0x7f14dc15e4c0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc15c660 con 0x7f14dc15e4c0 2024-03-30T13:12:14.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14c3fff640 1 -- 172.21.15.43:0/3642567609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d40b2040 con 0x7f14dc15e4c0 2024-03-30T13:12:14.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc15cc20 con 0x7f14dc15e4c0 2024-03-30T13:12:14.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.115+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc068900 con 0x7f14dc15e4c0 2024-03-30T13:12:14.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.119+0000 7f14c3fff640 1 -- 172.21.15.43:0/3642567609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f14d40c0020 con 0x7f14dc15e4c0 2024-03-30T13:12:14.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.119+0000 7f14c3fff640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f14c8067fc0 0x7f14c806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.119+0000 7f14c3fff640 1 -- 172.21.15.43:0/3642567609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f14d4121c90 con 0x7f14dc15e4c0 2024-03-30T13:12:14.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.119+0000 7f14db7fe640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f14c8067fc0 0x7f14c806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.119+0000 7f14db7fe640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f14c8067fc0 0x7f14c806a480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005890 tx=0x7f14d0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.123+0000 7f14c3fff640 1 -- 172.21.15.43:0/3642567609 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f14d40c7030 con 0x7f14dc15e4c0 2024-03-30T13:12:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.279+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14dc154700 con 0x7f14dc15e4c0 2024-03-30T13:12:14.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.279+0000 7f14c3fff640 1 -- 172.21.15.43:0/3642567609 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f14d4016560 con 0x7f14dc15e4c0 2024-03-30T13:12:14.288 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:12:14.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f14c8067fc0 msgr2=0x7f14c806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f14c8067fc0 0x7f14c806a480 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005890 tx=0x7f14d0009040 comp rx=0 tx=0).stop 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc15e4c0 msgr2=0x7f14dc14e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc15e4c0 0x7f14dc14e060 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f14d4009d90 tx=0x7f14d4019710 comp rx=0 tx=0).stop 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 shutdown_connections 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f14c8067fc0 0x7f14c806a480 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14dc15e4c0 0x7f14dc14e060 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f14dc0cf4f0 0x7f14dc0c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 --2- 172.21.15.43:0/3642567609 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f14dc0b9e30 0x7f14dc1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 >> 172.21.15.43:0/3642567609 conn(0x7f14dc0b6410 msgr2=0x7f14dc0b8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 shutdown_connections 2024-03-30T13:12:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.283+0000 7f14e1ffa640 1 -- 172.21.15.43:0/3642567609 wait complete. 2024-03-30T13:12:14.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-03-30T13:12:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f678ae3e640 1 Processor -- start 2024-03-30T13:12:14.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f678ae3e640 1 -- start start 2024-03-30T13:12:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f678ae3e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f678415c160 0x7f678406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f678ae3e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67840cd1b0 0x7f6784160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f678ae3e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6784160c80 con 0x7f678415fe20 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f678ae3e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f67841607d0 con 0x7f678415c160 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f678ae3e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f67840cd680 con 0x7f67840cd1b0 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f6783fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f678415fe20 0x7f67840ccaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34042/0 (socket says 172.21.15.43:34042) 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f6783fff640 1 -- 172.21.15.43:0/2841812032 learned_addr learned my addr 172.21.15.43:0/2841812032 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f6788bda640 1 --2- 172.21.15.43:0/2841812032 >> v2:172.21.15.43:3300/0 conn(0x7f678415c160 0x7f678406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f67837fe640 1 -- 172.21.15.43:0/2841812032 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2990783222 0 0) 0x7f6784160c80 con 0x7f678415fe20 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f67837fe640 1 -- 172.21.15.43:0/2841812032 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6770003530 con 0x7f678415fe20 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f67893db640 1 --2- 172.21.15.43:0/2841812032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67840cd1b0 0x7f6784160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f6788bda640 1 -- 172.21.15.43:0/2841812032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67840cd1b0 msgr2=0x7f6784160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f6788bda640 1 --2- 172.21.15.43:0/2841812032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67840cd1b0 0x7f6784160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f6788bda640 1 -- 172.21.15.43:0/2841812032 >> v1:172.21.15.43:6789/0 conn(0x7f678415fe20 legacy=0x7f67840ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f67837fe640 1 -- 172.21.15.43:0/2841812032 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3296952844 0 0) 0x7f6770003530 con 0x7f678415fe20 2024-03-30T13:12:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.423+0000 7f6788bda640 1 -- 172.21.15.43:0/2841812032 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67840cd960 con 0x7f678415c160 2024-03-30T13:12:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f6788bda640 1 --2- 172.21.15.43:0/2841812032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415c160 0x7f678406b660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097c0 tx=0x7f677c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236d43aca7377d00 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f67837fe640 1 -- 172.21.15.43:0/2841812032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0b0070 con 0x7f678415c160 2024-03-30T13:12:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f67837fe640 1 -- 172.21.15.43:0/2841812032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c0ab400 con 0x7f678415c160 2024-03-30T13:12:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f67837fe640 1 -- 172.21.15.43:0/2841812032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676c001a00 con 0x7f678415c160 2024-03-30T13:12:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- 172.21.15.43:0/2841812032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415c160 msgr2=0x7f678406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 --2- 172.21.15.43:0/2841812032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415c160 0x7f678406b660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f677c0097c0 tx=0x7f677c01c390 comp rx=0 tx=0).stop 2024-03-30T13:12:14.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- 172.21.15.43:0/2841812032 shutdown_connections 2024-03-30T13:12:14.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 --2- 172.21.15.43:0/2841812032 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f67840cd1b0 0x7f6784160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 --2- 172.21.15.43:0/2841812032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415c160 0x7f678406b660 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- 172.21.15.43:0/2841812032 >> 172.21.15.43:0/2841812032 conn(0x7f67840d0c90 msgr2=0x7f67840d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:14.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- 172.21.15.43:0/2841812032 shutdown_connections 2024-03-30T13:12:14.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- 172.21.15.43:0/2841812032 wait complete. 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 Processor -- start 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- start start 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67840cd1b0 0x7f678413b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f678415c160 0x7f678413d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415fe20 0x7f678415d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f67840cf420 con 0x7f678415fe20 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f67840cf2a0 con 0x7f67840cd1b0 2024-03-30T13:12:14.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f678ae3e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f678415d550 con 0x7f678415c160 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f67893db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415fe20 0x7f678415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f67893db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415fe20 0x7f678415d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52700/0 (socket says 172.21.15.43:52700) 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.427+0000 7f67893db640 1 -- 172.21.15.43:0/1172249983 learned_addr learned my addr 172.21.15.43:0/1172249983 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f6783fff640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f678415c160 0x7f678413d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f6788bda640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67840cd1b0 0x7f678413b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67893db640 1 -- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f678415c160 msgr2=0x7f678413d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67893db640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f678415c160 0x7f678413d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67893db640 1 -- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67840cd1b0 msgr2=0x7f678413b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67893db640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67840cd1b0 0x7f678413b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67893db640 1 -- 172.21.15.43:0/1172249983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6784154520 con 0x7f678415fe20 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67893db640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415fe20 0x7f678415d960 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f677400d560 tx=0x7f677400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67817fa640 1 -- 172.21.15.43:0/1172249983 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67740125f0 con 0x7f678415fe20 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67817fa640 1 -- 172.21.15.43:0/1172249983 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6774019070 con 0x7f678415fe20 2024-03-30T13:12:14.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67817fa640 1 -- 172.21.15.43:0/1172249983 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67740129f0 con 0x7f678415fe20 2024-03-30T13:12:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6784154800 con 0x7f678415fe20 2024-03-30T13:12:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6784154dc0 con 0x7f678415fe20 2024-03-30T13:12:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6784068900 con 0x7f678415fe20 2024-03-30T13:12:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67817fa640 1 -- 172.21.15.43:0/1172249983 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6774012b90 con 0x7f678415fe20 2024-03-30T13:12:14.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67817fa640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6764067fc0 0x7f676406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f67817fa640 1 -- 172.21.15.43:0/1172249983 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f6774086c10 con 0x7f678415fe20 2024-03-30T13:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f6788bda640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6764067fc0 0x7f676406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.431+0000 7f6788bda640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6764067fc0 0x7f676406a480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f677c0ad000 tx=0x7f677c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.439+0000 7f67817fa640 1 -- 172.21.15.43:0/1172249983 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f67740570e0 con 0x7f678415fe20 2024-03-30T13:12:14.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.591+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f678406e040 con 0x7f678415fe20 2024-03-30T13:12:14.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f67817fa640 1 -- 172.21.15.43:0/1172249983 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f67740569e0 con 0x7f678415fe20 2024-03-30T13:12:14.601 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6764067fc0 msgr2=0x7f676406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6764067fc0 0x7f676406a480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f677c0ad000 tx=0x7f677c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415fe20 msgr2=0x7f678415d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415fe20 0x7f678415d960 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f677400d560 tx=0x7f677400da20 comp rx=0 tx=0).stop 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 shutdown_connections 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6764067fc0 0x7f676406a480 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f678415fe20 0x7f678415d960 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f678415c160 0x7f678413d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 --2- 172.21.15.43:0/1172249983 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f67840cd1b0 0x7f678413b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 >> 172.21.15.43:0/1172249983 conn(0x7f67840d0c90 msgr2=0x7f6784072410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:14.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 shutdown_connections 2024-03-30T13:12:14.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.595+0000 7f678ae3e640 1 -- 172.21.15.43:0/1172249983 wait complete. 2024-03-30T13:12:14.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.743+0000 7efdea3f3640 1 Processor -- start 2024-03-30T13:12:14.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.743+0000 7efdea3f3640 1 -- start start 2024-03-30T13:12:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efdea3f3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 0x7efde4071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efdea3f3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde4072070 con 0x7efde4162b90 2024-03-30T13:12:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efdea3f3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efde406dc30 con 0x7efde406c670 2024-03-30T13:12:14.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efdea3f3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efde406e280 con 0x7efde4147e10 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efde4162b90 0x7efde406e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34046/0 (socket says 172.21.15.43:34046) 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde37fe640 1 -- 172.21.15.43:0/197353881 learned_addr learned my addr 172.21.15.43:0/197353881 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde3fff640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 0x7efde4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621472176 0 0) 0x7efde4072070 con 0x7efde4162b90 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdcc003530 con 0x7efde4162b90 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2996509005 0 0) 0x7efdcc003530 con 0x7efde4162b90 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 msgr2=0x7efde4071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 0x7efde4071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 >> v1:172.21.15.113:6789/0 conn(0x7efde406c670 legacy=0x7efde40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde406df10 con 0x7efde4162b90 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7efdd8001680 con 0x7efde4162b90 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 >> v1:172.21.15.43:6789/0 conn(0x7efde4162b90 legacy=0x7efde406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 0x7efdcc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdcc003f70 0x7efdcc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdcc004d50 0x7efdcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efde406e280 con 0x7efdcc004d50 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc00bb60 con 0x7efdcc003f70 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efdcc00bd60 con 0x7efde4147e10 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde8990640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdcc004d50 0x7efdcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde3fff640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 0x7efdcc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde37fe640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdcc003f70 0x7efdcc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde8990640 1 -- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 msgr2=0x7efdcc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde8990640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde4147e10 0x7efdcc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde8990640 1 -- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdcc003f70 msgr2=0x7efdcc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde8990640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdcc003f70 0x7efdcc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde8990640 1 -- 172.21.15.43:0/197353881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc003530 con 0x7efdcc004d50 2024-03-30T13:12:14.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.747+0000 7efde37fe640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdcc003f70 0x7efdcc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:14.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdcc004d50 0x7efdcc0051d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efddc001e70 tx=0x7efddc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a416b8c20f1f6395 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc001a90 con 0x7efdcc004d50 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efddc003b80 con 0x7efdcc004d50 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde2ffd640 1 -- 172.21.15.43:0/197353881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8001d50 con 0x7efdcc004d50 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- 172.21.15.43:0/197353881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdcc004d50 msgr2=0x7efdcc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 --2- 172.21.15.43:0/197353881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdcc004d50 0x7efdcc0051d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efddc001e70 tx=0x7efddc01c780 comp rx=0 tx=0).stop 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 -- 172.21.15.43:0/197353881 reap_dead start 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- 172.21.15.43:0/197353881 shutdown_connections 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- 172.21.15.43:0/197353881 >> 172.21.15.43:0/197353881 conn(0x7efde40cfee0 msgr2=0x7efde40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- 172.21.15.43:0/197353881 shutdown_connections 2024-03-30T13:12:14.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- 172.21.15.43:0/197353881 wait complete. 2024-03-30T13:12:14.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 Processor -- start 2024-03-30T13:12:14.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- start start 2024-03-30T13:12:14.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde415d860 0x7efde41458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde415dc40 0x7efde4145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efde415f520 0x7efde415f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efde40b3600 con 0x7efde415f520 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efde4072350 con 0x7efde415d860 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efdea3f3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efde40b3780 con 0x7efde415dc40 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efde415f520 0x7efde415f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efde415f520 0x7efde415f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52722/0 (socket says 172.21.15.43:52722) 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 -- 172.21.15.43:0/83746177 learned_addr learned my addr 172.21.15.43:0/83746177 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde3fff640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde415d860 0x7efde41458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde37fe640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde415dc40 0x7efde4145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 -- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde415dc40 msgr2=0x7efde4145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde415dc40 0x7efde4145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 -- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde415d860 msgr2=0x7efde41458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde415d860 0x7efde41458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.751+0000 7efde8990640 1 -- 172.21.15.43:0/83746177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde41468f0 con 0x7efde415f520 2024-03-30T13:12:14.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde8990640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efde415f520 0x7efde415f9a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efddc01cc60 tx=0x7efdcc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde0ff9640 1 -- 172.21.15.43:0/83746177 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc0b8070 con 0x7efde415f520 2024-03-30T13:12:14.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde0ff9640 1 -- 172.21.15.43:0/83746177 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efddc0012e0 con 0x7efde415f520 2024-03-30T13:12:14.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde0ff9640 1 -- 172.21.15.43:0/83746177 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efddc001a90 con 0x7efde415f520 2024-03-30T13:12:14.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde4146610 con 0x7efde415f520 2024-03-30T13:12:14.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde415ed20 con 0x7efde415f520 2024-03-30T13:12:14.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde4068890 con 0x7efde415f520 2024-03-30T13:12:14.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde0ff9640 1 -- 172.21.15.43:0/83746177 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efddc006b60 con 0x7efde415f520 2024-03-30T13:12:14.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde0ff9640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdd0067ff0 0x7efdd006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:14.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde0ff9640 1 -- 172.21.15.43:0/83746177 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7efddc11f630 con 0x7efde415f520 2024-03-30T13:12:14.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.755+0000 7efde3fff640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdd0067ff0 0x7efdd006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:14.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.759+0000 7efde3fff640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdd0067ff0 0x7efdd006a4b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efde41f78e0 tx=0x7efdd4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:14.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.759+0000 7efde0ff9640 1 -- 172.21.15.43:0/83746177 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7efddc0ac020 con 0x7efde415f520 2024-03-30T13:12:14.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.959+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efde40679e0 con 0x7efde415f520 2024-03-30T13:12:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.959+0000 7efde0ff9640 1 -- 172.21.15.43:0/83746177 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7efddc016490 con 0x7efde415f520 2024-03-30T13:12:14.967 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:14.967 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:12:14.967 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:12:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdd0067ff0 msgr2=0x7efdd006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdd0067ff0 0x7efdd006a4b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efde41f78e0 tx=0x7efdd4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efde415f520 msgr2=0x7efde415f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efde415f520 0x7efde415f9a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efddc01cc60 tx=0x7efdcc005840 comp rx=0 tx=0).stop 2024-03-30T13:12:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 shutdown_connections 2024-03-30T13:12:14.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdd0067ff0 0x7efdd006a4b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efde415f520 0x7efde415f9a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efde415dc40 0x7efde4145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 --2- 172.21.15.43:0/83746177 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efde415d860 0x7efde41458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 >> 172.21.15.43:0/83746177 conn(0x7efde40cfee0 msgr2=0x7efde4071740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 shutdown_connections 2024-03-30T13:12:14.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:14.963+0000 7efdea3f3640 1 -- 172.21.15.43:0/83746177 wait complete. 2024-03-30T13:12:14.986 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.095+0000 7f13e84ca640 1 Processor -- start 2024-03-30T13:12:15.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.095+0000 7f13e84ca640 1 -- start start 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e84ca640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e00afc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e84ca640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13e00b3b80 0x7f13e0162610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e84ca640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e00ad8e0 con 0x7f13e00b3720 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e84ca640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f13e00b4550 con 0x7f13e00b3b80 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e84ca640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f13e006ba70 con 0x7f13e015e370 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6a67640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13e00b3b80 0x7f13e0162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6a67640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13e00b3b80 0x7f13e0162610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52726/0 (socket says 172.21.15.43:52726) 2024-03-30T13:12:15.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6a67640 1 -- 172.21.15.43:0/4207412776 learned_addr learned my addr 172.21.15.43:0/4207412776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6266640 1 --2- 172.21.15.43:0/4207412776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e00afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6a67640 1 -- 172.21.15.43:0/4207412776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 msgr2=0x7f13e00afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6a67640 1 --2- 172.21.15.43:0/4207412776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e00afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6a67640 1 -- 172.21.15.43:0/4207412776 >> v1:172.21.15.113:6789/0 conn(0x7f13e00b3720 legacy=0x7f13e0161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.099+0000 7f13e6a67640 1 -- 172.21.15.43:0/4207412776 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e015e750 con 0x7f13e00b3b80 2024-03-30T13:12:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e6a67640 1 --2- 172.21.15.43:0/4207412776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3b80 0x7f13e0162610 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009e50 tx=0x7f13d801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5936aeb8c56c5ba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e5264640 1 -- 172.21.15.43:0/4207412776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80a58e0 con 0x7f13e00b3b80 2024-03-30T13:12:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e5264640 1 -- 172.21.15.43:0/4207412776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d80a5a80 con 0x7f13e00b3b80 2024-03-30T13:12:15.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e5264640 1 -- 172.21.15.43:0/4207412776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d80a5e50 con 0x7f13e00b3b80 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 -- 172.21.15.43:0/4207412776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3b80 msgr2=0x7f13e0162610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 --2- 172.21.15.43:0/4207412776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3b80 0x7f13e0162610 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009e50 tx=0x7f13d801ca10 comp rx=0 tx=0).stop 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 -- 172.21.15.43:0/4207412776 shutdown_connections 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 --2- 172.21.15.43:0/4207412776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3b80 0x7f13e0162610 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 --2- 172.21.15.43:0/4207412776 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e00afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 -- 172.21.15.43:0/4207412776 >> 172.21.15.43:0/4207412776 conn(0x7f13e00b2b30 msgr2=0x7f13e00d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 -- 172.21.15.43:0/4207412776 shutdown_connections 2024-03-30T13:12:15.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.103+0000 7f13e84ca640 1 -- 172.21.15.43:0/4207412776 wait complete. 2024-03-30T13:12:15.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 Processor -- start 2024-03-30T13:12:15.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 -- start start 2024-03-30T13:12:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3720 0x7f13e01610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e00b3b80 0x7f13e0151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e0151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f13e006b700 con 0x7f13e00b3720 2024-03-30T13:12:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f13e006ee60 con 0x7f13e00b3b80 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00b4010 con 0x7f13e015e370 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3720 0x7f13e01610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3720 0x7f13e01610a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52740/0 (socket says 172.21.15.43:52740) 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 -- 172.21.15.43:0/710124305 learned_addr learned my addr 172.21.15.43:0/710124305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e5a65640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e00b3b80 0x7f13e0151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 -- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 msgr2=0x7f13e0151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6a67640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e0151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e0151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 -- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e00b3b80 msgr2=0x7f13e0151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e00b3b80 0x7f13e0151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 -- 172.21.15.43:0/710124305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e014df20 con 0x7f13e00b3720 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e5a65640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e00b3b80 0x7f13e0151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e6266640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3720 0x7f13e01610a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f13d400d5f0 tx=0x7f13d400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13d2ffd640 1 -- 172.21.15.43:0/710124305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40125f0 con 0x7f13e00b3720 2024-03-30T13:12:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13d2ffd640 1 -- 172.21.15.43:0/710124305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d4019070 con 0x7f13e00b3720 2024-03-30T13:12:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e014e200 con 0x7f13e00b3720 2024-03-30T13:12:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13d2ffd640 1 -- 172.21.15.43:0/710124305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d40129d0 con 0x7f13e00b3720 2024-03-30T13:12:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e014e7c0 con 0x7f13e00b3720 2024-03-30T13:12:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.107+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e0068890 con 0x7f13e00b3720 2024-03-30T13:12:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.111+0000 7f13d2ffd640 1 -- 172.21.15.43:0/710124305 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f13d4012b70 con 0x7f13e00b3720 2024-03-30T13:12:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.111+0000 7f13d2ffd640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f13c4067fc0 0x7f13c406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.111+0000 7f13d2ffd640 1 -- 172.21.15.43:0/710124305 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f13d4086cb0 con 0x7f13e00b3720 2024-03-30T13:12:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.111+0000 7f13e5a65640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f13c4067fc0 0x7f13c406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.111+0000 7f13e5a65640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f13c4067fc0 0x7f13c406a480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13e01529e0 tx=0x7f13dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.115+0000 7f13d2ffd640 1 -- 172.21.15.43:0/710124305 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f13d4057180 con 0x7f13e00b3720 2024-03-30T13:12:15.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.279+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13e00d1470 con 0x7f13e00b3720 2024-03-30T13:12:15.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.279+0000 7f13d2ffd640 1 -- 172.21.15.43:0/710124305 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7f13d4056a80 con 0x7f13e00b3720 2024-03-30T13:12:15.286 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:15.286 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f13c4067fc0 msgr2=0x7f13c406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f13c4067fc0 0x7f13c406a480 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f13e01529e0 tx=0x7f13dc009040 comp rx=0 tx=0).stop 2024-03-30T13:12:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3720 msgr2=0x7f13e01610a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3720 0x7f13e01610a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f13d400d5f0 tx=0x7f13d400dab0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 shutdown_connections 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f13c4067fc0 0x7f13c406a480 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f13e015e370 0x7f13e0151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13e00b3b80 0x7f13e0151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 --2- 172.21.15.43:0/710124305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e00b3720 0x7f13e01610a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 >> 172.21.15.43:0/710124305 conn(0x7f13e00b2b30 msgr2=0x7f13e00adbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 shutdown_connections 2024-03-30T13:12:15.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.283+0000 7f13e84ca640 1 -- 172.21.15.43:0/710124305 wait complete. 2024-03-30T13:12:15.314 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T13:12:15.314 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe18b1e640 1 Processor -- start 2024-03-30T13:12:15.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe18b1e640 1 -- start start 2024-03-30T13:12:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe18b1e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbe1415e600 0x7fbe140b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe18b1e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe140d1100 0x7fbe140d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe18b1e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe140d3e60 con 0x7fbe140b8160 2024-03-30T13:12:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe18b1e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbe140b7450 con 0x7fbe140d1100 2024-03-30T13:12:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe18b1e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbe140d1aa0 con 0x7fbe1415e600 2024-03-30T13:12:15.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe11d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe140d1100 0x7fbe140d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe11d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe140d1100 0x7fbe140d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52746/0 (socket says 172.21.15.43:52746) 2024-03-30T13:12:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe11d9b640 1 -- 172.21.15.43:0/3706840054 learned_addr learned my addr 172.21.15.43:0/3706840054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe1259c640 1 --2- 172.21.15.43:0/3706840054 >> v2:172.21.15.113:3300/0 conn(0x7fbe1415e600 0x7fbe140b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe11d9b640 1 -- 172.21.15.43:0/3706840054 >> v2:172.21.15.113:3300/0 conn(0x7fbe1415e600 msgr2=0x7fbe140b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe11d9b640 1 --2- 172.21.15.43:0/3706840054 >> v2:172.21.15.113:3300/0 conn(0x7fbe1415e600 0x7fbe140b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe11d9b640 1 -- 172.21.15.43:0/3706840054 >> v1:172.21.15.113:6789/0 conn(0x7fbe140b8160 legacy=0x7fbe140b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:15.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.423+0000 7fbe11d9b640 1 -- 172.21.15.43:0/3706840054 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe140b7680 con 0x7fbe140d1100 2024-03-30T13:12:15.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe11d9b640 1 --2- 172.21.15.43:0/3706840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe140d1100 0x7fbe140d1560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbe140b53e0 tx=0x7fbe0801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e4918b828ec59b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe1159a640 1 -- 172.21.15.43:0/3706840054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe080b1070 con 0x7fbe140d1100 2024-03-30T13:12:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe1159a640 1 -- 172.21.15.43:0/3706840054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe080ac630 con 0x7fbe140d1100 2024-03-30T13:12:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe1159a640 1 -- 172.21.15.43:0/3706840054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe080a8050 con 0x7fbe140d1100 2024-03-30T13:12:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- 172.21.15.43:0/3706840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe140d1100 msgr2=0x7fbe140d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/3706840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe140d1100 0x7fbe140d1560 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbe140b53e0 tx=0x7fbe0801c4e0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- 172.21.15.43:0/3706840054 shutdown_connections 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/3706840054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe140d1100 0x7fbe140d1560 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/3706840054 >> v2:172.21.15.113:3300/0 conn(0x7fbe1415e600 0x7fbe140b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- 172.21.15.43:0/3706840054 >> 172.21.15.43:0/3706840054 conn(0x7fbe140b6230 msgr2=0x7fbe140b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- 172.21.15.43:0/3706840054 shutdown_connections 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- 172.21.15.43:0/3706840054 wait complete. 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 Processor -- start 2024-03-30T13:12:15.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- start start 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe140b8160 0x7fbe14150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe140d1100 0x7fbe141fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1415e600 0x7fbe140c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1416b150 con 0x7fbe1415e600 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1416afd0 con 0x7fbe140d1100 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.427+0000 7fbe18b1e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbe1416b2d0 con 0x7fbe140b8160 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1415e600 0x7fbe140c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1415e600 0x7fbe140c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52760/0 (socket says 172.21.15.43:52760) 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 -- 172.21.15.43:0/2134694391 learned_addr learned my addr 172.21.15.43:0/2134694391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe11d9b640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe140d1100 0x7fbe141fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe1259c640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe140b8160 0x7fbe14150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 -- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe140b8160 msgr2=0x7fbe14150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe140b8160 0x7fbe14150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 -- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe140d1100 msgr2=0x7fbe141fb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe140d1100 0x7fbe141fb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 -- 172.21.15.43:0/2134694391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe140d2380 con 0x7fbe1415e600 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe1259c640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe140b8160 0x7fbe14150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe12d9d640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1415e600 0x7fbe140c88b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00b970 tx=0x7fbe0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe02ffd640 1 -- 172.21.15.43:0/2134694391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c019070 con 0x7fbe1415e600 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe02ffd640 1 -- 172.21.15.43:0/2134694391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe0c00cb40 con 0x7fbe1415e600 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141fbf20 con 0x7fbe1415e600 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe02ffd640 1 -- 172.21.15.43:0/2134694391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c014440 con 0x7fbe1415e600 2024-03-30T13:12:15.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe140c8df0 con 0x7fbe1415e600 2024-03-30T13:12:15.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe141562e0 con 0x7fbe1415e600 2024-03-30T13:12:15.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe02ffd640 1 -- 172.21.15.43:0/2134694391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbe0c01c070 con 0x7fbe1415e600 2024-03-30T13:12:15.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe02ffd640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbdf4067fc0 0x7fbdf406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe02ffd640 1 -- 172.21.15.43:0/2134694391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fbe0c086540 con 0x7fbe1415e600 2024-03-30T13:12:15.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe1259c640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbdf4067fc0 0x7fbdf406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.431+0000 7fbe1259c640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbdf4067fc0 0x7fbdf406a480 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe040097a0 tx=0x7fbe04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.435+0000 7fbe02ffd640 1 -- 172.21.15.43:0/2134694391 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbe0c0568f0 con 0x7fbe1415e600 2024-03-30T13:12:15.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.799+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe14151140 con 0x7fbe1415e600 2024-03-30T13:12:15.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe02ffd640 1 -- 172.21.15.43:0/2134694391 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fbe0c0561f0 con 0x7fbe1415e600 2024-03-30T13:12:15.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:15.809 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:12:15.810 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbdf4067fc0 msgr2=0x7fbdf406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbdf4067fc0 0x7fbdf406a480 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe040097a0 tx=0x7fbe04008040 comp rx=0 tx=0).stop 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1415e600 msgr2=0x7fbe140c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1415e600 0x7fbe140c88b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00b970 tx=0x7fbe0c00be30 comp rx=0 tx=0).stop 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 shutdown_connections 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbdf4067fc0 0x7fbdf406a480 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe1415e600 0x7fbe140c88b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbe140d1100 0x7fbe141fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 --2- 172.21.15.43:0/2134694391 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbe140b8160 0x7fbe14150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 >> 172.21.15.43:0/2134694391 conn(0x7fbe140b6230 msgr2=0x7fbe1406c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 shutdown_connections 2024-03-30T13:12:15.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.803+0000 7fbe18b1e640 1 -- 172.21.15.43:0/2134694391 wait complete. 2024-03-30T13:12:15.827 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c8284640 1 Processor -- start 2024-03-30T13:12:15.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c8284640 1 -- start start 2024-03-30T13:12:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c8284640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98c00b2ef0 0x7f98c0144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c8284640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f98c015f7a0 0x7f98c006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c8284640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c00b4ba0 con 0x7f98c0144270 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c8284640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f98c0144fa0 con 0x7f98c00b2ef0 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c8284640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f98c00b33f0 con 0x7f98c015f7a0 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c581f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98c00b2ef0 0x7f98c0144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c6020640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f98c0144270 0x7f98c00b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34048/0 (socket says 172.21.15.43:34048) 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c6020640 1 -- 172.21.15.43:0/2759717317 learned_addr learned my addr 172.21.15.43:0/2759717317 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c6821640 1 --2- 172.21.15.43:0/2759717317 >> v2:172.21.15.113:3300/0 conn(0x7f98c015f7a0 0x7f98c006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c581f640 1 -- 172.21.15.43:0/2759717317 >> v2:172.21.15.113:3300/0 conn(0x7f98c015f7a0 msgr2=0x7f98c006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c581f640 1 --2- 172.21.15.43:0/2759717317 >> v2:172.21.15.113:3300/0 conn(0x7f98c015f7a0 0x7f98c006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c581f640 1 -- 172.21.15.43:0/2759717317 >> v1:172.21.15.43:6789/0 conn(0x7f98c0144270 legacy=0x7f98c00b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c501e640 1 -- 172.21.15.43:0/2759717317 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268212625 0 0) 0x7f98c00b4ba0 con 0x7f98c0144270 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c581f640 1 -- 172.21.15.43:0/2759717317 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c00b36d0 con 0x7f98c00b2ef0 2024-03-30T13:12:15.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.951+0000 7f98c6821640 1 --2- 172.21.15.43:0/2759717317 >> v2:172.21.15.113:3300/0 conn(0x7f98c015f7a0 0x7f98c006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:15.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 --2- 172.21.15.43:0/2759717317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c00b2ef0 0x7f98c0144650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f98c00b5430 tx=0x7f98b401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f73c75d168eae1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c501e640 1 -- 172.21.15.43:0/2759717317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40b2070 con 0x7f98c00b2ef0 2024-03-30T13:12:15.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c501e640 1 -- 172.21.15.43:0/2759717317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b40a9070 con 0x7f98c00b2ef0 2024-03-30T13:12:15.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c501e640 1 -- 172.21.15.43:0/2759717317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40ad880 con 0x7f98c00b2ef0 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- 172.21.15.43:0/2759717317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c00b2ef0 msgr2=0x7f98c0144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 --2- 172.21.15.43:0/2759717317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c00b2ef0 0x7f98c0144650 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f98c00b5430 tx=0x7f98b401c450 comp rx=0 tx=0).stop 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- 172.21.15.43:0/2759717317 shutdown_connections 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 --2- 172.21.15.43:0/2759717317 >> v2:172.21.15.113:3300/0 conn(0x7f98c015f7a0 0x7f98c006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 --2- 172.21.15.43:0/2759717317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c00b2ef0 0x7f98c0144650 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- 172.21.15.43:0/2759717317 >> 172.21.15.43:0/2759717317 conn(0x7f98c00cfee0 msgr2=0x7f98c00d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- 172.21.15.43:0/2759717317 shutdown_connections 2024-03-30T13:12:15.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- 172.21.15.43:0/2759717317 wait complete. 2024-03-30T13:12:15.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 Processor -- start 2024-03-30T13:12:15.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- start start 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98c00b2ef0 0x7f98c013e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c0144270 0x7f98c0143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98c015f7a0 0x7f98c0134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f98c00cdd10 con 0x7f98c0144270 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f98c015fbb0 con 0x7f98c00b2ef0 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c8284640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f98c00cde90 con 0x7f98c015f7a0 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c0144270 0x7f98c0143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c0144270 0x7f98c0143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52776/0 (socket says 172.21.15.43:52776) 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 -- 172.21.15.43:0/2554990995 learned_addr learned my addr 172.21.15.43:0/2554990995 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c6020640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98c00b2ef0 0x7f98c013e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c6821640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98c015f7a0 0x7f98c0134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 -- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98c015f7a0 msgr2=0x7f98c0134400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98c015f7a0 0x7f98c0134400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 -- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98c00b2ef0 msgr2=0x7f98c013e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98c00b2ef0 0x7f98c013e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.955+0000 7f98c581f640 1 -- 172.21.15.43:0/2554990995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c013ea70 con 0x7f98c0144270 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98c6020640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98c00b2ef0 0x7f98c013e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98c581f640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c0144270 0x7f98c0143310 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f98b401a080 tx=0x7f98b401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98beffd640 1 -- 172.21.15.43:0/2554990995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40b2050 con 0x7f98c0144270 2024-03-30T13:12:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98beffd640 1 -- 172.21.15.43:0/2554990995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b40be070 con 0x7f98c0144270 2024-03-30T13:12:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98beffd640 1 -- 172.21.15.43:0/2554990995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40b98b0 con 0x7f98c0144270 2024-03-30T13:12:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c015a700 con 0x7f98c0144270 2024-03-30T13:12:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c01362e0 con 0x7f98c0144270 2024-03-30T13:12:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c0068890 con 0x7f98c0144270 2024-03-30T13:12:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98beffd640 1 -- 172.21.15.43:0/2554990995 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f98b40ab070 con 0x7f98c0144270 2024-03-30T13:12:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98beffd640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f98ac067fc0 0x7f98ac06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98beffd640 1 -- 172.21.15.43:0/2554990995 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7f98b4120f80 con 0x7f98c0144270 2024-03-30T13:12:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.959+0000 7f98c6020640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f98ac067fc0 0x7f98ac06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.963+0000 7f98c6020640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f98ac067fc0 0x7f98ac06a480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98b00012f0 tx=0x7f98b0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:15.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:15.963+0000 7f98beffd640 1 -- 172.21.15.43:0/2554990995 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f98b4016070 con 0x7f98c0144270 2024-03-30T13:12:16.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.155+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98c0072030 con 0x7f98c0144270 2024-03-30T13:12:16.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98beffd640 1 -- 172.21.15.43:0/2554990995 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f98b40f1570 con 0x7f98c0144270 2024-03-30T13:12:16.165 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:16.165 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/1439293427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":1439293427},{"type":"v1","addr":"172.21.15.113:6835","nonce":1439293427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/1115636953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1115636953},{"type":"v1","addr":"172.21.15.113:6837","nonce":1115636953}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/1922289566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1922289566},{"type":"v1","addr":"172.21.15.43:6833","nonce":1922289566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:10:55.286025+0000","modified":"2024-03-30T13:11:18.390289+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4239},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4239":{"gid":4239,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/855012184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":855012184},{"type":"v1","addr":"172.21.15.43:6835","nonce":855012184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4239,"qdb_cluster":[4239]},"id":1}]} 2024-03-30T13:12:16.165 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 9 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f98ac067fc0 msgr2=0x7f98ac06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f98ac067fc0 0x7f98ac06a480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98b00012f0 tx=0x7f98b0009040 comp rx=0 tx=0).stop 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c0144270 msgr2=0x7f98c0143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c0144270 0x7f98c0143310 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f98b401a080 tx=0x7f98b401ac90 comp rx=0 tx=0).stop 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 shutdown_connections 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f98ac067fc0 0x7f98ac06a480 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f98c015f7a0 0x7f98c0134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98c0144270 0x7f98c0143310 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 --2- 172.21.15.43:0/2554990995 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f98c00b2ef0 0x7f98c013e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 >> 172.21.15.43:0/2554990995 conn(0x7f98c00cfee0 msgr2=0x7f98c015b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 shutdown_connections 2024-03-30T13:12:16.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.159+0000 7f98c8284640 1 -- 172.21.15.43:0/2554990995 wait complete. 2024-03-30T13:12:16.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:16.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde2377a640 1 Processor -- start 2024-03-30T13:12:16.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde2377a640 1 -- start start 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde2377a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c0b3720 0x7fde1c15f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde2377a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde1c06b660 0x7fde1c06bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde2377a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde1c15f870 con 0x7fde1c06f950 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde2377a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fde1c06dc30 con 0x7fde1c06b660 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde2377a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fde1c06def0 con 0x7fde1c0b3720 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21d17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde1c06b660 0x7fde1c06bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21516640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c0b3720 0x7fde1c15f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21d17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fde1c06b660 0x7fde1c06bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:35708/0 (socket says 172.21.15.43:35708) 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21d17640 1 -- 172.21.15.43:0/3906882734 learned_addr learned my addr 172.21.15.43:0/3906882734 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde20d15640 1 --1- 172.21.15.43:0/3906882734 >> v1:172.21.15.113:6789/0 conn(0x7fde1c06f950 0x7fde1c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:55632/0 (socket says 172.21.15.43:55632) 2024-03-30T13:12:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21d17640 1 -- 172.21.15.43:0/3906882734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c0b3720 msgr2=0x7fde1c15f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21d17640 1 --2- 172.21.15.43:0/3906882734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c0b3720 0x7fde1c15f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21d17640 1 -- 172.21.15.43:0/3906882734 >> v1:172.21.15.113:6789/0 conn(0x7fde1c06f950 legacy=0x7fde1c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.295+0000 7fde21d17640 1 -- 172.21.15.43:0/3906882734 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c06e1d0 con 0x7fde1c06b660 2024-03-30T13:12:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde21d17640 1 --2- 172.21.15.43:0/3906882734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 0x7fde1c06bb10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e50 tx=0x7fde1401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ee65c8f702aff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde13fff640 1 -- 172.21.15.43:0/3906882734 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140a58d0 con 0x7fde1c06b660 2024-03-30T13:12:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde13fff640 1 -- 172.21.15.43:0/3906882734 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde140a5a70 con 0x7fde1c06b660 2024-03-30T13:12:16.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde13fff640 1 -- 172.21.15.43:0/3906882734 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140a5e40 con 0x7fde1c06b660 2024-03-30T13:12:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 -- 172.21.15.43:0/3906882734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 msgr2=0x7fde1c06bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 --2- 172.21.15.43:0/3906882734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 0x7fde1c06bb10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde14009e50 tx=0x7fde1401ca00 comp rx=0 tx=0).stop 2024-03-30T13:12:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 -- 172.21.15.43:0/3906882734 shutdown_connections 2024-03-30T13:12:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 --2- 172.21.15.43:0/3906882734 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 0x7fde1c06bb10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 --2- 172.21.15.43:0/3906882734 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c0b3720 0x7fde1c15f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 -- 172.21.15.43:0/3906882734 >> 172.21.15.43:0/3906882734 conn(0x7fde1c0b2b30 msgr2=0x7fde1c0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 -- 172.21.15.43:0/3906882734 shutdown_connections 2024-03-30T13:12:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.299+0000 7fde2377a640 1 -- 172.21.15.43:0/3906882734 wait complete. 2024-03-30T13:12:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 Processor -- start 2024-03-30T13:12:16.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 -- start start 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 0x7fde1c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c06f950 0x7fde1c150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b3720 0x7fde1c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c0b01c0 con 0x7fde1c0b3720 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fde1c0b0040 con 0x7fde1c06b660 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fde1c06c460 con 0x7fde1c06f950 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b3720 0x7fde1c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b3720 0x7fde1c0cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52780/0 (socket says 172.21.15.43:52780) 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 -- 172.21.15.43:0/3636282364 learned_addr learned my addr 172.21.15.43:0/3636282364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde20d15640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c06f950 0x7fde1c150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21516640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 0x7fde1c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 -- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c06f950 msgr2=0x7fde1c150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c06f950 0x7fde1c150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 -- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 msgr2=0x7fde1c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 0x7fde1c150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 -- 172.21.15.43:0/3636282364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde1c0cd760 con 0x7fde1c0b3720 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde21d17640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b3720 0x7fde1c0cd220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fde1401a630 tx=0x7fde140197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde11ffb640 1 -- 172.21.15.43:0/3636282364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140a58d0 con 0x7fde1c0b3720 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde11ffb640 1 -- 172.21.15.43:0/3636282364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde140be070 con 0x7fde1c0b3720 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde1c151390 con 0x7fde1c0b3720 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde11ffb640 1 -- 172.21.15.43:0/3636282364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde14009510 con 0x7fde1c0b3720 2024-03-30T13:12:16.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde1c14fc20 con 0x7fde1c0b3720 2024-03-30T13:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.303+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde1c068890 con 0x7fde1c0b3720 2024-03-30T13:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.307+0000 7fde11ffb640 1 -- 172.21.15.43:0/3636282364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fde140b8470 con 0x7fde1c0b3720 2024-03-30T13:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.307+0000 7fde11ffb640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fde00067fc0 0x7fde0006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.307+0000 7fde11ffb640 1 -- 172.21.15.43:0/3636282364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fde14122930 con 0x7fde1c0b3720 2024-03-30T13:12:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.307+0000 7fde21516640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fde00067fc0 0x7fde0006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.307+0000 7fde21516640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fde00067fc0 0x7fde0006a480 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fde1c158c80 tx=0x7fde18008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.311+0000 7fde11ffb640 1 -- 172.21.15.43:0/3636282364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fde1c0cd760 con 0x7fde1c0b3720 2024-03-30T13:12:16.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.491+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde1c1561b0 con 0x7fde1c0b3720 2024-03-30T13:12:16.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde11ffb640 1 -- 172.21.15.43:0/3636282364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18589 (secure 0 0 0) 0x7fde14016620 con 0x7fde1c0b3720 2024-03-30T13:12:16.501 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:16.501 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:11:37.119347+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:10:53.630387+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-30T13:10:54.248332+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-03-30T13:10:57.972722+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]},{"pgid":"4.6","mappings":[{"from":7,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fde00067fc0 msgr2=0x7fde0006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fde00067fc0 0x7fde0006a480 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fde1c158c80 tx=0x7fde18008040 comp rx=0 tx=0).stop 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b3720 msgr2=0x7fde1c0cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b3720 0x7fde1c0cd220 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fde1401a630 tx=0x7fde140197a0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 shutdown_connections 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fde00067fc0 0x7fde0006a480 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde1c0b3720 0x7fde1c0cd220 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde1c06f950 0x7fde1c150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 --2- 172.21.15.43:0/3636282364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde1c06b660 0x7fde1c150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 >> 172.21.15.43:0/3636282364 conn(0x7fde1c0b2b30 msgr2=0x7fde1c06f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:16.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 shutdown_connections 2024-03-30T13:12:16.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.495+0000 7fde2377a640 1 -- 172.21.15.43:0/3636282364 wait complete. 2024-03-30T13:12:16.525 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T13:12:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05f2df640 1 Processor -- start 2024-03-30T13:12:16.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05f2df640 1 -- start start 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05f2df640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 0x7fd0580cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05f2df640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd058070a00 0x7fd05806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05f2df640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0580cfc00 con 0x7fd05806ff20 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05f2df640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd05806e110 con 0x7fd058070a00 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05f2df640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd058070350 con 0x7fd058145d20 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd05806ff20 0x7fd05815fde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34054/0 (socket says 172.21.15.43:34054) 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 -- 172.21.15.43:0/3670834940 learned_addr learned my addr 172.21.15.43:0/3670834940 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05c87a640 1 --2- 172.21.15.43:0/3670834940 >> v2:172.21.15.113:3300/0 conn(0x7fd058070a00 0x7fd05806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d07b640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 0x7fd0580cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 169973755 0 0) 0x7fd0500014a0 con 0x7fd05806ff20 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd044003530 con 0x7fd05806ff20 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3352047383 0 0) 0x7fd044003530 con 0x7fd05806ff20 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 msgr2=0x7fd0580cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 0x7fd0580cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 >> v2:172.21.15.113:3300/0 conn(0x7fd058070a00 msgr2=0x7fd05806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 --2- 172.21.15.43:0/3670834940 >> v2:172.21.15.113:3300/0 conn(0x7fd058070a00 0x7fd05806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd058146100 con 0x7fd05806ff20 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fd050001bd0 con 0x7fd05806ff20 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 >> v1:172.21.15.43:6789/0 conn(0x7fd05806ff20 legacy=0x7fd05815fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 0x7fd0440039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044003f70 0x7fd044004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd05806e110 con 0x7fd044004d50 2024-03-30T13:12:16.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd058070350 con 0x7fd044003f70 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd04400ba50 con 0x7fd058145d20 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d07b640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 0x7fd0440039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05c87a640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044003f70 0x7fd044004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 -- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 msgr2=0x7fd0440039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 0x7fd0440039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 -- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044003f70 msgr2=0x7fd044004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044003f70 0x7fd044004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d87c640 1 -- 172.21.15.43:0/3670834940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd044003530 con 0x7fd044004d50 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05d07b640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd058145d20 0x7fd0440039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:16.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.635+0000 7fd05c87a640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd044003f70 0x7fd044004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.639+0000 7fd05d87c640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd05000aa50 tx=0x7fd05001fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db341577840dcc61 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.639+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000a670 con 0x7fd044004d50 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.639+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0500ad070 con 0x7fd044004d50 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.639+0000 7fd04ffff640 1 -- 172.21.15.43:0/3670834940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500a9980 con 0x7fd044004d50 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- 172.21.15.43:0/3670834940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd044004d50 msgr2=0x7fd0440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 --2- 172.21.15.43:0/3670834940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd05000aa50 tx=0x7fd05001fb00 comp rx=0 tx=0).stop 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 -- 172.21.15.43:0/3670834940 reap_dead start 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- 172.21.15.43:0/3670834940 shutdown_connections 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- 172.21.15.43:0/3670834940 >> 172.21.15.43:0/3670834940 conn(0x7fd05806e410 msgr2=0x7fd05806e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- 172.21.15.43:0/3670834940 shutdown_connections 2024-03-30T13:12:16.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- 172.21.15.43:0/3670834940 wait complete. 2024-03-30T13:12:16.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 Processor -- start 2024-03-30T13:12:16.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- start start 2024-03-30T13:12:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd044004d50 0x7fd058158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05815b390 0x7fd05815b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd058141870 0x7fd058141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd0580d0440 con 0x7fd058141870 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd0580d0160 con 0x7fd05815b390 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd0580d0720 con 0x7fd044004d50 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd058141870 0x7fd058141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd058141870 0x7fd058141cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52798/0 (socket says 172.21.15.43:52798) 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 -- 172.21.15.43:0/637092910 learned_addr learned my addr 172.21.15.43:0/637092910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05c87a640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05815b390 0x7fd05815b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d07b640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd044004d50 0x7fd058158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 -- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd044004d50 msgr2=0x7fd058158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd044004d50 0x7fd058158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 -- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05815b390 msgr2=0x7fd05815b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05815b390 0x7fd05815b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 -- 172.21.15.43:0/637092910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0581592b0 con 0x7fd058141870 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05c87a640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05815b390 0x7fd05815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05d87c640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd058141870 0x7fd058141cf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd05000aa50 tx=0x7fd044005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd04dffb640 1 -- 172.21.15.43:0/637092910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000a670 con 0x7fd058141870 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd04dffb640 1 -- 172.21.15.43:0/637092910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd050004d70 con 0x7fd058141870 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0581425d0 con 0x7fd058141870 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd04dffb640 1 -- 172.21.15.43:0/637092910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500a9720 con 0x7fd058141870 2024-03-30T13:12:16.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.643+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd058142b90 con 0x7fd058141870 2024-03-30T13:12:16.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.647+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd058068890 con 0x7fd058141870 2024-03-30T13:12:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.647+0000 7fd04dffb640 1 -- 172.21.15.43:0/637092910 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd0500a98c0 con 0x7fd058141870 2024-03-30T13:12:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.647+0000 7fd04dffb640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd03c067fc0 0x7fd03c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.647+0000 7fd04dffb640 1 -- 172.21.15.43:0/637092910 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6590+0+0 (secure 0 0 0) 0x7fd0501230a0 con 0x7fd058141870 2024-03-30T13:12:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.647+0000 7fd05d07b640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd03c067fc0 0x7fd03c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:16.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.647+0000 7fd05d07b640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd03c067fc0 0x7fd03c06a480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd048009ab0 tx=0x7fd048008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:16.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.651+0000 7fd04dffb640 1 -- 172.21.15.43:0/637092910 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd0580cfc00 con 0x7fd058141870 2024-03-30T13:12:16.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.851+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd0580ca2f0 con 0x7fd058141870 2024-03-30T13:12:16.860 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-03-30T13:12:16.855+0000 7f98346a4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T13:12:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.855+0000 7fd04dffb640 1 -- 172.21.15.43:0/637092910 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fd0500f3450 con 0x7fd058141870 2024-03-30T13:12:16.863 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T13:12:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd03c067fc0 msgr2=0x7fd03c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd03c067fc0 0x7fd03c06a480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd048009ab0 tx=0x7fd048008040 comp rx=0 tx=0).stop 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd058141870 msgr2=0x7fd058141cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd058141870 0x7fd058141cf0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd05000aa50 tx=0x7fd044005840 comp rx=0 tx=0).stop 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 shutdown_connections 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd03c067fc0 0x7fd03c06a480 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd058141870 0x7fd058141cf0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd05815b390 0x7fd05815b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 --2- 172.21.15.43:0/637092910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd044004d50 0x7fd058158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 >> 172.21.15.43:0/637092910 conn(0x7fd05806e410 msgr2=0x7fd05806d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 shutdown_connections 2024-03-30T13:12:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.863+0000 7fd05f2df640 1 -- 172.21.15.43:0/637092910 wait complete. 2024-03-30T13:12:16.871 INFO:tasks.ceph.mds.c.smithi043.stderr: -5279> 2024-03-30T13:11:00.132+0000 7f4242b42640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T13:12:16.888 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T13:12:16.899 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-03-30T13:12:16.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfeb8e640 1 Processor -- start 2024-03-30T13:12:16.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfeb8e640 1 -- start start 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfeb8e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fadf813f140 0x7fadf81606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfeb8e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadf806cd80 0x7fadf80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfeb8e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf806d710 con 0x7fadf815ff40 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfeb8e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fadf806e320 con 0x7fadf806cd80 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfeb8e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fadf806d9f0 con 0x7fadf813f140 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfd12b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fadf806cd80 0x7fadf80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadf7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fadf815ff40 0x7fadf806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34060/0 (socket says 172.21.15.43:34060) 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadf7fff640 1 -- 172.21.15.43:0/3230967965 learned_addr learned my addr 172.21.15.43:0/3230967965 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.991+0000 7fadfc92a640 1 --2- 172.21.15.43:0/3230967965 >> v2:172.21.15.113:3300/0 conn(0x7fadf813f140 0x7fadf81606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:17.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfd12b640 1 -- 172.21.15.43:0/3230967965 >> v2:172.21.15.113:3300/0 conn(0x7fadf813f140 msgr2=0x7fadf81606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfd12b640 1 --2- 172.21.15.43:0/3230967965 >> v2:172.21.15.113:3300/0 conn(0x7fadf813f140 0x7fadf81606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfd12b640 1 -- 172.21.15.43:0/3230967965 >> v1:172.21.15.43:6789/0 conn(0x7fadf815ff40 legacy=0x7fadf806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfd12b640 1 -- 172.21.15.43:0/3230967965 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf806dcd0 con 0x7fadf806cd80 2024-03-30T13:12:17.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadf77fe640 1 -- 172.21.15.43:0/3230967965 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435518413 0 0) 0x7fadf806d710 con 0x7fadf815ff40 2024-03-30T13:12:17.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfd12b640 1 --2- 172.21.15.43:0/3230967965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf80ce940 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fadf80cfe30 tx=0x7fadf001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f8a7086b5b6543 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:17.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadf77fe640 1 -- 172.21.15.43:0/3230967965 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00b2070 con 0x7fadf806cd80 2024-03-30T13:12:17.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadf77fe640 1 -- 172.21.15.43:0/3230967965 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf00a59e0 con 0x7fadf806cd80 2024-03-30T13:12:17.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadf77fe640 1 -- 172.21.15.43:0/3230967965 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00a5db0 con 0x7fadf806cd80 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3230967965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 msgr2=0x7fadf80ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3230967965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf80ce940 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fadf80cfe30 tx=0x7fadf001c900 comp rx=0 tx=0).stop 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3230967965 shutdown_connections 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3230967965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf80ce940 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3230967965 >> v2:172.21.15.113:3300/0 conn(0x7fadf813f140 0x7fadf81606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3230967965 >> 172.21.15.43:0/3230967965 conn(0x7fadf80d00a0 msgr2=0x7fadf80d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3230967965 shutdown_connections 2024-03-30T13:12:17.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.995+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3230967965 wait complete. 2024-03-30T13:12:17.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 Processor -- start 2024-03-30T13:12:17.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 -- start start 2024-03-30T13:12:17.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf813a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:17.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadf813f140 0x7fadf813d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:17.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadf815ff40 0x7fadf813e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:17.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fadf80cf810 con 0x7fadf806cd80 2024-03-30T13:12:17.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fadf80cf690 con 0x7fadf813f140 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fadf80cf990 con 0x7fadf815ff40 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf813a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf813a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52840/0 (socket says 172.21.15.43:52840) 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 -- 172.21.15.43:0/3997704046 learned_addr learned my addr 172.21.15.43:0/3997704046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadf7fff640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadf813f140 0x7fadf813d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfd12b640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadf815ff40 0x7fadf813e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 -- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadf815ff40 msgr2=0x7fadf813e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadf815ff40 0x7fadf813e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 -- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadf813f140 msgr2=0x7fadf813d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadf813f140 0x7fadf813d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 -- 172.21.15.43:0/3997704046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf813c890 con 0x7fadf806cd80 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfd12b640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadf815ff40 0x7fadf813e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:17.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfc92a640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf813a9b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fade800d800 tx=0x7fade800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:17.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadf57fa640 1 -- 172.21.15.43:0/3997704046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade800b6c0 con 0x7fadf806cd80 2024-03-30T13:12:17.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadf57fa640 1 -- 172.21.15.43:0/3997704046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade8019070 con 0x7fadf806cd80 2024-03-30T13:12:17.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadf57fa640 1 -- 172.21.15.43:0/3997704046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade800ba60 con 0x7fadf806cd80 2024-03-30T13:12:17.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf813cb70 con 0x7fadf806cd80 2024-03-30T13:12:17.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf813d130 con 0x7fadf806cd80 2024-03-30T13:12:17.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:16.999+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf80688e0 con 0x7fadf806cd80 2024-03-30T13:12:17.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.003+0000 7fadf57fa640 1 -- 172.21.15.43:0/3997704046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fade800bc00 con 0x7fadf806cd80 2024-03-30T13:12:17.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.003+0000 7fadf57fa640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fade4068040 0x7fade406a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:17.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.003+0000 7fadf57fa640 1 -- 172.21.15.43:0/3997704046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6676+0+0 (secure 0 0 0) 0x7fade8086c10 con 0x7fadf806cd80 2024-03-30T13:12:17.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.003+0000 7fadf7fff640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fade4068040 0x7fade406a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:17.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.003+0000 7fadf7fff640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fade4068040 0x7fade406a500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadf806c010 tx=0x7fadec007d10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.007+0000 7fadf57fa640 1 -- 172.21.15.43:0/3997704046 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fade8011340 con 0x7fadf806cd80 2024-03-30T13:12:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.207+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fadf8072280 con 0x7fadf806cd80 2024-03-30T13:12:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadf57fa640 1 -- 172.21.15.43:0/3997704046 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fade8057160 con 0x7fadf806cd80 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fade4068040 msgr2=0x7fade406a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fade4068040 0x7fade406a500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadf806c010 tx=0x7fadec007d10 comp rx=0 tx=0).stop 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 msgr2=0x7fadf813a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf813a9b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fade800d800 tx=0x7fade800dcc0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 shutdown_connections 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fade4068040 0x7fade406a500 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.859+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fadf815ff40 0x7fadf813e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.863+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fadf813f140 0x7fadf813d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.863+0000 7fadfeb8e640 1 --2- 172.21.15.43:0/3997704046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fadf806cd80 0x7fadf813a9b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.863+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 >> 172.21.15.43:0/3997704046 conn(0x7fadf80d00a0 msgr2=0x7fadf80b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.863+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 shutdown_connections 2024-03-30T13:12:17.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:17.863+0000 7fadfeb8e640 1 -- 172.21.15.43:0/3997704046 wait complete. 2024-03-30T13:12:17.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T13:12:18.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a97020640 1 Processor -- start 2024-03-30T13:12:18.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a97020640 1 -- start start 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a97020640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a900b5550 0x7f2a90160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a97020640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a90073490 0x7f2a9006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a97020640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a9015d090 con 0x7f2a900b5930 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a97020640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9015ce30 con 0x7f2a90073490 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a97020640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2a9015d370 con 0x7f2a900b5550 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a955bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a90073490 0x7f2a9006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a955bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a90073490 0x7f2a9006c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52850/0 (socket says 172.21.15.43:52850) 2024-03-30T13:12:18.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a955bd640 1 -- 172.21.15.43:0/832469700 learned_addr learned my addr 172.21.15.43:0/832469700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:18.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a94dbc640 1 --2- 172.21.15.43:0/832469700 >> v2:172.21.15.113:3300/0 conn(0x7f2a900b5550 0x7f2a90160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:18.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a955bd640 1 -- 172.21.15.43:0/832469700 >> v2:172.21.15.113:3300/0 conn(0x7f2a900b5550 msgr2=0x7f2a90160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:18.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a955bd640 1 --2- 172.21.15.43:0/832469700 >> v2:172.21.15.113:3300/0 conn(0x7f2a900b5550 0x7f2a90160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a955bd640 1 -- 172.21.15.43:0/832469700 >> v1:172.21.15.113:6789/0 conn(0x7f2a900b5930 legacy=0x7f2a9015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:18.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.067+0000 7f2a955bd640 1 -- 172.21.15.43:0/832469700 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9015fe10 con 0x7f2a90073490 2024-03-30T13:12:18.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a955bd640 1 --2- 172.21.15.43:0/832469700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a90073490 0x7f2a9006c670 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a900b29b0 tx=0x7f2a8801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3ef0050b06bcad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:18.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/832469700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a880a5790 con 0x7f2a90073490 2024-03-30T13:12:18.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/832469700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a880a5930 con 0x7f2a90073490 2024-03-30T13:12:18.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/832469700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a880a5d00 con 0x7f2a90073490 2024-03-30T13:12:18.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 -- 172.21.15.43:0/832469700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a90073490 msgr2=0x7f2a9006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:18.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 --2- 172.21.15.43:0/832469700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a90073490 0x7f2a9006c670 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2a900b29b0 tx=0x7f2a8801c9c0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 -- 172.21.15.43:0/832469700 shutdown_connections 2024-03-30T13:12:18.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 --2- 172.21.15.43:0/832469700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a90073490 0x7f2a9006c670 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 --2- 172.21.15.43:0/832469700 >> v2:172.21.15.113:3300/0 conn(0x7f2a900b5550 0x7f2a90160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 -- 172.21.15.43:0/832469700 >> 172.21.15.43:0/832469700 conn(0x7f2a900b1b80 msgr2=0x7f2a900b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:18.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 -- 172.21.15.43:0/832469700 shutdown_connections 2024-03-30T13:12:18.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.071+0000 7f2a97020640 1 -- 172.21.15.43:0/832469700 wait complete. 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 Processor -- start 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 -- start start 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a90073490 0x7f2a900cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900b5550 0x7f2a900c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a900b5930 0x7f2a900c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90169290 con 0x7f2a900b5550 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90169110 con 0x7f2a900b5930 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a90169410 con 0x7f2a90073490 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900b5550 0x7f2a900c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:18.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900b5550 0x7f2a900c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52860/0 (socket says 172.21.15.43:52860) 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 -- 172.21.15.43:0/2904408767 learned_addr learned my addr 172.21.15.43:0/2904408767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a94dbc640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a90073490 0x7f2a900cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a955bd640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a900b5930 0x7f2a900c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 -- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a90073490 msgr2=0x7f2a900cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a90073490 0x7f2a900cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 -- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a900b5930 msgr2=0x7f2a900c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a900b5930 0x7f2a900c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 -- 172.21.15.43:0/2904408767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a90157470 con 0x7f2a900b5550 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8ffff640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900b5550 0x7f2a900c9200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c340 tx=0x7f2a8000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/2904408767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a800052d0 con 0x7f2a900b5550 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/2904408767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a80005470 con 0x7f2a900b5550 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a90157750 con 0x7f2a900b5550 2024-03-30T13:12:18.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/2904408767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80007260 con 0x7f2a900b5550 2024-03-30T13:12:18.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a90157d10 con 0x7f2a900b5550 2024-03-30T13:12:18.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.075+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a90157f10 con 0x7f2a900b5550 2024-03-30T13:12:18.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.079+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/2904408767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2a80007400 con 0x7f2a900b5550 2024-03-30T13:12:18.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.079+0000 7f2a8d7fa640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a70067fc0 0x7f2a7006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:18.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.079+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/2904408767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6676+0+0 (secure 0 0 0) 0x7f2a80086030 con 0x7f2a900b5550 2024-03-30T13:12:18.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.079+0000 7f2a94dbc640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a70067fc0 0x7f2a7006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:18.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.079+0000 7f2a94dbc640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a70067fc0 0x7f2a7006a480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007a60 tx=0x7f2a78008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:18.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.083+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/2904408767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2a90157f10 con 0x7f2a900b5550 2024-03-30T13:12:18.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.243+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2a900687f0 con 0x7f2a900b5550 2024-03-30T13:12:18.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.867+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/2904408767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f2a80055f30 con 0x7f2a900b5550 2024-03-30T13:12:18.874 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-03-30T13:12:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a70067fc0 msgr2=0x7f2a7006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a70067fc0 0x7f2a7006a480 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007a60 tx=0x7f2a78008040 comp rx=0 tx=0).stop 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900b5550 msgr2=0x7f2a900c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900b5550 0x7f2a900c9200 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c340 tx=0x7f2a8000c800 comp rx=0 tx=0).stop 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 shutdown_connections 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a70067fc0 0x7f2a7006a480 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a900b5930 0x7f2a900c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900b5550 0x7f2a900c9200 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 --2- 172.21.15.43:0/2904408767 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a90073490 0x7f2a900cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 >> 172.21.15.43:0/2904408767 conn(0x7f2a900b1b80 msgr2=0x7f2a9015edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 shutdown_connections 2024-03-30T13:12:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:18.879+0000 7f2a97020640 1 -- 172.21.15.43:0/2904408767 wait complete. 2024-03-30T13:12:18.908 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T13:12:19.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.019+0000 7f7e9a63e640 1 Processor -- start 2024-03-30T13:12:19.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.019+0000 7f7e9a63e640 1 -- start start 2024-03-30T13:12:19.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.019+0000 7f7e9a63e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7e940b5550 0x7f7e94160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.019+0000 7f7e9a63e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 0x7f7e9415c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.019+0000 7f7e9a63e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e9415d090 con 0x7f7e94073490 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.019+0000 7f7e9a63e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7e9415d370 con 0x7f7e940b5550 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.019+0000 7f7e9a63e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9415fe10 con 0x7f7e940b5930 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e98bdb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7e94073490 0x7f7e9406c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34062/0 (socket says 172.21.15.43:34062) 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e98bdb640 1 -- 172.21.15.43:0/2377018816 learned_addr learned my addr 172.21.15.43:0/2377018816 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e93fff640 1 --2- 172.21.15.43:0/2377018816 >> v2:172.21.15.113:3300/0 conn(0x7f7e940b5550 0x7f7e94160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787293829 0 0) 0x7f7e9415d090 con 0x7f7e94073490 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 0x7f7e9415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e7c003530 con 0x7f7e94073490 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4211919067 0 0) 0x7f7e7c003530 con 0x7f7e94073490 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 msgr2=0x7f7e9415c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:19.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 0x7f7e9415c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 >> v2:172.21.15.113:3300/0 conn(0x7f7e940b5550 msgr2=0x7f7e94160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 --2- 172.21.15.43:0/2377018816 >> v2:172.21.15.113:3300/0 conn(0x7f7e940b5550 0x7f7e94160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e941600f0 con 0x7f7e94073490 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f7e8c001a00 con 0x7f7e94073490 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 >> v1:172.21.15.43:6789/0 conn(0x7f7e94073490 legacy=0x7f7e9406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e7c0039c0 0x7f7e7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e7c004810 0x7f7e7c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 0x7f7e7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9415d370 con 0x7f7e7c004810 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9415fe10 con 0x7f7e7c0039c0 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7e7c00b9d0 con 0x7f7e940b5930 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e7c004810 0x7f7e7c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e93fff640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e7c0039c0 0x7f7e7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e98bdb640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 0x7f7e7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 -- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 msgr2=0x7f7e7c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:19.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e940b5930 0x7f7e7c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 -- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e7c0039c0 msgr2=0x7f7e7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:19.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e7c0039c0 0x7f7e7c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 -- 172.21.15.43:0/2377018816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7c003530 con 0x7f7e7c004810 2024-03-30T13:12:19.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e937fe640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e7c004810 0x7f7e7c004c90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009ab0 tx=0x7f7e8801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93245b87a35b2e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:19.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880096b0 con 0x7f7e7c004810 2024-03-30T13:12:19.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e880a6460 con 0x7f7e7c004810 2024-03-30T13:12:19.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.023+0000 7f7e92ffd640 1 -- 172.21.15.43:0/2377018816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880a6850 con 0x7f7e7c004810 2024-03-30T13:12:19.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- 172.21.15.43:0/2377018816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e7c004810 msgr2=0x7f7e7c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:19.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 --2- 172.21.15.43:0/2377018816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e7c004810 0x7f7e7c004c90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009ab0 tx=0x7f7e8801ca50 comp rx=0 tx=0).stop 2024-03-30T13:12:19.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 -- 172.21.15.43:0/2377018816 reap_dead start 2024-03-30T13:12:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- 172.21.15.43:0/2377018816 shutdown_connections 2024-03-30T13:12:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- 172.21.15.43:0/2377018816 >> 172.21.15.43:0/2377018816 conn(0x7f7e940b1b80 msgr2=0x7f7e940b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- 172.21.15.43:0/2377018816 shutdown_connections 2024-03-30T13:12:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- 172.21.15.43:0/2377018816 wait complete. 2024-03-30T13:12:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 Processor -- start 2024-03-30T13:12:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- start start 2024-03-30T13:12:19.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e7c004810 0x7f7e9415a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e94154c80 0x7f7e940ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940c9200 0x7f7e940c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94160cd0 con 0x7f7e940c9200 2024-03-30T13:12:19.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7e94160b50 con 0x7f7e94154c80 2024-03-30T13:12:19.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9406cd80 con 0x7f7e7c004810 2024-03-30T13:12:19.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940c9200 0x7f7e940c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940c9200 0x7f7e940c9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52880/0 (socket says 172.21.15.43:52880) 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 -- 172.21.15.43:0/1608168673 learned_addr learned my addr 172.21.15.43:0/1608168673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e93fff640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e7c004810 0x7f7e9415a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e937fe640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e94154c80 0x7f7e940ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 -- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e7c004810 msgr2=0x7f7e9415a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e7c004810 0x7f7e9415a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 -- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e94154c80 msgr2=0x7f7e940ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e94154c80 0x7f7e940ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 -- 172.21.15.43:0/1608168673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e94159360 con 0x7f7e940c9200 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e98bdb640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940c9200 0x7f7e940c9680 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c001340 tx=0x7f7e8c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e90ff9640 1 -- 172.21.15.43:0/1608168673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c010040 con 0x7f7e940c9200 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e90ff9640 1 -- 172.21.15.43:0/1608168673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e8c01a070 con 0x7f7e940c9200 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e94159640 con 0x7f7e940c9200 2024-03-30T13:12:19.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e90ff9640 1 -- 172.21.15.43:0/1608168673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00fba0 con 0x7f7e940c9200 2024-03-30T13:12:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.027+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e94159c00 con 0x7f7e940c9200 2024-03-30T13:12:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.031+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e940c9bc0 con 0x7f7e940c9200 2024-03-30T13:12:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.031+0000 7f7e90ff9640 1 -- 172.21.15.43:0/1608168673 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7e8c00fd40 con 0x7f7e940c9200 2024-03-30T13:12:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.031+0000 7f7e90ff9640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7e80067ff0 0x7f7e8006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:19.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.031+0000 7f7e90ff9640 1 -- 172.21.15.43:0/1608168673 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6238+0+0 (secure 0 0 0) 0x7f7e8c0896e0 con 0x7f7e940c9200 2024-03-30T13:12:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.031+0000 7f7e93fff640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7e80067ff0 0x7f7e8006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.031+0000 7f7e93fff640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7e80067ff0 0x7f7e8006a4b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009ab0 tx=0x7f7e84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:19.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.035+0000 7f7e90ff9640 1 -- 172.21.15.43:0/1608168673 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7e8c059be0 con 0x7f7e940c9200 2024-03-30T13:12:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.195+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7e940687f0 con 0x7f7e940c9200 2024-03-30T13:12:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.875+0000 7f7e90ff9640 1 -- 172.21.15.43:0/1608168673 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7f7e8c0594e0 con 0x7f7e940c9200 2024-03-30T13:12:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-03-30T13:12:19.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.879+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7e80067ff0 msgr2=0x7f7e8006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:19.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.879+0000 7f7e9a63e640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7e80067ff0 0x7f7e8006a4b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009ab0 tx=0x7f7e84008040 comp rx=0 tx=0).stop 2024-03-30T13:12:19.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.879+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940c9200 msgr2=0x7f7e940c9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:19.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.879+0000 7f7e9a63e640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940c9200 0x7f7e940c9680 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c001340 tx=0x7f7e8c011c20 comp rx=0 tx=0).stop 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 shutdown_connections 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7e80067ff0 0x7f7e8006a4b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e940c9200 0x7f7e940c9680 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7e94154c80 0x7f7e940ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 --2- 172.21.15.43:0/1608168673 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7e7c004810 0x7f7e9415a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 >> 172.21.15.43:0/1608168673 conn(0x7f7e940b1b80 msgr2=0x7f7e940ca450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 shutdown_connections 2024-03-30T13:12:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:19.883+0000 7f7e9a63e640 1 -- 172.21.15.43:0/1608168673 wait complete. 2024-03-30T13:12:19.917 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-03-30T13:12:20.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.043+0000 7fc1dc618640 1 Processor -- start 2024-03-30T13:12:20.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.043+0000 7fc1dc618640 1 -- start start 2024-03-30T13:12:20.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.043+0000 7fc1dc618640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1d4160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.043+0000 7fc1dc618640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc1d40b5930 0x7fc1d415c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.043+0000 7fc1dc618640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d415d090 con 0x7fc1d4073490 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dc618640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d415d370 con 0x7fc1d40b5930 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dc618640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d415fe10 con 0x7fc1d40b5550 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc1d4073490 0x7fc1d406c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34066/0 (socket says 172.21.15.43:34066) 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 -- 172.21.15.43:0/1639234390 learned_addr learned my addr 172.21.15.43:0/1639234390 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d9bb3640 1 --2- 172.21.15.43:0/1639234390 >> v2:172.21.15.113:3300/0 conn(0x7fc1d40b5930 0x7fc1d415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1da3b4640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1d4160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107432848 0 0) 0x7fc1d415d090 con 0x7fc1d4073490 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1c0003530 con 0x7fc1d4073490 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1673496986 0 0) 0x7fc1c0003530 con 0x7fc1d4073490 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 msgr2=0x7fc1d4160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1d4160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 >> v2:172.21.15.113:3300/0 conn(0x7fc1d40b5930 msgr2=0x7fc1d415c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 --2- 172.21.15.43:0/1639234390 >> v2:172.21.15.113:3300/0 conn(0x7fc1d40b5930 0x7fc1d415c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d41600f0 con 0x7fc1d4073490 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fc1cc001680 con 0x7fc1d4073490 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1da3b4640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1d4160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 >> v1:172.21.15.43:6789/0 conn(0x7fc1d4073490 legacy=0x7fc1d406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1c00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c0003f00 0x7fc1c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c0004d50 0x7fc1c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d415d370 con 0x7fc1c0004d50 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d415fe10 con 0x7fc1c0003f00 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c000ba00 con 0x7fc1d40b5550 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c0004d50 0x7fc1c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1da3b4640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1c00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d9bb3640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c0003f00 0x7fc1c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 -- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 msgr2=0x7fc1c00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1c00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 -- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c0003f00 msgr2=0x7fc1c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c0003f00 0x7fc1c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 -- 172.21.15.43:0/1639234390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c0003530 con 0x7fc1c0004d50 2024-03-30T13:12:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d9bb3640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1c0003f00 0x7fc1c0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:20.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1da3b4640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d40b5550 0x7fc1c00039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1dabb5640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c0004d50 0x7fc1c00051d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00c240 tx=0x7fc1cc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49c3454a3f22af8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc007d60 con 0x7fc1c0004d50 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1cc0a9020 con 0x7fc1c0004d50 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.047+0000 7fc1d93b2640 1 -- 172.21.15.43:0/1639234390 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0a5bc0 con 0x7fc1c0004d50 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- 172.21.15.43:0/1639234390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c0004d50 msgr2=0x7fc1c00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 --2- 172.21.15.43:0/1639234390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1c0004d50 0x7fc1c00051d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc00c240 tx=0x7fc1cc01ccc0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dabb5640 1 -- 172.21.15.43:0/1639234390 reap_dead start 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- 172.21.15.43:0/1639234390 shutdown_connections 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- 172.21.15.43:0/1639234390 >> 172.21.15.43:0/1639234390 conn(0x7fc1d40b1b80 msgr2=0x7fc1d40b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:20.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- 172.21.15.43:0/1639234390 shutdown_connections 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- 172.21.15.43:0/1639234390 wait complete. 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 Processor -- start 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- start start 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40cf450 0x7fc1d4155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d41560a0 0x7fc1d41579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d4157f20 0x7fc1d40ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d415cde0 con 0x7fc1d40cf450 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d415cc60 con 0x7fc1d41560a0 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d406cd80 con 0x7fc1d4157f20 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40cf450 0x7fc1d4155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40cf450 0x7fc1d4155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52912/0 (socket says 172.21.15.43:52912) 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 -- 172.21.15.43:0/3406466482 learned_addr learned my addr 172.21.15.43:0/3406466482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1d9bb3640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d41560a0 0x7fc1d41579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dabb5640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d4157f20 0x7fc1d40ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 -- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d4157f20 msgr2=0x7fc1d40ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d4157f20 0x7fc1d40ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 -- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d41560a0 msgr2=0x7fc1d41579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d41560a0 0x7fc1d41579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 -- 172.21.15.43:0/3406466482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d41f7a00 con 0x7fc1d40cf450 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dabb5640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d4157f20 0x7fc1d40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1da3b4640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40cf450 0x7fc1d4155b60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0007fa0 tx=0x7fc1d000e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1caffd640 1 -- 172.21.15.43:0/3406466482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0015610 con 0x7fc1d40cf450 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1caffd640 1 -- 172.21.15.43:0/3406466482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d001a070 con 0x7fc1d40cf450 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1caffd640 1 -- 172.21.15.43:0/3406466482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00159b0 con 0x7fc1d40cf450 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d41f7c70 con 0x7fc1d40cf450 2024-03-30T13:12:20.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.051+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d41f8230 con 0x7fc1d40cf450 2024-03-30T13:12:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.055+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d40687f0 con 0x7fc1d40cf450 2024-03-30T13:12:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.055+0000 7fc1caffd640 1 -- 172.21.15.43:0/3406466482 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc1d0012030 con 0x7fc1d40cf450 2024-03-30T13:12:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.055+0000 7fc1caffd640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc1b8067fc0 0x7fc1b806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.055+0000 7fc1caffd640 1 -- 172.21.15.43:0/3406466482 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5795+0+0 (secure 0 0 0) 0x7fc1d00871b0 con 0x7fc1d40cf450 2024-03-30T13:12:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.055+0000 7fc1d9bb3640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc1b8067fc0 0x7fc1b806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.055+0000 7fc1d9bb3640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc1b8067fc0 0x7fc1b806a480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40cae90 tx=0x7fc1c4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:20.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.059+0000 7fc1caffd640 1 -- 172.21.15.43:0/3406466482 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc1d0010040 con 0x7fc1d40cf450 2024-03-30T13:12:20.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.223+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc1d4156480 con 0x7fc1d40cf450 2024-03-30T13:12:20.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.919+0000 7fc1caffd640 1 -- 172.21.15.43:0/3406466482 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7fc1d0057900 con 0x7fc1d40cf450 2024-03-30T13:12:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data_ec' removed 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc1b8067fc0 msgr2=0x7fc1b806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc1b8067fc0 0x7fc1b806a480 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40cae90 tx=0x7fc1c4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40cf450 msgr2=0x7fc1d4155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40cf450 0x7fc1d4155b60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0007fa0 tx=0x7fc1d000e5d0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 shutdown_connections 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc1b8067fc0 0x7fc1b806a480 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc1d4157f20 0x7fc1d40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc1d41560a0 0x7fc1d41579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 --2- 172.21.15.43:0/3406466482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1d40cf450 0x7fc1d4155b60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:20.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.927+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 >> 172.21.15.43:0/3406466482 conn(0x7fc1d40b1b80 msgr2=0x7fc1d4158ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:20.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.931+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 shutdown_connections 2024-03-30T13:12:20.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:20.931+0000 7fc1dc618640 1 -- 172.21.15.43:0/3406466482 wait complete. 2024-03-30T13:12:20.960 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-30T13:12:20.960 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-30T13:12:20.960 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T13:12:20.960 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:12:20.961 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-30T13:12:20.961 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-30T13:12:20.961 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T13:12:20.961 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:12:20.961 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-30T13:12:20.961 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-30T13:12:20.961 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T13:12:20.962 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:12:20.962 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-30T13:12:20.962 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-30T13:12:20.962 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T13:12:20.962 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:12:20.971 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T13:12:20.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-30T13:12:20.973 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T13:12:20.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-30T13:12:20.976 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T13:12:20.976 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-30T13:12:20.977 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T13:12:20.977 DEBUG:teuthology.orchestra.run.smithi113:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-30T13:12:21.017 INFO:tasks.ceph.mds.c:Started 2024-03-30T13:12:21.018 DEBUG:teuthology.parallel:result is None 2024-03-30T13:12:21.019 INFO:tasks.ceph.mds.a:Started 2024-03-30T13:12:21.019 DEBUG:teuthology.parallel:result is None 2024-03-30T13:12:21.022 INFO:tasks.ceph.mds.b:Started 2024-03-30T13:12:21.022 DEBUG:teuthology.parallel:result is None 2024-03-30T13:12:21.023 INFO:tasks.ceph.mds.d:Started 2024-03-30T13:12:21.023 DEBUG:teuthology.parallel:result is None 2024-03-30T13:12:21.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-30T13:12:21.088 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-03-30T13:12:21.088 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-03-30T13:12:21.089 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-03-30T13:12:21.083+0000 7fca20d6a640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-03-30T13:12:21.094 INFO:tasks.ceph.mds.b.smithi113.stdout:starting mds.b at 2024-03-30T13:12:21.096 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-03-30T13:12:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.131+0000 7f9656240640 1 Processor -- start 2024-03-30T13:12:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.131+0000 7f9656240640 1 -- start start 2024-03-30T13:12:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9650147a50 0x7f96500c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f9650161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96500cfce0 con 0x7f96501624f0 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f96500c8f40 con 0x7f9650147a50 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9650162920 con 0x7f96500cedd0 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9650147a50 0x7f96500c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f96501624f0 0x7f96500c6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34088/0 (socket says 172.21.15.43:34088) 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964ffff640 1 -- 172.21.15.43:0/3127119896 learned_addr learned my addr 172.21.15.43:0/3127119896 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964effd640 1 --2- 172.21.15.43:0/3127119896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f9650161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964f7fe640 1 -- 172.21.15.43:0/3127119896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 msgr2=0x7f9650161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964f7fe640 1 --2- 172.21.15.43:0/3127119896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f9650161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964f7fe640 1 -- 172.21.15.43:0/3127119896 >> v1:172.21.15.43:6789/0 conn(0x7f96501624f0 legacy=0x7f96500c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964f7fe640 1 -- 172.21.15.43:0/3127119896 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96500cf390 con 0x7f9650147a50 2024-03-30T13:12:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964e7fc640 1 -- 172.21.15.43:0/3127119896 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360543985 0 0) 0x7f96500cfce0 con 0x7f96501624f0 2024-03-30T13:12:21.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964f7fe640 1 --2- 172.21.15.43:0/3127119896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96500c8a00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96400097f0 tx=0x7f964001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d4c93a307d9322 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:21.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964e7fc640 1 -- 172.21.15.43:0/3127119896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96400b0070 con 0x7f9650147a50 2024-03-30T13:12:21.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964e7fc640 1 -- 172.21.15.43:0/3127119896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96400ab600 con 0x7f9650147a50 2024-03-30T13:12:21.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f964e7fc640 1 -- 172.21.15.43:0/3127119896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96400ab9d0 con 0x7f9650147a50 2024-03-30T13:12:21.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- 172.21.15.43:0/3127119896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 msgr2=0x7f96500c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:21.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 --2- 172.21.15.43:0/3127119896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96500c8a00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96400097f0 tx=0x7f964001c4a0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- 172.21.15.43:0/3127119896 shutdown_connections 2024-03-30T13:12:21.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 --2- 172.21.15.43:0/3127119896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f9650161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 --2- 172.21.15.43:0/3127119896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96500c8a00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- 172.21.15.43:0/3127119896 >> 172.21.15.43:0/3127119896 conn(0x7f96500b3c80 msgr2=0x7f96500b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- 172.21.15.43:0/3127119896 shutdown_connections 2024-03-30T13:12:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.135+0000 7f9656240640 1 -- 172.21.15.43:0/3127119896 wait complete. 2024-03-30T13:12:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 Processor -- start 2024-03-30T13:12:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 -- start start 2024-03-30T13:12:21.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f96501428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:21.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96501618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:21.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96501624f0 0x7f965015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:21.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f96500b5100 con 0x7f9650147a50 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f96500b4f80 con 0x7f96501624f0 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9650147e30 con 0x7f96500cedd0 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96501618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96501618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52948/0 (socket says 172.21.15.43:52948) 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 -- 172.21.15.43:0/2728037702 learned_addr learned my addr 172.21.15.43:0/2728037702 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964f7fe640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f96501428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964ffff640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96501624f0 0x7f965015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 -- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 msgr2=0x7f96501428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f96501428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 -- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96501624f0 msgr2=0x7f965015fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96501624f0 0x7f965015fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 -- 172.21.15.43:0/2728037702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9650160880 con 0x7f9650147a50 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f964effd640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96501618f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f964400e870 tx=0x7f964400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9637fff640 1 -- 172.21.15.43:0/2728037702 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644017070 con 0x7f9650147a50 2024-03-30T13:12:21.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9637fff640 1 -- 172.21.15.43:0/2728037702 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964400c8e0 con 0x7f9650147a50 2024-03-30T13:12:21.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9650160e10 con 0x7f9650147a50 2024-03-30T13:12:21.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9637fff640 1 -- 172.21.15.43:0/2728037702 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400cc90 con 0x7f9650147a50 2024-03-30T13:12:21.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96501613d0 con 0x7f9650147a50 2024-03-30T13:12:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.139+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96500d1e20 con 0x7f9650147a50 2024-03-30T13:12:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.143+0000 7f9637fff640 1 -- 172.21.15.43:0/2728037702 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f964401a070 con 0x7f9650147a50 2024-03-30T13:12:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.143+0000 7f9637fff640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f963c067fc0 0x7f963c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.143+0000 7f9637fff640 1 -- 172.21.15.43:0/2728037702 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5335+0+0 (secure 0 0 0) 0x7f96440861a0 con 0x7f9650147a50 2024-03-30T13:12:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.143+0000 7f964f7fe640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f963c067fc0 0x7f963c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.143+0000 7f964f7fe640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f963c067fc0 0x7f963c06a480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f96400ad000 tx=0x7f96400a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.147+0000 7f9637fff640 1 -- 172.21.15.43:0/2728037702 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f96500d1e20 con 0x7f9650147a50 2024-03-30T13:12:21.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.319+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f9650068890 con 0x7f9650147a50 2024-03-30T13:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.947+0000 7f9637fff640 1 -- 172.21.15.43:0/2728037702 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) ==== 97+0+0 (secure 0 0 0) 0x7f9644056480 con 0x7f9650147a50 2024-03-30T13:12:21.955 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-03-30T13:12:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f963c067fc0 msgr2=0x7f963c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f963c067fc0 0x7f963c06a480 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f96400ad000 tx=0x7f96400a6000 comp rx=0 tx=0).stop 2024-03-30T13:12:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 msgr2=0x7f96501618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:21.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96501618f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f964400e870 tx=0x7f964400ec70 comp rx=0 tx=0).stop 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 shutdown_connections 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f963c067fc0 0x7f963c06a480 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f96501624f0 0x7f965015fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9650147a50 0x7f96501618f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 --2- 172.21.15.43:0/2728037702 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f96500cedd0 0x7f96501428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 >> 172.21.15.43:0/2728037702 conn(0x7f96500b3c80 msgr2=0x7f96500b29c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 shutdown_connections 2024-03-30T13:12:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:21.951+0000 7f9656240640 1 -- 172.21.15.43:0/2728037702 wait complete. 2024-03-30T13:12:21.981 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-30T13:12:22.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f31f68640 1 Processor -- start 2024-03-30T13:12:22.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f31f68640 1 -- start start 2024-03-30T13:12:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f31f68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 0x7f6f2c0b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f31f68640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f2c0d1100 0x7f6f2c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f31f68640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f2c0d3e60 con 0x7f6f2c0b8160 2024-03-30T13:12:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f31f68640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6f2c0b7450 con 0x7f6f2c0d1100 2024-03-30T13:12:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f31f68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2c0d1aa0 con 0x7f6f2c15e600 2024-03-30T13:12:22.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6f2c0b8160 0x7f6f2c0b85f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40888/0 (socket says 172.21.15.43:40888) 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2bfff640 1 -- 172.21.15.43:0/1379347112 learned_addr learned my addr 172.21.15.43:0/1379347112 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2b7fe640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 0x7f6f2c0b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2affd640 1 --2- 172.21.15.43:0/1379347112 >> v2:172.21.15.113:3300/0 conn(0x7f6f2c0d1100 0x7f6f2c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 260641429 0 0) 0x7f6f2c0d3e60 con 0x7f6f2c0b8160 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f14003530 con 0x7f6f2c0b8160 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 276655401 0 0) 0x7f6f14003530 con 0x7f6f2c0b8160 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 msgr2=0x7f6f2c0b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 0x7f6f2c0b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 >> v2:172.21.15.113:3300/0 conn(0x7f6f2c0d1100 msgr2=0x7f6f2c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 --2- 172.21.15.43:0/1379347112 >> v2:172.21.15.113:3300/0 conn(0x7f6f2c0d1100 0x7f6f2c0d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2c0b7680 con 0x7f6f2c0b8160 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f6f24001a00 con 0x7f6f2c0b8160 2024-03-30T13:12:22.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 >> v1:172.21.15.43:6789/0 conn(0x7f6f2c0b8160 legacy=0x7f6f2c0b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f140039c0 0x7f6f14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f14004810 0x7f6f14004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 0x7f6f140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1400b9d0 con 0x7f6f14004810 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1400bb50 con 0x7f6f140039c0 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1400bcd0 con 0x7f6f2c15e600 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2affd640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f14004810 0x7f6f14004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2b7fe640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f140039c0 0x7f6f14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2bfff640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 0x7f6f140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2affd640 1 -- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 msgr2=0x7f6f140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2affd640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 0x7f6f140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2affd640 1 -- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f140039c0 msgr2=0x7f6f14003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2affd640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f140039c0 0x7f6f14003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2affd640 1 -- 172.21.15.43:0/1379347112 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f14003530 con 0x7f6f14004810 2024-03-30T13:12:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2b7fe640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f140039c0 0x7f6f14003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.103+0000 7f6f2bfff640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c15e600 0x7f6f140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2affd640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f14004810 0x7f6f14004c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009ae0 tx=0x7f6f2001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314d914c71c6db7c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f200b2070 con 0x7f6f14004810 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f200a55c0 con 0x7f6f14004810 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2a7fc640 1 -- 172.21.15.43:0/1379347112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f200096b0 con 0x7f6f14004810 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- 172.21.15.43:0/1379347112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f14004810 msgr2=0x7f6f14004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 --2- 172.21.15.43:0/1379347112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f14004810 0x7f6f14004c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6f20009ae0 tx=0x7f6f2001cab0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2bfff640 1 -- 172.21.15.43:0/1379347112 reap_dead start 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- 172.21.15.43:0/1379347112 shutdown_connections 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- 172.21.15.43:0/1379347112 >> 172.21.15.43:0/1379347112 conn(0x7f6f2c0b6230 msgr2=0x7f6f2c0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- 172.21.15.43:0/1379347112 shutdown_connections 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- 172.21.15.43:0/1379347112 wait complete. 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 Processor -- start 2024-03-30T13:12:22.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- start start 2024-03-30T13:12:22.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f2c14d840 0x7f6f2c15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2c0cbe00 0x7f6f2c0cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c0cd3a0 0x7f6f2c0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c0b7c40 con 0x7f6f2c14d840 2024-03-30T13:12:22.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6f2c0d4140 con 0x7f6f2c0cbe00 2024-03-30T13:12:22.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f31f68640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2c0b9f00 con 0x7f6f2c0cd3a0 2024-03-30T13:12:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f2c14d840 0x7f6f2c15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f2c14d840 0x7f6f2c15ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52574/0 (socket says 172.21.15.43:52574) 2024-03-30T13:12:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.107+0000 7f6f2b7fe640 1 -- 172.21.15.43:0/2635304959 learned_addr learned my addr 172.21.15.43:0/2635304959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2affd640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2c0cbe00 0x7f6f2c0cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2bfff640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c0cd3a0 0x7f6f2c0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2b7fe640 1 -- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c0cd3a0 msgr2=0x7f6f2c0cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2b7fe640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c0cd3a0 0x7f6f2c0cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2b7fe640 1 -- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2c0cbe00 msgr2=0x7f6f2c0cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2b7fe640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2c0cbe00 0x7f6f2c0cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2b7fe640 1 -- 172.21.15.43:0/2635304959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2c0cc300 con 0x7f6f2c14d840 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2b7fe640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f2c14d840 0x7f6f2c15ffd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c010900 tx=0x7f6f1c010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f13fff640 1 -- 172.21.15.43:0/2635304959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00f040 con 0x7f6f2c14d840 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f13fff640 1 -- 172.21.15.43:0/2635304959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f1c004220 con 0x7f6f2c14d840 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2c0cc5e0 con 0x7f6f2c14d840 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f13fff640 1 -- 172.21.15.43:0/2635304959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c00e9c0 con 0x7f6f2c14d840 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f2c0ccba0 con 0x7f6f2c14d840 2024-03-30T13:12:22.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2c068890 con 0x7f6f2c14d840 2024-03-30T13:12:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f13fff640 1 -- 172.21.15.43:0/2635304959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6f1c00eb60 con 0x7f6f2c14d840 2024-03-30T13:12:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f13fff640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f18067fc0 0x7f6f1806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f13fff640 1 -- 172.21.15.43:0/2635304959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f6f1c086a40 con 0x7f6f2c14d840 2024-03-30T13:12:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2affd640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f18067fc0 0x7f6f1806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.111+0000 7f6f2affd640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f18067fc0 0x7f6f1806a480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f200046d0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.115+0000 7f6f13fff640 1 -- 172.21.15.43:0/2635304959 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6f1c018030 con 0x7f6f2c14d840 2024-03-30T13:12:22.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.335+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f6f2c0c7a20 con 0x7f6f2c14d840 2024-03-30T13:12:22.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.335+0000 7f6f13fff640 1 -- 172.21.15.43:0/2635304959 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) ==== 80+0+6645 (secure 0 0 0) 0x7f6f1c057390 con 0x7f6f2c14d840 2024-03-30T13:12:22.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:22.344 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhmqjrKHRAAjgqPklYwNqmXlXxyG2hCDg==", 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-03-30T13:12:22.345 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhmszFnLBAAQW0A5lFQf9NCopOQYF0KgA==", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhmv5Q0JhAAFNDkQ/t0fNOi08HbPPZ51g==", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-03-30T13:12:22.346 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhm7Y4NNRAAUfV2sqtMJLlNrUjGSOeMPQ==", 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.347 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAoDwhmodjkDxAAnARCBa5r+k4VpsrBHZJPdA==", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQArDwhmh68CFBAAwSBpkmAtLrFsUUPDlfLsPg==", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.348 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAuDwhmWpP5GhAAOxvQoZdxhmbslCPu3aB+sw==", 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAxDwhmKGaFIhAAggPT08CCwHt3qW8OCLR7Xw==", 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQA3DwhmcUBAKBAAnRIdvtxsWe8tCKU74LjnGQ==", 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.350 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQA6DwhmWILuLBAA1SK/P0b7xFhvb1Nw6Etn2w==", 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.351 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQA9Dwhm+c8JMxAACTA0hZnr45rL7JeRXc8EwA==", 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.352 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQBADwhmDuL9ORAA/qZhjTe4/hHKQe+VNHvjxg==", 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhmDh1uOhAAyQ23sd4h6j2HVM6D/hX5WA==", 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.353 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAnDwhm7Sz7BhAAV0khY4znoozzYORr7aJISQ==", 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.354 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAnDwhmCPsZBBAARwmR4u1Gdq/f7UsCnCHyEg==", 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.355 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAnDwhmqHP/CxAAyKG1FC4XCpBgsEHA/lKHYg==", 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhm0ewGBxAAKMIPF4J4BajUN5chvFSaHQ==", 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.356 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mds", 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQBHDwhmimiOCBAAI3W86Tyd2nnJtyPVEnJBow==", 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mds" 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mgr", 2024-03-30T13:12:22.357 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQBHDwhm25KOCBAAZJgLdLF5WiP+S3H2StDmRg==", 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-osd", 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQBHDwhm+KqOCBAA0zXTwnKTN/qTOwmfRCsjwA==", 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-osd" 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd", 2024-03-30T13:12:22.358 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQBHDwhmj8COCBAARbV0tfKvEYrn3BQgOfxlrw==", 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQBHDwhm7taOCBAAyRPuQqEtI4yhERALCL08Iw==", 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rgw", 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQBHDwhmoO2OCBAAjORM8FPNs+ptI9H+Z4lkFA==", 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhmD5t8GhAAhvu7f7K2maRFCWA/n1i6bQ==", 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-03-30T13:12:22.360 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAmDwhmpMenFxAA0TnJp3fiTI61U2P/GDktCQ==", 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-30T13:12:22.361 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:12:22.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f18067fc0 msgr2=0x7f6f1806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f18067fc0 0x7f6f1806a480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f200046d0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f2c14d840 msgr2=0x7f6f2c15ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f2c14d840 0x7f6f2c15ffd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c010900 tx=0x7f6f1c010dc0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 shutdown_connections 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f18067fc0 0x7f6f1806a480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f2c0cd3a0 0x7f6f2c0cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f2c0cbe00 0x7f6f2c0cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 --2- 172.21.15.43:0/2635304959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f2c14d840 0x7f6f2c15ffd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 >> 172.21.15.43:0/2635304959 conn(0x7f6f2c0b6230 msgr2=0x7f6f2c15b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 shutdown_connections 2024-03-30T13:12:22.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.339+0000 7f6f31f68640 1 -- 172.21.15.43:0/2635304959 wait complete. 2024-03-30T13:12:22.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-30T13:12:22.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d4207640 1 Processor -- start 2024-03-30T13:12:22.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d4207640 1 -- start start 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d4207640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd3cc15a840 0x7fd3cc15e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d4207640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3cc0b3f00 0x7fd3cc06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d4207640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3cc072cb0 con 0x7fd3cc15e940 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d4207640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd3cc15eda0 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d4207640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd3cc072f90 con 0x7fd3cc15a840 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.475+0000 7fd3d27a4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3cc0b3f00 0x7fd3cc06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d27a4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd3cc0b3f00 0x7fd3cc06e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52578/0 (socket says 172.21.15.43:52578) 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d27a4640 1 -- 172.21.15.43:0/3987583871 learned_addr learned my addr 172.21.15.43:0/3987583871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d1fa3640 1 --2- 172.21.15.43:0/3987583871 >> v2:172.21.15.113:3300/0 conn(0x7fd3cc15a840 0x7fd3cc15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d27a4640 1 -- 172.21.15.43:0/3987583871 >> v2:172.21.15.113:3300/0 conn(0x7fd3cc15a840 msgr2=0x7fd3cc15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:22.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d27a4640 1 --2- 172.21.15.43:0/3987583871 >> v2:172.21.15.113:3300/0 conn(0x7fd3cc15a840 0x7fd3cc15e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d27a4640 1 -- 172.21.15.43:0/3987583871 >> v1:172.21.15.113:6789/0 conn(0x7fd3cc15e940 legacy=0x7fd3cc06da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:22.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d27a4640 1 -- 172.21.15.43:0/3987583871 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc073270 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d27a4640 1 --2- 172.21.15.43:0/3987583871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc06e160 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc06d0b0 tx=0x7fd3c401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bf02beefa04d56 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d0fa1640 1 -- 172.21.15.43:0/3987583871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40a57d0 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d0fa1640 1 -- 172.21.15.43:0/3987583871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c40a5970 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d0fa1640 1 -- 172.21.15.43:0/3987583871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40a5d40 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 -- 172.21.15.43:0/3987583871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 msgr2=0x7fd3cc06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 --2- 172.21.15.43:0/3987583871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc06e160 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc06d0b0 tx=0x7fd3c401c8f0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 -- 172.21.15.43:0/3987583871 shutdown_connections 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 --2- 172.21.15.43:0/3987583871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc06e160 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 --2- 172.21.15.43:0/3987583871 >> v2:172.21.15.113:3300/0 conn(0x7fd3cc15a840 0x7fd3cc15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 -- 172.21.15.43:0/3987583871 >> 172.21.15.43:0/3987583871 conn(0x7fd3cc0cfee0 msgr2=0x7fd3cc0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 -- 172.21.15.43:0/3987583871 shutdown_connections 2024-03-30T13:12:22.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.479+0000 7fd3d4207640 1 -- 172.21.15.43:0/3987583871 wait complete. 2024-03-30T13:12:22.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 Processor -- start 2024-03-30T13:12:22.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 -- start start 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc1f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3cc15a840 0x7fd3cc1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3cc15e940 0x7fd3cc0ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc0b44d0 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd3cc0b4350 con 0x7fd3cc15a840 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd3cc0728d0 con 0x7fd3cc15e940 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc1f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc1f82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52590/0 (socket says 172.21.15.43:52590) 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 -- 172.21.15.43:0/2250988716 learned_addr learned my addr 172.21.15.43:0/2250988716 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d17a2640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3cc15a840 0x7fd3cc1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d27a4640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3cc15e940 0x7fd3cc0ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 -- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3cc15e940 msgr2=0x7fd3cc0ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3cc15e940 0x7fd3cc0ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 -- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3cc15a840 msgr2=0x7fd3cc1595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3cc15a840 0x7fd3cc1595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 -- 172.21.15.43:0/2250988716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3cc156870 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d27a4640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3cc15e940 0x7fd3cc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d1fa3640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc1f82f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d800 tx=0x7fd3c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3c27fc640 1 -- 172.21.15.43:0/2250988716 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800b6c0 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3c27fc640 1 -- 172.21.15.43:0/2250988716 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3c8019070 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3cc156b50 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3c27fc640 1 -- 172.21.15.43:0/2250988716 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800bae0 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3cc157110 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3cc068890 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.483+0000 7fd3c27fc640 1 -- 172.21.15.43:0/2250988716 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd3c800bc80 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.487+0000 7fd3c27fc640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd3b0067fc0 0x7fd3b006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.487+0000 7fd3c27fc640 1 -- 172.21.15.43:0/2250988716 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fd3c80868b0 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.487+0000 7fd3d17a2640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd3b0067fc0 0x7fd3b006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.487+0000 7fd3d17a2640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd3b0067fc0 0x7fd3b006a480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc14bba0 tx=0x7fd3bc008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.491+0000 7fd3c27fc640 1 -- 172.21.15.43:0/2250988716 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fd3c8057220 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.703+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd3cc0719d0 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3c27fc640 1 -- 172.21.15.43:0/2250988716 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fd3c8056b20 con 0x7fd3cc0b3f00 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd3b0067fc0 msgr2=0x7fd3b006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd3b0067fc0 0x7fd3b006a480 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc14bba0 tx=0x7fd3bc008040 comp rx=0 tx=0).stop 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 msgr2=0x7fd3cc1f82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc1f82f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd3c800d800 tx=0x7fd3c800dcc0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 shutdown_connections 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd3b0067fc0 0x7fd3b006a480 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd3cc15e940 0x7fd3cc0ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.707+0000 7fd3d4207640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd3cc15a840 0x7fd3cc1595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.711+0000 7fd3d4207640 1 --2- 172.21.15.43:0/2250988716 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd3cc0b3f00 0x7fd3cc1f82f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.711+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 >> 172.21.15.43:0/2250988716 conn(0x7fd3cc0cfee0 msgr2=0x7fd3cc06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.711+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 shutdown_connections 2024-03-30T13:12:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.711+0000 7fd3d4207640 1 -- 172.21.15.43:0/2250988716 wait complete. 2024-03-30T13:12:22.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-30T13:12:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe8930ec640 1 Processor -- start 2024-03-30T13:12:22.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe8930ec640 1 -- start start 2024-03-30T13:12:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe8930ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe88c144270 0x7fe88c144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe8930ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe88c0b2ef0 0x7fe88c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe8930ec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe88c0b33f0 con 0x7fe88c0b4490 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe8930ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe88c144fa0 con 0x7fe88c144270 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe8930ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe88c15fce0 con 0x7fe88c0b2ef0 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe890e88640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe88c144270 0x7fe88c144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe891689640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe88c0b4490 0x7fe88c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40904/0 (socket says 172.21.15.43:40904) 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe891689640 1 -- 172.21.15.43:0/152829750 learned_addr learned my addr 172.21.15.43:0/152829750 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe88b7fe640 1 -- 172.21.15.43:0/152829750 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 732451868 0 0) 0x7fe88c0b33f0 con 0x7fe88c0b4490 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe88bfff640 1 --2- 172.21.15.43:0/152829750 >> v2:172.21.15.113:3300/0 conn(0x7fe88c0b2ef0 0x7fe88c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe88b7fe640 1 -- 172.21.15.43:0/152829750 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe874003530 con 0x7fe88c0b4490 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe890e88640 1 -- 172.21.15.43:0/152829750 >> v2:172.21.15.113:3300/0 conn(0x7fe88c0b2ef0 msgr2=0x7fe88c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe890e88640 1 --2- 172.21.15.43:0/152829750 >> v2:172.21.15.113:3300/0 conn(0x7fe88c0b2ef0 0x7fe88c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe890e88640 1 -- 172.21.15.43:0/152829750 >> v1:172.21.15.43:6789/0 conn(0x7fe88c0b4490 legacy=0x7fe88c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe890e88640 1 -- 172.21.15.43:0/152829750 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c0b36d0 con 0x7fe88c144270 2024-03-30T13:12:22.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.851+0000 7fe88bfff640 1 --2- 172.21.15.43:0/152829750 >> v2:172.21.15.113:3300/0 conn(0x7fe88c0b2ef0 0x7fe88c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe890e88640 1 --2- 172.21.15.43:0/152829750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c144650 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0b4e70 tx=0x7fe87c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d745bbefe95790ae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe88b7fe640 1 -- 172.21.15.43:0/152829750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0ab650 con 0x7fe88c144270 2024-03-30T13:12:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe88b7fe640 1 -- 172.21.15.43:0/152829750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe87c0b0070 con 0x7fe88c144270 2024-03-30T13:12:22.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe88b7fe640 1 -- 172.21.15.43:0/152829750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe87c0aba10 con 0x7fe88c144270 2024-03-30T13:12:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- 172.21.15.43:0/152829750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 msgr2=0x7fe88c144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 --2- 172.21.15.43:0/152829750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c144650 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fe88c0b4e70 tx=0x7fe87c01c450 comp rx=0 tx=0).stop 2024-03-30T13:12:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- 172.21.15.43:0/152829750 shutdown_connections 2024-03-30T13:12:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 --2- 172.21.15.43:0/152829750 >> v2:172.21.15.113:3300/0 conn(0x7fe88c0b2ef0 0x7fe88c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 --2- 172.21.15.43:0/152829750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c144650 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- 172.21.15.43:0/152829750 >> 172.21.15.43:0/152829750 conn(0x7fe88c0cfee0 msgr2=0x7fe88c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:22.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- 172.21.15.43:0/152829750 shutdown_connections 2024-03-30T13:12:22.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- 172.21.15.43:0/152829750 wait complete. 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 Processor -- start 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- start start 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe88c0b2ef0 0x7fe88c13f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe88c0b4490 0x7fe88c15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c0cdd10 con 0x7fe88c144270 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c06cf90 con 0x7fe88c0b4490 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe8930ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe88c0cde90 con 0x7fe88c0b2ef0 2024-03-30T13:12:22.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c15d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52612/0 (socket says 172.21.15.43:52612) 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 -- 172.21.15.43:0/2101368024 learned_addr learned my addr 172.21.15.43:0/2101368024 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe890e88640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe88c0b2ef0 0x7fe88c13f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe88bfff640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe88c0b4490 0x7fe88c15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 -- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe88c0b2ef0 msgr2=0x7fe88c13f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe88c0b2ef0 0x7fe88c13f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 -- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe88c0b4490 msgr2=0x7fe88c15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe88c0b4490 0x7fe88c15ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 -- 172.21.15.43:0/2101368024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c15c3c0 con 0x7fe88c144270 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe88bfff640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe88c0b4490 0x7fe88c15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.855+0000 7fe891689640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c15d350 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe88400cb70 tx=0x7fe88400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8897fa640 1 -- 172.21.15.43:0/2101368024 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884017070 con 0x7fe88c144270 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8897fa640 1 -- 172.21.15.43:0/2101368024 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe88400ad40 con 0x7fe88c144270 2024-03-30T13:12:22.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c15c660 con 0x7fe88c144270 2024-03-30T13:12:22.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8897fa640 1 -- 172.21.15.43:0/2101368024 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884012620 con 0x7fe88c144270 2024-03-30T13:12:22.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c13f5f0 con 0x7fe88c144270 2024-03-30T13:12:22.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe88c068890 con 0x7fe88c144270 2024-03-30T13:12:22.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8897fa640 1 -- 172.21.15.43:0/2101368024 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe88401a070 con 0x7fe88c144270 2024-03-30T13:12:22.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8897fa640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe878067fc0 0x7fe87806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:22.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe8897fa640 1 -- 172.21.15.43:0/2101368024 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fe8840860f0 con 0x7fe88c144270 2024-03-30T13:12:22.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe890e88640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe878067fc0 0x7fe87806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:22.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.859+0000 7fe890e88640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe878067fc0 0x7fe87806a480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0af000 tx=0x7fe87c0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:22.863+0000 7fe8897fa640 1 -- 172.21.15.43:0/2101368024 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe884056af0 con 0x7fe88c144270 2024-03-30T13:12:23.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.087+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fe88c169120 con 0x7fe88c144270 2024-03-30T13:12:23.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8897fa640 1 -- 172.21.15.43:0/2101368024 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fe8840563f0 con 0x7fe88c144270 2024-03-30T13:12:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe878067fc0 msgr2=0x7fe87806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe878067fc0 0x7fe87806a480 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe87c0af000 tx=0x7fe87c0a7000 comp rx=0 tx=0).stop 2024-03-30T13:12:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 msgr2=0x7fe88c15d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c15d350 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe88400cb70 tx=0x7fe88400a430 comp rx=0 tx=0).stop 2024-03-30T13:12:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 shutdown_connections 2024-03-30T13:12:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe878067fc0 0x7fe87806a480 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe88c144270 0x7fe88c15d350 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe88c0b4490 0x7fe88c15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 --2- 172.21.15.43:0/2101368024 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe88c0b2ef0 0x7fe88c13f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 >> 172.21.15.43:0/2101368024 conn(0x7fe88c0cfee0 msgr2=0x7fe88c143a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 shutdown_connections 2024-03-30T13:12:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.091+0000 7fe8930ec640 1 -- 172.21.15.43:0/2101368024 wait complete. 2024-03-30T13:12:23.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-30T13:12:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f9392ab2640 1 Processor -- start 2024-03-30T13:12:23.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f9392ab2640 1 -- start start 2024-03-30T13:12:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f9392ab2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f938c15e600 0x7f938c0b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f9392ab2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f938c0d1100 0x7f938c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f9392ab2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f938c0d3e60 con 0x7f938c0b8160 2024-03-30T13:12:23.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f9392ab2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f938c0b7450 con 0x7f938c0d1100 2024-03-30T13:12:23.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f9392ab2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f938c0d1aa0 con 0x7f938c15e600 2024-03-30T13:12:23.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f938bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f938c0d1100 0x7f938c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f938bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f938c0d1100 0x7f938c0d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52628/0 (socket says 172.21.15.43:52628) 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f938bfff640 1 -- 172.21.15.43:0/3333376590 learned_addr learned my addr 172.21.15.43:0/3333376590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f939084e640 1 --2- 172.21.15.43:0/3333376590 >> v2:172.21.15.113:3300/0 conn(0x7f938c15e600 0x7f938c0b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f938bfff640 1 -- 172.21.15.43:0/3333376590 >> v2:172.21.15.113:3300/0 conn(0x7f938c15e600 msgr2=0x7f938c0b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f938bfff640 1 --2- 172.21.15.43:0/3333376590 >> v2:172.21.15.113:3300/0 conn(0x7f938c15e600 0x7f938c0b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f938bfff640 1 -- 172.21.15.43:0/3333376590 >> v1:172.21.15.113:6789/0 conn(0x7f938c0b8160 legacy=0x7f938c0b85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f938bfff640 1 -- 172.21.15.43:0/3333376590 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c0b7680 con 0x7f938c0d1100 2024-03-30T13:12:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.239+0000 7f939084e640 1 --2- 172.21.15.43:0/3333376590 >> v2:172.21.15.113:3300/0 conn(0x7f938c15e600 0x7f938c0b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:23.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f938bfff640 1 --2- 172.21.15.43:0/3333376590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c0d1100 0x7f938c0d1560 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f938c0b53e0 tx=0x7f938001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688b952625ad4277 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:23.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f938b7fe640 1 -- 172.21.15.43:0/3333376590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800a5760 con 0x7f938c0d1100 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f938b7fe640 1 -- 172.21.15.43:0/3333376590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93800a5900 con 0x7f938c0d1100 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f938b7fe640 1 -- 172.21.15.43:0/3333376590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800a5cd0 con 0x7f938c0d1100 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- 172.21.15.43:0/3333376590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c0d1100 msgr2=0x7f938c0d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 --2- 172.21.15.43:0/3333376590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c0d1100 0x7f938c0d1560 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f938c0b53e0 tx=0x7f938001c9b0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- 172.21.15.43:0/3333376590 shutdown_connections 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 --2- 172.21.15.43:0/3333376590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c0d1100 0x7f938c0d1560 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 --2- 172.21.15.43:0/3333376590 >> v2:172.21.15.113:3300/0 conn(0x7f938c15e600 0x7f938c0b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- 172.21.15.43:0/3333376590 >> 172.21.15.43:0/3333376590 conn(0x7f938c0b6230 msgr2=0x7f938c0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- 172.21.15.43:0/3333376590 shutdown_connections 2024-03-30T13:12:23.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- 172.21.15.43:0/3333376590 wait complete. 2024-03-30T13:12:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 Processor -- start 2024-03-30T13:12:23.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- start start 2024-03-30T13:12:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f938c0b8160 0x7f938c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f938c0d1100 0x7f938c1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c15e600 0x7f938c144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f938c16b150 con 0x7f938c15e600 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f938c16afd0 con 0x7f938c0b8160 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.243+0000 7f9392ab2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f938c16b2d0 con 0x7f938c0d1100 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c15e600 0x7f938c144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c15e600 0x7f938c144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52640/0 (socket says 172.21.15.43:52640) 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 -- 172.21.15.43:0/1612114357 learned_addr learned my addr 172.21.15.43:0/1612114357 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939084e640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f938c0b8160 0x7f938c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f938bfff640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f938c0d1100 0x7f938c1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 -- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f938c0d1100 msgr2=0x7f938c1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f938c0d1100 0x7f938c1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 -- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f938c0b8160 msgr2=0x7f938c15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f938c0b8160 0x7f938c15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 -- 172.21.15.43:0/1612114357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938c1619e0 con 0x7f938c15e600 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939104f640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c15e600 0x7f938c144ff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f937c00b440 tx=0x7f937c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:23.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f93897fa640 1 -- 172.21.15.43:0/1612114357 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c015020 con 0x7f938c15e600 2024-03-30T13:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f93897fa640 1 -- 172.21.15.43:0/1612114357 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f937c011850 con 0x7f938c15e600 2024-03-30T13:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938c15b580 con 0x7f938c15e600 2024-03-30T13:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f93897fa640 1 -- 172.21.15.43:0/1612114357 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c011c40 con 0x7f938c15e600 2024-03-30T13:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938c15bb40 con 0x7f938c15e600 2024-03-30T13:12:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938c068890 con 0x7f938c15e600 2024-03-30T13:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f93897fa640 1 -- 172.21.15.43:0/1612114357 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f937c018020 con 0x7f938c15e600 2024-03-30T13:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f93897fa640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9378067ff0 0x7f937806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f93897fa640 1 -- 172.21.15.43:0/1612114357 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f937c0861f0 con 0x7f938c15e600 2024-03-30T13:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.247+0000 7f939084e640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9378067ff0 0x7f937806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.251+0000 7f939084e640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9378067ff0 0x7f937806a4b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9384006fd0 tx=0x7f9384008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.251+0000 7f93897fa640 1 -- 172.21.15.43:0/1612114357 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f937c01f020 con 0x7f938c15e600 2024-03-30T13:12:23.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.475+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f938c14f710 con 0x7f938c15e600 2024-03-30T13:12:23.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f93897fa640 1 -- 172.21.15.43:0/1612114357 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) ==== 90+0+0 (secure 0 0 0) 0x7f937c0172c0 con 0x7f938c15e600 2024-03-30T13:12:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9378067ff0 msgr2=0x7f937806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9378067ff0 0x7f937806a4b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9384006fd0 tx=0x7f9384008040 comp rx=0 tx=0).stop 2024-03-30T13:12:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c15e600 msgr2=0x7f938c144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c15e600 0x7f938c144ff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f937c00b440 tx=0x7f937c00b900 comp rx=0 tx=0).stop 2024-03-30T13:12:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 shutdown_connections 2024-03-30T13:12:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9378067ff0 0x7f937806a4b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f938c15e600 0x7f938c144ff0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f938c0d1100 0x7f938c1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 --2- 172.21.15.43:0/1612114357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f938c0b8160 0x7f938c15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 >> 172.21.15.43:0/1612114357 conn(0x7f938c0b6230 msgr2=0x7f938c15d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 shutdown_connections 2024-03-30T13:12:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.479+0000 7f9392ab2640 1 -- 172.21.15.43:0/1612114357 wait complete. 2024-03-30T13:12:23.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-30T13:12:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.615+0000 7f5ffaf47640 1 Processor -- start 2024-03-30T13:12:23.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.615+0000 7f5ffaf47640 1 -- start start 2024-03-30T13:12:23.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ffaf47640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 0x7f5ff406e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ffaf47640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff406c480 con 0x7f5ff406d780 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ffaf47640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ff4167110 con 0x7f5ff406dbc0 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ffaf47640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff406ee40 con 0x7f5ff4159680 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5ff406d780 0x7f5ff406b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40912/0 (socket says 172.21.15.43:40912) 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 -- 172.21.15.43:0/1451364422 learned_addr learned my addr 172.21.15.43:0/1451364422 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff8ce3640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 0x7f5ff406e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2829278136 0 0) 0x7f5ff406c480 con 0x7f5ff406d780 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fdc003530 con 0x7f5ff406d780 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1082139557 0 0) 0x7f5fdc003530 con 0x7f5ff406d780 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 msgr2=0x7f5ff406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 0x7f5ff406e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 >> v1:172.21.15.113:6789/0 conn(0x7f5ff406dbc0 legacy=0x7f5ff406bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff4166350 con 0x7f5ff406d780 2024-03-30T13:12:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f5fe8001680 con 0x7f5ff406d780 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 >> v1:172.21.15.43:6789/0 conn(0x7f5ff406d780 legacy=0x7f5ff406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fdc0039f0 0x7f5fdc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fdc004810 0x7f5fdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 0x7f5fdc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff406ee40 con 0x7f5fdc004810 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc00ba80 con 0x7f5fdc0039f0 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5fdc00bc80 con 0x7f5ff4159680 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fdc004810 0x7f5fdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff8ce3640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fdc0039f0 0x7f5fdc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff94e4640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 0x7f5fdc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 -- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 msgr2=0x7f5fdc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4159680 0x7f5fdc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 -- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fdc0039f0 msgr2=0x7f5fdc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5fdc0039f0 0x7f5fdc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.619+0000 7f5ff3fff640 1 -- 172.21.15.43:0/1451364422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc003530 con 0x7f5fdc004810 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fdc004810 0x7f5fdc004c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c210 tx=0x7f5fe801f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b0ba71de7e35820 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8007d20 con 0x7f5fdc004810 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe8005ae0 con 0x7f5fdc004810 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff37fe640 1 -- 172.21.15.43:0/1451364422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8007070 con 0x7f5fdc004810 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- 172.21.15.43:0/1451364422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fdc004810 msgr2=0x7f5fdc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 --2- 172.21.15.43:0/1451364422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fdc004810 0x7f5fdc004c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c210 tx=0x7f5fe801f390 comp rx=0 tx=0).stop 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff94e4640 1 -- 172.21.15.43:0/1451364422 reap_dead start 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- 172.21.15.43:0/1451364422 shutdown_connections 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- 172.21.15.43:0/1451364422 >> 172.21.15.43:0/1451364422 conn(0x7f5ff4072500 msgr2=0x7f5ff4072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- 172.21.15.43:0/1451364422 shutdown_connections 2024-03-30T13:12:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- 172.21.15.43:0/1451364422 wait complete. 2024-03-30T13:12:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 Processor -- start 2024-03-30T13:12:23.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- start start 2024-03-30T13:12:23.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff4145390 0x7f5ff4162cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff4152680 0x7f5ff415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4163230 0x7f5ff415ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4166bf0 con 0x7f5ff4152680 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4166910 con 0x7f5ff4145390 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ffaf47640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5ff406e000 con 0x7f5ff4163230 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff4152680 0x7f5ff415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff4152680 0x7f5ff415e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52658/0 (socket says 172.21.15.43:52658) 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 -- 172.21.15.43:0/4143981084 learned_addr learned my addr 172.21.15.43:0/4143981084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff94e4640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4163230 0x7f5ff415ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 -- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4163230 msgr2=0x7f5ff415ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4163230 0x7f5ff415ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 -- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff4145390 msgr2=0x7f5ff4162cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff8ce3640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff4145390 0x7f5ff4162cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff4145390 0x7f5ff4162cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.623+0000 7f5ff3fff640 1 -- 172.21.15.43:0/4143981084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff41636b0 con 0x7f5ff4152680 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff3fff640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff4152680 0x7f5ff415e6e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5fe801f1c0 tx=0x7f5fdc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff17fa640 1 -- 172.21.15.43:0/4143981084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80048b0 con 0x7f5ff4152680 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff17fa640 1 -- 172.21.15.43:0/4143981084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe8004a50 con 0x7f5ff4152680 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff4163990 con 0x7f5ff4152680 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff17fa640 1 -- 172.21.15.43:0/4143981084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8004e00 con 0x7f5ff4152680 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff4163f50 con 0x7f5ff4152680 2024-03-30T13:12:23.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff4068890 con 0x7f5ff4152680 2024-03-30T13:12:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff17fa640 1 -- 172.21.15.43:0/4143981084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5fe8007070 con 0x7f5ff4152680 2024-03-30T13:12:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff17fa640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fe0067fc0 0x7f5fe006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff17fa640 1 -- 172.21.15.43:0/4143981084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f5fe8123ae0 con 0x7f5ff4152680 2024-03-30T13:12:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff8ce3640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fe0067fc0 0x7f5fe006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:23.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.627+0000 7f5ff8ce3640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fe0067fc0 0x7f5fe006a480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009970 tx=0x7f5fe4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:23.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.631+0000 7f5ff17fa640 1 -- 172.21.15.43:0/4143981084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5fe80c9080 con 0x7f5ff4152680 2024-03-30T13:12:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.851+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f5ff4073870 con 0x7f5ff4152680 2024-03-30T13:12:23.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ff17fa640 1 -- 172.21.15.43:0/4143981084 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f5fe80190a0 con 0x7f5ff4152680 2024-03-30T13:12:23.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fe0067fc0 msgr2=0x7f5fe006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fe0067fc0 0x7f5fe006a480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009970 tx=0x7f5fe4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff4152680 msgr2=0x7f5ff415e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff4152680 0x7f5ff415e6e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5fe801f1c0 tx=0x7f5fdc005840 comp rx=0 tx=0).stop 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 shutdown_connections 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5fe0067fc0 0x7f5fe006a480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ff4163230 0x7f5ff415ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ff4152680 0x7f5ff415e6e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 --2- 172.21.15.43:0/4143981084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ff4145390 0x7f5ff4162cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 >> 172.21.15.43:0/4143981084 conn(0x7f5ff4072500 msgr2=0x7f5ff40b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 shutdown_connections 2024-03-30T13:12:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.855+0000 7f5ffaf47640 1 -- 172.21.15.43:0/4143981084 wait complete. 2024-03-30T13:12:23.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-30T13:12:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a4587640 1 Processor -- start 2024-03-30T13:12:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a4587640 1 -- start start 2024-03-30T13:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a4587640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f229c0cc910 0x7f229c166ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a4587640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229c0ccef0 con 0x7f229c06c670 2024-03-30T13:12:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a4587640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f229c0b3710 con 0x7f229c0b41f0 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a4587640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f229c0b34c0 con 0x7f229c0cc910 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a2323640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f229c06c670 0x7f229c06e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40920/0 (socket says 172.21.15.43:40920) 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a2323640 1 -- 172.21.15.43:0/1356416478 learned_addr learned my addr 172.21.15.43:0/1356416478 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a2b24640 1 --2- 172.21.15.43:0/1356416478 >> v2:172.21.15.113:3300/0 conn(0x7f229c0cc910 0x7f229c166ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096315886 0 0) 0x7f229c0ccef0 con 0x7f229c06c670 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2288003530 con 0x7f229c06c670 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4051433064 0 0) 0x7f2288003530 con 0x7f229c06c670 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 >> v2:172.21.15.113:3300/0 conn(0x7f229c0cc910 msgr2=0x7f229c166ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 --2- 172.21.15.43:0/1356416478 >> v2:172.21.15.113:3300/0 conn(0x7f229c0cc910 0x7f229c166ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 >> v1:172.21.15.113:6789/0 conn(0x7f229c0b41f0 legacy=0x7f229c06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c0b39f0 con 0x7f229c06c670 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f2298001680 con 0x7f229c06c670 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 >> v1:172.21.15.43:6789/0 conn(0x7f229c06c670 legacy=0x7f229c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2288003a10 0x7f2288003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2288004810 0x7f2288004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f228800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0b34c0 con 0x7f22880051d0 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f228800c0d0 con 0x7f2288004810 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:23.999+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f228800c2d0 con 0x7f2288003a10 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f228800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2323640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2288003a10 0x7f2288003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 -- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2288003a10 msgr2=0x7f2288003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a1b22640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2288004810 0x7f2288004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2288003a10 0x7f2288003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 -- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2288004810 msgr2=0x7f2288004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2288004810 0x7f2288004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 -- 172.21.15.43:0/1356416478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288003530 con 0x7f22880051d0 2024-03-30T13:12:24.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2323640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2288003a10 0x7f2288003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 -- 172.21.15.43:0/1356416478 reap_dead start 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a2b24640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f228800b9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2294009fc0 tx=0x7f229401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb554c742ccb390f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294009c10 con 0x7f22880051d0 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22940a5cd0 con 0x7f22880051d0 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a1321640 1 -- 172.21.15.43:0/1356416478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294003be0 con 0x7f22880051d0 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a4587640 1 -- 172.21.15.43:0/1356416478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 msgr2=0x7f228800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a4587640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f228800b9d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2294009fc0 tx=0x7f229401ccc0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a4587640 1 -- 172.21.15.43:0/1356416478 shutdown_connections 2024-03-30T13:12:24.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a4587640 1 --2- 172.21.15.43:0/1356416478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f228800b9d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a4587640 1 -- 172.21.15.43:0/1356416478 >> 172.21.15.43:0/1356416478 conn(0x7f229c0cfee0 msgr2=0x7f229c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:24.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a4587640 1 -- 172.21.15.43:0/1356416478 shutdown_connections 2024-03-30T13:12:24.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.003+0000 7f22a4587640 1 -- 172.21.15.43:0/1356416478 wait complete. 2024-03-30T13:12:24.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 Processor -- start 2024-03-30T13:12:24.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 -- start start 2024-03-30T13:12:24.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f229c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f229c15f500 0x7f229c151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229c1f3f80 0x7f229c13acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f229c145e80 con 0x7f22880051d0 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f229c0ccd20 con 0x7f229c1f3f80 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f229c146000 con 0x7f229c15f500 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f229c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f229c15c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52680/0 (socket says 172.21.15.43:52680) 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 -- 172.21.15.43:0/813105086 learned_addr learned my addr 172.21.15.43:0/813105086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 -- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f229c15f500 msgr2=0x7f229c151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a1b22640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f229c15f500 0x7f229c151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2b24640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229c1f3f80 0x7f229c13acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f229c15f500 0x7f229c151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 -- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229c1f3f80 msgr2=0x7f229c13acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229c1f3f80 0x7f229c13acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 -- 172.21.15.43:0/813105086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f229c1521a0 con 0x7f22880051d0 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a1b22640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f229c15f500 0x7f229c151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2b24640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229c1f3f80 0x7f229c13acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a2323640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f229c15c9e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f229800d7d0 tx=0x7f229800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f2292ffd640 1 -- 172.21.15.43:0/813105086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298019070 con 0x7f22880051d0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f2292ffd640 1 -- 172.21.15.43:0/813105086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22980072d0 con 0x7f22880051d0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f229c15e460 con 0x7f22880051d0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f2292ffd640 1 -- 172.21.15.43:0/813105086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298007680 con 0x7f22880051d0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f229c15ea20 con 0x7f22880051d0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f229c15f140 con 0x7f22880051d0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f2292ffd640 1 -- 172.21.15.43:0/813105086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f229801c070 con 0x7f22880051d0 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.007+0000 7f2292ffd640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2280067ff0 0x7f228006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.011+0000 7f2292ffd640 1 -- 172.21.15.43:0/813105086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f2298085ca0 con 0x7f22880051d0 2024-03-30T13:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.011+0000 7f22a1b22640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2280067ff0 0x7f228006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.011+0000 7f22a1b22640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2280067ff0 0x7f228006a4b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f229c1f7c90 tx=0x7f228c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.015+0000 7f2292ffd640 1 -- 172.21.15.43:0/813105086 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f22980566a0 con 0x7f22880051d0 2024-03-30T13:12:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.239+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f229c068890 con 0x7f22880051d0 2024-03-30T13:12:24.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f2292ffd640 1 -- 172.21.15.43:0/813105086 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) ==== 97+0+0 (secure 0 0 0) 0x7f2298055fa0 con 0x7f22880051d0 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2280067ff0 msgr2=0x7f228006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2280067ff0 0x7f228006a4b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f229c1f7c90 tx=0x7f228c009040 comp rx=0 tx=0).stop 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 msgr2=0x7f229c15c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f229c15c9e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f229800d7d0 tx=0x7f229800dc90 comp rx=0 tx=0).stop 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 shutdown_connections 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2280067ff0 0x7f228006a4b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f229c1f3f80 0x7f229c13acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f229c15f500 0x7f229c151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 --2- 172.21.15.43:0/813105086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22880051d0 0x7f229c15c9e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 >> 172.21.15.43:0/813105086 conn(0x7f229c0cfee0 msgr2=0x7f229c140e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 shutdown_connections 2024-03-30T13:12:24.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.243+0000 7f22a4587640 1 -- 172.21.15.43:0/813105086 wait complete. 2024-03-30T13:12:24.270 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-30T13:12:24.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.383+0000 7f2d76815640 1 Processor -- start 2024-03-30T13:12:24.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.383+0000 7f2d76815640 1 -- start start 2024-03-30T13:12:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d76815640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2d7006c670 0x7f2d700cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d76815640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 0x7f2d700ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d76815640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d700ccef0 con 0x7f2d700b41f0 2024-03-30T13:12:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d76815640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2d700b34c0 con 0x7f2d7006c670 2024-03-30T13:12:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d76815640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2d700b3710 con 0x7f2d7006ca50 2024-03-30T13:12:24.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2d700b41f0 0x7f2d7006e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40934/0 (socket says 172.21.15.43:40934) 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6f7fe640 1 -- 172.21.15.43:0/185690559 learned_addr learned my addr 172.21.15.43:0/185690559 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d74db2640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 0x7f2d700ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/185690559 >> v2:172.21.15.113:3300/0 conn(0x7f2d7006c670 0x7f2d700cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1565627243 0 0) 0x7f2d700ccef0 con 0x7f2d700b41f0 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d5c003530 con 0x7f2d700b41f0 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3347392797 0 0) 0x7f2d5c003530 con 0x7f2d700b41f0 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 msgr2=0x7f2d700ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 0x7f2d700ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 >> v2:172.21.15.113:3300/0 conn(0x7f2d7006c670 msgr2=0x7f2d700cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 --2- 172.21.15.43:0/185690559 >> v2:172.21.15.113:3300/0 conn(0x7f2d7006c670 0x7f2d700cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d700b39f0 con 0x7f2d700b41f0 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f2d58001680 con 0x7f2d700b41f0 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 >> v1:172.21.15.43:6789/0 conn(0x7f2d700b41f0 legacy=0x7f2d7006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d5c0039c0 0x7f2d5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 0x7f2d5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c004d50 0x7f2d5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700b34c0 con 0x7f2d5c0039c0 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700b3710 con 0x7f2d5c004d50 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5c00b9d0 con 0x7f2d7006ca50 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d5c0039c0 0x7f2d5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d74db2640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c004d50 0x7f2d5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6f7fe640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 0x7f2d5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6ffff640 1 -- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 msgr2=0x7f2d5c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7006ca50 0x7f2d5c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6ffff640 1 -- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c004d50 msgr2=0x7f2d5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c004d50 0x7f2d5c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.387+0000 7f2d6ffff640 1 -- 172.21.15.43:0/185690559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5c003530 con 0x7f2d5c0039c0 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d5c0039c0 0x7f2d5c003de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009fc0 tx=0x7f2d6001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236033ac7440dcf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600b2070 con 0x7f2d5c0039c0 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d600a5b40 con 0x7f2d5c0039c0 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6effd640 1 -- 172.21.15.43:0/185690559 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60009c10 con 0x7f2d5c0039c0 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- 172.21.15.43:0/185690559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d5c0039c0 msgr2=0x7f2d5c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 --2- 172.21.15.43:0/185690559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d5c0039c0 0x7f2d5c003de0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009fc0 tx=0x7f2d6001ccc0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d74db2640 1 -- 172.21.15.43:0/185690559 reap_dead start 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- 172.21.15.43:0/185690559 shutdown_connections 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- 172.21.15.43:0/185690559 >> 172.21.15.43:0/185690559 conn(0x7f2d700cfee0 msgr2=0x7f2d700d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- 172.21.15.43:0/185690559 shutdown_connections 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- 172.21.15.43:0/185690559 wait complete. 2024-03-30T13:12:24.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 Processor -- start 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- start start 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c0039c0 0x7f2d7015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7015f380 0x7f2d7015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7015ddf0 0x7f2d7013e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700cef40 con 0x7f2d7015f380 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2d700cedc0 con 0x7f2d5c0039c0 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d76815640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2d700cf0c0 con 0x7f2d7015ddf0 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7015f380 0x7f2d7015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7015f380 0x7f2d7015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52702/0 (socket says 172.21.15.43:52702) 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 -- 172.21.15.43:0/227026829 learned_addr learned my addr 172.21.15.43:0/227026829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c0039c0 0x7f2d7015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d74db2640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7015ddf0 0x7f2d7013e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 -- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7015ddf0 msgr2=0x7f2d7013e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7015ddf0 0x7f2d7013e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 -- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c0039c0 msgr2=0x7f2d7015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c0039c0 0x7f2d7015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 -- 172.21.15.43:0/227026829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7015c190 con 0x7f2d7015f380 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.391+0000 7f2d6f7fe640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7015f380 0x7f2d7015d8b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e850 tx=0x7f2d5800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6cff9640 1 -- 172.21.15.43:0/227026829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800c870 con 0x7f2d7015f380 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6cff9640 1 -- 172.21.15.43:0/227026829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d58018070 con 0x7f2d7015f380 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7015e2e0 con 0x7f2d7015f380 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6cff9640 1 -- 172.21.15.43:0/227026829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800cc70 con 0x7f2d7015f380 2024-03-30T13:12:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d7015e8a0 con 0x7f2d7015f380 2024-03-30T13:12:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d70068890 con 0x7f2d7015f380 2024-03-30T13:12:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6cff9640 1 -- 172.21.15.43:0/227026829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2d5801d460 con 0x7f2d7015f380 2024-03-30T13:12:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6cff9640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2d50067fc0 0x7f2d5006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6cff9640 1 -- 172.21.15.43:0/227026829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f2d58086210 con 0x7f2d7015f380 2024-03-30T13:12:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2d50067fc0 0x7f2d5006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.395+0000 7f2d6ffff640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2d50067fc0 0x7f2d5006a480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d600af000 tx=0x7f2d60004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.399+0000 7f2d6cff9640 1 -- 172.21.15.43:0/227026829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2d58056b80 con 0x7f2d7015f380 2024-03-30T13:12:24.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.619+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f2d700b32a0 con 0x7f2d7015f380 2024-03-30T13:12:24.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.623+0000 7f2d6cff9640 1 -- 172.21.15.43:0/227026829 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) ==== 90+0+0 (secure 0 0 0) 0x7f2d58056480 con 0x7f2d7015f380 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2d50067fc0 msgr2=0x7f2d5006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2d50067fc0 0x7f2d5006a480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2d600af000 tx=0x7f2d60004ac0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7015f380 msgr2=0x7f2d7015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7015f380 0x7f2d7015d8b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e850 tx=0x7f2d5800ed10 comp rx=0 tx=0).stop 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 shutdown_connections 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2d50067fc0 0x7f2d5006a480 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2d7015ddf0 0x7f2d7013e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2d7015f380 0x7f2d7015d8b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 --2- 172.21.15.43:0/227026829 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2d5c0039c0 0x7f2d7015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 >> 172.21.15.43:0/227026829 conn(0x7f2d700cfee0 msgr2=0x7f2d7013dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 shutdown_connections 2024-03-30T13:12:24.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.627+0000 7f2d76815640 1 -- 172.21.15.43:0/227026829 wait complete. 2024-03-30T13:12:24.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:24.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.755+0000 7fe0f4d81640 1 Processor -- start 2024-03-30T13:12:24.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.755+0000 7fe0f4d81640 1 -- start start 2024-03-30T13:12:24.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0f4d81640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f0144270 0x7fe0f0144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0f4d81640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0f00b2ef0 0x7fe0f015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0f4d81640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f00b33f0 con 0x7fe0f00b4490 2024-03-30T13:12:24.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0f4d81640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f0144fa0 con 0x7fe0f00b2ef0 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0f4d81640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f015fce0 con 0x7fe0f0144270 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0e5d9b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0f00b2ef0 0x7fe0f015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0ee59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f0144270 0x7fe0f0144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0ee59c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f0144270 0x7fe0f0144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:36972/0 (socket says 172.21.15.43:36972) 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0ee59c640 1 -- 172.21.15.43:0/1091301469 learned_addr learned my addr 172.21.15.43:0/1091301469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0e5d9b640 1 -- 172.21.15.43:0/1091301469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f0144270 msgr2=0x7fe0f0144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0e5d9b640 1 --2- 172.21.15.43:0/1091301469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f0144270 0x7fe0f0144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0e5d9b640 1 -- 172.21.15.43:0/1091301469 >> v1:172.21.15.113:6789/0 conn(0x7fe0f00b4490 legacy=0x7fe0f006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0e5d9b640 1 -- 172.21.15.43:0/1091301469 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f00b36d0 con 0x7fe0f00b2ef0 2024-03-30T13:12:24.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0e5d9b640 1 --2- 172.21.15.43:0/1091301469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f015f7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097c0 tx=0x7fe0d801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ec866a5f2770c66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0edd9b640 1 -- 172.21.15.43:0/1091301469 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80a7070 con 0x7fe0f00b2ef0 2024-03-30T13:12:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0edd9b640 1 -- 172.21.15.43:0/1091301469 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d80b0070 con 0x7fe0f00b2ef0 2024-03-30T13:12:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.759+0000 7fe0edd9b640 1 -- 172.21.15.43:0/1091301469 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d80ab890 con 0x7fe0f00b2ef0 2024-03-30T13:12:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- 172.21.15.43:0/1091301469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 msgr2=0x7fe0f015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/1091301469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f015f7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80097c0 tx=0x7fe0d801c470 comp rx=0 tx=0).stop 2024-03-30T13:12:24.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- 172.21.15.43:0/1091301469 shutdown_connections 2024-03-30T13:12:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/1091301469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f015f7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/1091301469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f0144270 0x7fe0f0144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- 172.21.15.43:0/1091301469 >> 172.21.15.43:0/1091301469 conn(0x7fe0f00cfee0 msgr2=0x7fe0f00d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- 172.21.15.43:0/1091301469 shutdown_connections 2024-03-30T13:12:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- 172.21.15.43:0/1091301469 wait complete. 2024-03-30T13:12:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 Processor -- start 2024-03-30T13:12:24.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- start start 2024-03-30T13:12:24.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f0165a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f00b4490 0x7fe0f0159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0144270 0x7fe0f01404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f00b4000 con 0x7fe0f0144270 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f006cf90 con 0x7fe0f00b2ef0 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f00cdd10 con 0x7fe0f00b4490 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0144270 0x7fe0f01404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0144270 0x7fe0f01404e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52708/0 (socket says 172.21.15.43:52708) 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 -- 172.21.15.43:0/682108365 learned_addr learned my addr 172.21.15.43:0/682108365 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0e5d9b640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f00b4490 0x7fe0f0159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0ee59c640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f0165a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 -- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f00b4490 msgr2=0x7fe0f0159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f00b4490 0x7fe0f0159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 -- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 msgr2=0x7fe0f0165a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f0165a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 -- 172.21.15.43:0/682108365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f0162e50 con 0x7fe0f0144270 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0ee59c640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f0165a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0eed9d640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0144270 0x7fe0f01404e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c770 tx=0x7fe0e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0e77fe640 1 -- 172.21.15.43:0/682108365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8003b80 con 0x7fe0f0144270 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0e77fe640 1 -- 172.21.15.43:0/682108365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e8003d20 con 0x7fe0f0144270 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f0163130 con 0x7fe0f0144270 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0e77fe640 1 -- 172.21.15.43:0/682108365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800aa90 con 0x7fe0f0144270 2024-03-30T13:12:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.763+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01636f0 con 0x7fe0f0144270 2024-03-30T13:12:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.767+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f0068890 con 0x7fe0f0144270 2024-03-30T13:12:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.767+0000 7fe0e77fe640 1 -- 172.21.15.43:0/682108365 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe0e800ac30 con 0x7fe0f0144270 2024-03-30T13:12:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.767+0000 7fe0e77fe640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0d0067fc0 0x7fe0d006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.767+0000 7fe0e77fe640 1 -- 172.21.15.43:0/682108365 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7fe0e80862f0 con 0x7fe0f0144270 2024-03-30T13:12:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.767+0000 7fe0ee59c640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0d0067fc0 0x7fe0d006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.767+0000 7fe0ee59c640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0d0067fc0 0x7fe0d006a480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe0f01622c0 tx=0x7fe0e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:24.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.771+0000 7fe0e77fe640 1 -- 172.21.15.43:0/682108365 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe0e8056bc0 con 0x7fe0f0144270 2024-03-30T13:12:24.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.939+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe0f015f150 con 0x7fe0f0144270 2024-03-30T13:12:24.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.939+0000 7fe0e77fe640 1 -- 172.21.15.43:0/682108365 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13140 (secure 0 0 0) 0x7fe0e80564c0 con 0x7fe0f0144270 2024-03-30T13:12:24.947 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:24.947 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":37,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:12:21.961841+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0d0067fc0 msgr2=0x7fe0d006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0d0067fc0 0x7fe0d006a480 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe0f01622c0 tx=0x7fe0e0008040 comp rx=0 tx=0).stop 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0144270 msgr2=0x7fe0f01404e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0144270 0x7fe0f01404e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c770 tx=0x7fe0e800cc30 comp rx=0 tx=0).stop 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 shutdown_connections 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0d0067fc0 0x7fe0d006a480 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0f0144270 0x7fe0f01404e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0f00b4490 0x7fe0f0159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 --2- 172.21.15.43:0/682108365 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0f00b2ef0 0x7fe0f0165a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 >> 172.21.15.43:0/682108365 conn(0x7fe0f00cfee0 msgr2=0x7fe0f0067400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 shutdown_connections 2024-03-30T13:12:24.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:24.943+0000 7fe0f4d81640 1 -- 172.21.15.43:0/682108365 wait complete. 2024-03-30T13:12:24.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:12:24.966 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:12:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.087+0000 7f8380763640 1 Processor -- start 2024-03-30T13:12:25.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.087+0000 7f8380763640 1 -- start start 2024-03-30T13:12:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.087+0000 7f8380763640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 0x7f837815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.087+0000 7f8380763640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8378162830 con 0x7f8378155160 2024-03-30T13:12:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f8380763640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8378162b10 con 0x7f837806cd80 2024-03-30T13:12:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f8380763640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8378162df0 con 0x7f83780b2e80 2024-03-30T13:12:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837e4ff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8378155160 0x7f837806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40950/0 (socket says 172.21.15.43:40950) 2024-03-30T13:12:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837e4ff640 1 -- 172.21.15.43:0/3826911620 learned_addr learned my addr 172.21.15.43:0/3826911620 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 0x7f837815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 608052676 0 0) 0x7f8378162830 con 0x7f8378155160 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8364003530 con 0x7f8378155160 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3055032105 0 0) 0x7f8364003530 con 0x7f8378155160 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 msgr2=0x7f837815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 0x7f837815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 >> v1:172.21.15.113:6789/0 conn(0x7f837806cd80 legacy=0x7f8378071ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83780719e0 con 0x7f8378155160 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f8374001680 con 0x7f8378155160 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 >> v1:172.21.15.43:6789/0 conn(0x7f8378155160 legacy=0x7f837806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83640039f0 0x7f8364003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 0x7f8364004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8364004d50 0x7f83640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8378162df0 con 0x7f8364004d50 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f836400ba80 con 0x7f83640039f0 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f836400bc80 con 0x7f83780b2e80 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8364004d50 0x7f83640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837dcfe640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 0x7f8364004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837e4ff640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83640039f0 0x7f8364003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 -- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 msgr2=0x7f8364004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f83780b2e80 0x7f8364004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 -- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83640039f0 msgr2=0x7f8364003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f83640039f0 0x7f8364003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 -- 172.21.15.43:0/3826911620 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8364003530 con 0x7f8364004d50 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837ed00640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8364004d50 0x7f83640051d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8370009f90 tx=0x7f837001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464b2ae948b0f10d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83700b2070 con 0x7f8364004d50 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83700a5c50 con 0x7f8364004d50 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.091+0000 7f837d4fd640 1 -- 172.21.15.43:0/3826911620 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370009c10 con 0x7f8364004d50 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/3826911620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8364004d50 msgr2=0x7f83640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 --2- 172.21.15.43:0/3826911620 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8364004d50 0x7f83640051d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8370009f90 tx=0x7f837001cc40 comp rx=0 tx=0).stop 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837ed00640 1 -- 172.21.15.43:0/3826911620 reap_dead start 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/3826911620 shutdown_connections 2024-03-30T13:12:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/3826911620 >> 172.21.15.43:0/3826911620 conn(0x7f83780d00a0 msgr2=0x7f83780d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/3826911620 shutdown_connections 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/3826911620 wait complete. 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 Processor -- start 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- start start 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f837815d5f0 0x7f8378152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815d9d0 0x7f83781582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8378158810 0x7f8378155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f837815f760 con 0x7f837815d5f0 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f837815f5e0 con 0x7f837815d9d0 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f837815f8e0 con 0x7f8378158810 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f837815d5f0 0x7f8378152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f837815d5f0 0x7f8378152bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52728/0 (socket says 172.21.15.43:52728) 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 -- 172.21.15.43:0/232754235 learned_addr learned my addr 172.21.15.43:0/232754235 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837ed00640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8378158810 0x7f8378155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837dcfe640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815d9d0 0x7f83781582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 -- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8378158810 msgr2=0x7f8378155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8378158810 0x7f8378155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 -- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815d9d0 msgr2=0x7f83781582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815d9d0 0x7f83781582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 -- 172.21.15.43:0/232754235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8378151b30 con 0x7f837815d5f0 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837dcfe640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815d9d0 0x7f83781582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837ed00640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8378158810 0x7f8378155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f837e4ff640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f837815d5f0 0x7f8378152bd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8374001060 tx=0x7f837400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f836effd640 1 -- 172.21.15.43:0/232754235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837400c8d0 con 0x7f837815d5f0 2024-03-30T13:12:25.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f836effd640 1 -- 172.21.15.43:0/232754235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8374018070 con 0x7f837815d5f0 2024-03-30T13:12:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8378151e10 con 0x7f837815d5f0 2024-03-30T13:12:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f836effd640 1 -- 172.21.15.43:0/232754235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837400ccb0 con 0x7f837815d5f0 2024-03-30T13:12:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83781523d0 con 0x7f837815d5f0 2024-03-30T13:12:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.095+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8378068900 con 0x7f837815d5f0 2024-03-30T13:12:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.099+0000 7f836effd640 1 -- 172.21.15.43:0/232754235 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f837401d460 con 0x7f837815d5f0 2024-03-30T13:12:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.099+0000 7f836effd640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f835c067fc0 0x7f835c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.099+0000 7f836effd640 1 -- 172.21.15.43:0/232754235 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f8374086ea0 con 0x7f837815d5f0 2024-03-30T13:12:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.099+0000 7f837dcfe640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f835c067fc0 0x7f835c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.099+0000 7f837dcfe640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f835c067fc0 0x7f835c06a480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f837815dff0 tx=0x7f8368009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.103+0000 7f836effd640 1 -- 172.21.15.43:0/232754235 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f83740577f0 con 0x7f837815d5f0 2024-03-30T13:12:25.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.263+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83780af040 con 0x7f837815d5f0 2024-03-30T13:12:25.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f836effd640 1 -- 172.21.15.43:0/232754235 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f83740570f0 con 0x7f837815d5f0 2024-03-30T13:12:25.273 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f835c067fc0 msgr2=0x7f835c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f835c067fc0 0x7f835c06a480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f837815dff0 tx=0x7f8368009040 comp rx=0 tx=0).stop 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f837815d5f0 msgr2=0x7f8378152bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f837815d5f0 0x7f8378152bd0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8374001060 tx=0x7f837400ed50 comp rx=0 tx=0).stop 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 shutdown_connections 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f835c067fc0 0x7f835c06a480 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8378158810 0x7f8378155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f837815d9d0 0x7f83781582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 --2- 172.21.15.43:0/232754235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f837815d5f0 0x7f8378152bd0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 >> 172.21.15.43:0/232754235 conn(0x7f83780d00a0 msgr2=0x7f837806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 shutdown_connections 2024-03-30T13:12:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.267+0000 7f8380763640 1 -- 172.21.15.43:0/232754235 wait complete. 2024-03-30T13:12:25.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-30T13:12:25.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f133db4d640 1 Processor -- start 2024-03-30T13:12:25.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f133db4d640 1 -- start start 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f133db4d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 0x7f13380b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f133db4d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1338072cb0 con 0x7f133815a840 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f133db4d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1338072f90 con 0x7f133806ea60 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f133db4d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1338073270 con 0x7f133815eb10 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f13377fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f133815a840 0x7f133815e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40962/0 (socket says 172.21.15.43:40962) 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f13377fe640 1 -- 172.21.15.43:0/1644575323 learned_addr learned my addr 172.21.15.43:0/1644575323 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f1336ffd640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 0x7f13380b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3195814785 0 0) 0x7f1338072cb0 con 0x7f133815a840 2024-03-30T13:12:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.411+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1320003530 con 0x7f133815a840 2024-03-30T13:12:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 872599877 0 0) 0x7f1320003530 con 0x7f133815a840 2024-03-30T13:12:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 msgr2=0x7f13380b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 0x7f13380b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 >> v1:172.21.15.113:6789/0 conn(0x7f133806ea60 legacy=0x7f13380b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1338073550 con 0x7f133815a840 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f1328001680 con 0x7f133815a840 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 >> v1:172.21.15.43:6789/0 conn(0x7f133815a840 legacy=0x7f133815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13200039f0 0x7f1320003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 0x7f1320004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1320004d50 0x7f13200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1338073270 con 0x7f1320004d50 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f132000cc10 con 0x7f13200039f0 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1320005a20 con 0x7f133815eb10 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1320004d50 0x7f13200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1336ffd640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 0x7f1320004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13377fe640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13200039f0 0x7f1320003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 -- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 msgr2=0x7f1320004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f133815eb10 0x7f1320004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 -- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13200039f0 msgr2=0x7f1320003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f13200039f0 0x7f1320003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 -- 172.21.15.43:0/1644575323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1320003530 con 0x7f1320004d50 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1320004d50 0x7f13200051d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1330001e40 tx=0x7f133001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6ebdd9ef2b246f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330001a90 con 0x7f1320004d50 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1330003b80 con 0x7f1320004d50 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f13367fc640 1 -- 172.21.15.43:0/1644575323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330005080 con 0x7f1320004d50 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f133db4d640 1 -- 172.21.15.43:0/1644575323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1320004d50 msgr2=0x7f13200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f133db4d640 1 --2- 172.21.15.43:0/1644575323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1320004d50 0x7f13200051d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1330001e40 tx=0x7f133001c760 comp rx=0 tx=0).stop 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f1337fff640 1 -- 172.21.15.43:0/1644575323 reap_dead start 2024-03-30T13:12:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f133db4d640 1 -- 172.21.15.43:0/1644575323 shutdown_connections 2024-03-30T13:12:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f133db4d640 1 -- 172.21.15.43:0/1644575323 >> 172.21.15.43:0/1644575323 conn(0x7f13380cfee0 msgr2=0x7f13380d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f133db4d640 1 -- 172.21.15.43:0/1644575323 shutdown_connections 2024-03-30T13:12:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.415+0000 7f133db4d640 1 -- 172.21.15.43:0/1644575323 wait complete. 2024-03-30T13:12:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 Processor -- start 2024-03-30T13:12:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 -- start start 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320004d50 0x7f133815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338147e60 0x7f1338146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1338147910 0x7f13381591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f133815f270 con 0x7f1338147e60 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f133815f0f0 con 0x7f1338147910 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f13380728d0 con 0x7f1320004d50 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338147e60 0x7f1338146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338147e60 0x7f1338146b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52756/0 (socket says 172.21.15.43:52756) 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 -- 172.21.15.43:0/2787874432 learned_addr learned my addr 172.21.15.43:0/2787874432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f13377fe640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320004d50 0x7f133815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1337fff640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1338147910 0x7f13381591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 -- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320004d50 msgr2=0x7f133815ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320004d50 0x7f133815ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 -- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1338147910 msgr2=0x7f13381591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1338147910 0x7f13381591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 -- 172.21.15.43:0/2787874432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1338149000 con 0x7f1338147e60 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f1336ffd640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338147e60 0x7f1338146b70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f132c0115e0 tx=0x7f132c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f131ffff640 1 -- 172.21.15.43:0/2787874432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c015870 con 0x7f1338147e60 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f131ffff640 1 -- 172.21.15.43:0/2787874432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132c015a10 con 0x7f1338147e60 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13380ceb70 con 0x7f1338147e60 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f131ffff640 1 -- 172.21.15.43:0/2787874432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c015de0 con 0x7f1338147e60 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13381473c0 con 0x7f1338147e60 2024-03-30T13:12:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1338068890 con 0x7f1338147e60 2024-03-30T13:12:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.419+0000 7f131ffff640 1 -- 172.21.15.43:0/2787874432 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f132c00f480 con 0x7f1338147e60 2024-03-30T13:12:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.423+0000 7f131ffff640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1324067ff0 0x7f132406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.423+0000 7f131ffff640 1 -- 172.21.15.43:0/2787874432 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f132c086ba0 con 0x7f1338147e60 2024-03-30T13:12:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.423+0000 7f13377fe640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1324067ff0 0x7f132406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.423+0000 7f13377fe640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1324067ff0 0x7f132406a4b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1328001000 tx=0x7f1328009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.427+0000 7f131ffff640 1 -- 172.21.15.43:0/2787874432 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f132c019030 con 0x7f1338147e60 2024-03-30T13:12:25.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.587+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f13380719d0 con 0x7f1338147e60 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f131ffff640 1 -- 172.21.15.43:0/2787874432 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) ==== 84+0+65 (secure 0 0 0) 0x7f132c056f50 con 0x7f1338147e60 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:25.597 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-03-30T13:12:25.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1324067ff0 msgr2=0x7f132406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1324067ff0 0x7f132406a4b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1328001000 tx=0x7f1328009040 comp rx=0 tx=0).stop 2024-03-30T13:12:25.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338147e60 msgr2=0x7f1338146b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338147e60 0x7f1338146b70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f132c0115e0 tx=0x7f132c011aa0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 shutdown_connections 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1324067ff0 0x7f132406a4b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1338147910 0x7f13381591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1338147e60 0x7f1338146b70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 --2- 172.21.15.43:0/2787874432 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1320004d50 0x7f133815ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 >> 172.21.15.43:0/2787874432 conn(0x7f13380cfee0 msgr2=0x7f13380b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 shutdown_connections 2024-03-30T13:12:25.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.591+0000 7f133db4d640 1 -- 172.21.15.43:0/2787874432 wait complete. 2024-03-30T13:12:25.615 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-30T13:12:25.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-30T13:12:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461f0ea640 1 Processor -- start 2024-03-30T13:12:25.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461f0ea640 1 -- start start 2024-03-30T13:12:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461f0ea640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f46181449a0 0x7f461815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461f0ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f461815f220 0x7f46180ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461f0ea640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4618144d80 con 0x7f46180ce860 2024-03-30T13:12:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461f0ea640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f461815fc80 con 0x7f461815f220 2024-03-30T13:12:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461f0ea640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f461815f6c0 con 0x7f46181449a0 2024-03-30T13:12:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f4617fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f461815f220 0x7f46180ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461d687640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f46180ce860 0x7f4618072200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40978/0 (socket says 172.21.15.43:40978) 2024-03-30T13:12:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f4617fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f461815f220 0x7f46180ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52768/0 (socket says 172.21.15.43:52768) 2024-03-30T13:12:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.735+0000 7f461d687640 1 -- 172.21.15.43:0/33593433 learned_addr learned my addr 172.21.15.43:0/33593433 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461ce86640 1 --2- 172.21.15.43:0/33593433 >> v2:172.21.15.113:3300/0 conn(0x7f46181449a0 0x7f461815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f46177fe640 1 -- 172.21.15.43:0/33593433 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 675503011 0 0) 0x7f4618144d80 con 0x7f46180ce860 2024-03-30T13:12:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f46177fe640 1 -- 172.21.15.43:0/33593433 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4608003530 con 0x7f46180ce860 2024-03-30T13:12:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f4617fff640 1 -- 172.21.15.43:0/33593433 >> v2:172.21.15.113:3300/0 conn(0x7f46181449a0 msgr2=0x7f461815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f4617fff640 1 --2- 172.21.15.43:0/33593433 >> v2:172.21.15.113:3300/0 conn(0x7f46181449a0 0x7f461815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f4617fff640 1 -- 172.21.15.43:0/33593433 >> v1:172.21.15.43:6789/0 conn(0x7f46180ce860 legacy=0x7f4618072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f4617fff640 1 -- 172.21.15.43:0/33593433 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461815f9a0 con 0x7f461815f220 2024-03-30T13:12:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f4617fff640 1 --2- 172.21.15.43:0/33593433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f46180ce320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f461815fe00 tx=0x7f460401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa73f57e9c276e19 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f46177fe640 1 -- 172.21.15.43:0/33593433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040b1070 con 0x7f461815f220 2024-03-30T13:12:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f46177fe640 1 -- 172.21.15.43:0/33593433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46040ac640 con 0x7f461815f220 2024-03-30T13:12:25.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f46177fe640 1 -- 172.21.15.43:0/33593433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46040aca10 con 0x7f461815f220 2024-03-30T13:12:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 -- 172.21.15.43:0/33593433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 msgr2=0x7f46180ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 --2- 172.21.15.43:0/33593433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f46180ce320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f461815fe00 tx=0x7f460401c310 comp rx=0 tx=0).stop 2024-03-30T13:12:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 -- 172.21.15.43:0/33593433 shutdown_connections 2024-03-30T13:12:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 --2- 172.21.15.43:0/33593433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f46180ce320 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 --2- 172.21.15.43:0/33593433 >> v2:172.21.15.113:3300/0 conn(0x7f46181449a0 0x7f461815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 -- 172.21.15.43:0/33593433 >> 172.21.15.43:0/33593433 conn(0x7f46180b1b80 msgr2=0x7f46180b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 -- 172.21.15.43:0/33593433 shutdown_connections 2024-03-30T13:12:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.739+0000 7f461f0ea640 1 -- 172.21.15.43:0/33593433 wait complete. 2024-03-30T13:12:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 Processor -- start 2024-03-30T13:12:25.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 -- start start 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46180ce860 0x7f461813f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46181449a0 0x7f461815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f461815cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f461806d670 con 0x7f461815f220 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f46180cf220 con 0x7f46181449a0 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f461806d7f0 con 0x7f46180ce860 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f461815cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f461815cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52784/0 (socket says 172.21.15.43:52784) 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 -- 172.21.15.43:0/3368600227 learned_addr learned my addr 172.21.15.43:0/3368600227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461ce86640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46180ce860 0x7f461813f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f4617fff640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46181449a0 0x7f461815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 -- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46180ce860 msgr2=0x7f461813f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46180ce860 0x7f461813f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 -- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46181449a0 msgr2=0x7f461815a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46181449a0 0x7f461815a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 -- 172.21.15.43:0/3368600227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461813e690 con 0x7f461815f220 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461d687640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f461815cb20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f461000cb40 tx=0x7f461000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f46157fa640 1 -- 172.21.15.43:0/3368600227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610017070 con 0x7f461815f220 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f46157fa640 1 -- 172.21.15.43:0/3368600227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461000aca0 con 0x7f461815f220 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f46157fa640 1 -- 172.21.15.43:0/3368600227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610012620 con 0x7f461815f220 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461813e970 con 0x7f461815f220 2024-03-30T13:12:25.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461813ef30 con 0x7f461815f220 2024-03-30T13:12:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.743+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46180687f0 con 0x7f461815f220 2024-03-30T13:12:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.747+0000 7f46157fa640 1 -- 172.21.15.43:0/3368600227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f461001a070 con 0x7f461815f220 2024-03-30T13:12:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.747+0000 7f46157fa640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45fc067fc0 0x7f45fc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.747+0000 7f46157fa640 1 -- 172.21.15.43:0/3368600227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5249+0+0 (secure 0 0 0) 0x7f4610086060 con 0x7f461815f220 2024-03-30T13:12:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.747+0000 7f461ce86640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45fc067fc0 0x7f45fc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.747+0000 7f461ce86640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45fc067fc0 0x7f45fc06a480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f460c009b60 tx=0x7f460c009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:25.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.751+0000 7f46157fa640 1 -- 172.21.15.43:0/3368600227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4610056380 con 0x7f461815f220 2024-03-30T13:12:25.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.911+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f461806f260 con 0x7f461815f220 2024-03-30T13:12:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f46157fa640 1 -- 172.21.15.43:0/3368600227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) ==== 138+0+0 (secure 0 0 0) 0x7f461005a680 con 0x7f461815f220 2024-03-30T13:12:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-03-30T13:12:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45fc067fc0 msgr2=0x7f45fc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45fc067fc0 0x7f45fc06a480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f460c009b60 tx=0x7f460c009190 comp rx=0 tx=0).stop 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 msgr2=0x7f461815cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f461815cb20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f461000cb40 tx=0x7f461000a060 comp rx=0 tx=0).stop 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 shutdown_connections 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45fc067fc0 0x7f45fc06a480 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f461815f220 0x7f461815cb20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f46181449a0 0x7f461815a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 --2- 172.21.15.43:0/3368600227 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f46180ce860 0x7f461813f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 >> 172.21.15.43:0/3368600227 conn(0x7f46180b1b80 msgr2=0x7f46180b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 shutdown_connections 2024-03-30T13:12:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:25.963+0000 7f461f0ea640 1 -- 172.21.15.43:0/3368600227 wait complete. 2024-03-30T13:12:25.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-30T13:12:26.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f8a5d640 1 Processor -- start 2024-03-30T13:12:26.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f8a5d640 1 -- start start 2024-03-30T13:12:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f8a5d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f52f415d130 0x7f52f406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f8a5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 0x7f52f406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f8a5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52f4142550 con 0x7f52f406c2e0 2024-03-30T13:12:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f8a5d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f52f40cf830 con 0x7f52f415d130 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f8a5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f52f406b800 con 0x7f52f4141fa0 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f52f406c2e0 0x7f52f4141890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:40992/0 (socket says 172.21.15.43:40992) 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 -- 172.21.15.43:0/3377690625 learned_addr learned my addr 172.21.15.43:0/3377690625 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f2d9d640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 0x7f52f406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f259c640 1 --2- 172.21.15.43:0/3377690625 >> v2:172.21.15.113:3300/0 conn(0x7f52f415d130 0x7f52f406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2120214766 0 0) 0x7f52f4142550 con 0x7f52f406c2e0 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52dc003530 con 0x7f52f406c2e0 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 887067803 0 0) 0x7f52dc003530 con 0x7f52f406c2e0 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 msgr2=0x7f52f406d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 0x7f52f406d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 >> v2:172.21.15.113:3300/0 conn(0x7f52f415d130 msgr2=0x7f52f406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 --2- 172.21.15.43:0/3377690625 >> v2:172.21.15.113:3300/0 conn(0x7f52f415d130 0x7f52f406d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f406bae0 con 0x7f52f406c2e0 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f52e8001680 con 0x7f52f406c2e0 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 >> v1:172.21.15.43:6789/0 conn(0x7f52f406c2e0 legacy=0x7f52f4141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 0x7f52dc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52dc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc004d50 0x7f52dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52f40cf830 con 0x7f52dc003f40 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52f406b800 con 0x7f52dc004d50 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc00ba20 con 0x7f52f4141fa0 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52dc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f259c640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 0x7f52dc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f2d9d640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc004d50 0x7f52dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 -- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 msgr2=0x7f52dc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f4141fa0 0x7f52dc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 -- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc004d50 msgr2=0x7f52dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52dc004d50 0x7f52dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.115+0000 7f52f1d9b640 1 -- 172.21.15.43:0/3377690625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc003530 con 0x7f52dc003f40 2024-03-30T13:12:26.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f1d9b640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52dc004810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c240 tx=0x7f52e801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2196244f8ddb7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8007d40 con 0x7f52dc003f40 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e80a9020 con 0x7f52dc003f40 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f159a640 1 -- 172.21.15.43:0/3377690625 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e8005ae0 con 0x7f52dc003f40 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3377690625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 msgr2=0x7f52dc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f8a5d640 1 --2- 172.21.15.43:0/3377690625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52dc004810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f52e800c240 tx=0x7f52e801cc60 comp rx=0 tx=0).stop 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f2d9d640 1 -- 172.21.15.43:0/3377690625 reap_dead start 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3377690625 shutdown_connections 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3377690625 >> 172.21.15.43:0/3377690625 conn(0x7f52f40b1c60 msgr2=0x7f52f40b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3377690625 shutdown_connections 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3377690625 wait complete. 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.119+0000 7f52f8a5d640 1 Processor -- start 2024-03-30T13:12:26.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 -- start start 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52f4154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52f40b36c0 0x7f52f4151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f415b170 0x7f52f415b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f52f40cf3c0 con 0x7f52dc003f40 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f52f40cf240 con 0x7f52f40b36c0 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f52f40cf540 con 0x7f52f415b170 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52f4154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52f4154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52802/0 (socket says 172.21.15.43:52802) 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 -- 172.21.15.43:0/3579016892 learned_addr learned my addr 172.21.15.43:0/3579016892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f2d9d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f415b170 0x7f52f415b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f1d9b640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52f40b36c0 0x7f52f4151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 -- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f415b170 msgr2=0x7f52f415b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f415b170 0x7f52f415b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 -- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52f40b36c0 msgr2=0x7f52f4151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52f40b36c0 0x7f52f4151fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 -- 172.21.15.43:0/3579016892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f40b3e30 con 0x7f52dc003f40 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f2d9d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f415b170 0x7f52f415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f259c640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52f4154c70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52e4011610 tx=0x7f52e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52e2ffd640 1 -- 172.21.15.43:0/3579016892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4015870 con 0x7f52dc003f40 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52e2ffd640 1 -- 172.21.15.43:0/3579016892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e4015a10 con 0x7f52dc003f40 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52e2ffd640 1 -- 172.21.15.43:0/3579016892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e4015dc0 con 0x7f52dc003f40 2024-03-30T13:12:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f40b4110 con 0x7f52dc003f40 2024-03-30T13:12:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f40b46d0 con 0x7f52dc003f40 2024-03-30T13:12:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f4068900 con 0x7f52dc003f40 2024-03-30T13:12:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52e2ffd640 1 -- 172.21.15.43:0/3579016892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f52e400f480 con 0x7f52dc003f40 2024-03-30T13:12:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52e2ffd640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f52d4067fc0 0x7f52d406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.123+0000 7f52e2ffd640 1 -- 172.21.15.43:0/3579016892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5619+0+0 (secure 0 0 0) 0x7f52e4086e00 con 0x7f52dc003f40 2024-03-30T13:12:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.127+0000 7f52f1d9b640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f52d4067fc0 0x7f52d406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.127+0000 7f52f1d9b640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f52d4067fc0 0x7f52d406a480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f52e80aa000 tx=0x7f52f4142720 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:26.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.127+0000 7f52e2ffd640 1 -- 172.21.15.43:0/3579016892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f52e4019030 con 0x7f52dc003f40 2024-03-30T13:12:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.291+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f52f40d0cb0 con 0x7f52dc003f40 2024-03-30T13:12:26.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:26.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:26.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.967+0000 7f52e2ffd640 1 -- 172.21.15.43:0/3579016892 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) ==== 170+0+0 (secure 0 0 0) 0x7f52e4057670 con 0x7f52dc003f40 2024-03-30T13:12:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-03-30T13:12:26.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.971+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f52d4067fc0 msgr2=0x7f52d406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.971+0000 7f52f8a5d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f52d4067fc0 0x7f52d406a480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f52e80aa000 tx=0x7f52f4142720 comp rx=0 tx=0).stop 2024-03-30T13:12:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.971+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 msgr2=0x7f52f4154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:26.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.971+0000 7f52f8a5d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52f4154c70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52e4011610 tx=0x7f52e4011ad0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.975+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 shutdown_connections 2024-03-30T13:12:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.975+0000 7f52f8a5d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f52d4067fc0 0x7f52d406a480 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.975+0000 7f52f8a5d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f52f415b170 0x7f52f415b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.975+0000 7f52f8a5d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f52f40b36c0 0x7f52f4151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.975+0000 7f52f8a5d640 1 --2- 172.21.15.43:0/3579016892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f52dc003f40 0x7f52f4154c70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.975+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 >> 172.21.15.43:0/3579016892 conn(0x7f52f40b1c60 msgr2=0x7f52f413f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.975+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 shutdown_connections 2024-03-30T13:12:26.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:26.979+0000 7f52f8a5d640 1 -- 172.21.15.43:0/3579016892 wait complete. 2024-03-30T13:12:27.016 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-30T13:12:27.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.135+0000 7f1abfee8640 1 Processor -- start 2024-03-30T13:12:27.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.135+0000 7f1abfee8640 1 -- start start 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abfee8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80b9e30 0x7f1ab80cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abfee8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1ab80cf4f0 0x7f1ab80d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abfee8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab806c850 con 0x7f1ab815e4c0 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abfee8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab80ba290 con 0x7f1ab80cf4f0 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abfee8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab815e8a0 con 0x7f1ab80b9e30 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abdc84640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1ab815e4c0 0x7f1ab806c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41000/0 (socket says 172.21.15.43:41000) 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abdc84640 1 -- 172.21.15.43:0/3409543553 learned_addr learned my addr 172.21.15.43:0/3409543553 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abd483640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80b9e30 0x7f1ab80cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abe485640 1 --2- 172.21.15.43:0/3409543553 >> v2:172.21.15.113:3300/0 conn(0x7f1ab80cf4f0 0x7f1ab80d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228643199 0 0) 0x7f1ab806c850 con 0x7f1ab815e4c0 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1aa4003530 con 0x7f1ab815e4c0 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2157681728 0 0) 0x7f1aa4003530 con 0x7f1ab815e4c0 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80b9e30 msgr2=0x7f1ab80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80b9e30 0x7f1ab80cefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 >> v2:172.21.15.113:3300/0 conn(0x7f1ab80cf4f0 msgr2=0x7f1ab80d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 --2- 172.21.15.43:0/3409543553 >> v2:172.21.15.113:3300/0 conn(0x7f1ab80cf4f0 0x7f1ab80d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab806cb30 con 0x7f1ab815e4c0 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f1ab4001a00 con 0x7f1ab815e4c0 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 >> v1:172.21.15.43:6789/0 conn(0x7f1ab815e4c0 legacy=0x7f1ab806c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa40039c0 0x7f1aa4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80b9e30 0x7f1aa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa4004d50 0x7f1aa40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab815e8a0 con 0x7f1aa4004d50 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa400b9d0 con 0x7f1aa40039c0 2024-03-30T13:12:27.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa400bb50 con 0x7f1ab80b9e30 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abe485640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa4004d50 0x7f1aa40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abdc84640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa40039c0 0x7f1aa4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abe485640 1 -- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80b9e30 msgr2=0x7f1aa4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abe485640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80b9e30 0x7f1aa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abe485640 1 -- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa40039c0 msgr2=0x7f1aa4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abe485640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa40039c0 0x7f1aa4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abe485640 1 -- 172.21.15.43:0/3409543553 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa4003530 con 0x7f1aa4004d50 2024-03-30T13:12:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.139+0000 7f1abdc84640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa40039c0 0x7f1aa4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abe485640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa4004d50 0x7f1aa40051d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ae0 tx=0x7f1ab001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d41fd8319d6583 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00b2070 con 0x7f1aa4004d50 2024-03-30T13:12:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab00a56a0 con 0x7f1aa4004d50 2024-03-30T13:12:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abcc82640 1 -- 172.21.15.43:0/3409543553 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00096b0 con 0x7f1aa4004d50 2024-03-30T13:12:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- 172.21.15.43:0/3409543553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa4004d50 msgr2=0x7f1aa40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 --2- 172.21.15.43:0/3409543553 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1aa4004d50 0x7f1aa40051d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ae0 tx=0x7f1ab001cab0 comp rx=0 tx=0).stop 2024-03-30T13:12:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abe485640 1 -- 172.21.15.43:0/3409543553 reap_dead start 2024-03-30T13:12:27.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- 172.21.15.43:0/3409543553 shutdown_connections 2024-03-30T13:12:27.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- 172.21.15.43:0/3409543553 >> 172.21.15.43:0/3409543553 conn(0x7f1ab80b6410 msgr2=0x7f1ab80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:27.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- 172.21.15.43:0/3409543553 shutdown_connections 2024-03-30T13:12:27.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- 172.21.15.43:0/3409543553 wait complete. 2024-03-30T13:12:27.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 Processor -- start 2024-03-30T13:12:27.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- start start 2024-03-30T13:12:27.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa4004d50 0x7f1ab8159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ab80c98b0 0x7f1ab80b79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80c9d30 0x7f1ab80b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab80cf130 con 0x7f1ab80c98b0 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab80cefb0 con 0x7f1aa4004d50 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab80cf2b0 con 0x7f1ab80c9d30 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ab80c98b0 0x7f1ab80b79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ab80c98b0 0x7f1ab80b79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52824/0 (socket says 172.21.15.43:52824) 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 -- 172.21.15.43:0/1793791979 learned_addr learned my addr 172.21.15.43:0/1793791979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abe485640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80c9d30 0x7f1ab80b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abdc84640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa4004d50 0x7f1ab8159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 -- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80c9d30 msgr2=0x7f1ab80b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80c9d30 0x7f1ab80b7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 -- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa4004d50 msgr2=0x7f1ab8159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:27.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa4004d50 0x7f1ab8159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 -- 172.21.15.43:0/1793791979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab80c8b30 con 0x7f1ab80c98b0 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abdc84640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa4004d50 0x7f1ab8159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abd483640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ab80c98b0 0x7f1ab80b79d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800e7c0 tx=0x7f1aa800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1aae7fc640 1 -- 172.21.15.43:0/1793791979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8017070 con 0x7f1ab80c98b0 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1aae7fc640 1 -- 172.21.15.43:0/1793791979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa800c8c0 con 0x7f1ab80c98b0 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1aae7fc640 1 -- 172.21.15.43:0/1793791979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa800cc90 con 0x7f1ab80c98b0 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab80c8dd0 con 0x7f1ab80c98b0 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.143+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab80c9390 con 0x7f1ab80c98b0 2024-03-30T13:12:27.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.147+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8068900 con 0x7f1ab80c98b0 2024-03-30T13:12:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.147+0000 7f1aae7fc640 1 -- 172.21.15.43:0/1793791979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1aa801a070 con 0x7f1ab80c98b0 2024-03-30T13:12:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.147+0000 7f1aae7fc640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a9c067ff0 0x7f1a9c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.147+0000 7f1aae7fc640 1 -- 172.21.15.43:0/1793791979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6001+0+0 (secure 0 0 0) 0x7f1aa8086480 con 0x7f1ab80c98b0 2024-03-30T13:12:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.147+0000 7f1abdc84640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a9c067ff0 0x7f1a9c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:27.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.147+0000 7f1abdc84640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a9c067ff0 0x7f1a9c06a4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4001b70 tx=0x7f1ab4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.151+0000 7f1aae7fc640 1 -- 172.21.15.43:0/1793791979 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1aa8056b00 con 0x7f1ab80c98b0 2024-03-30T13:12:27.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:27.351+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1ab80b6fd0 con 0x7f1ab80c98b0 2024-03-30T13:12:28.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.007+0000 7f1aae7fc640 1 -- 172.21.15.43:0/1793791979 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f1aa8056400 con 0x7f1ab80c98b0 2024-03-30T13:12:28.025 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 5 and data pool 6 2024-03-30T13:12:28.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.023+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a9c067ff0 msgr2=0x7f1a9c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.023+0000 7f1abfee8640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a9c067ff0 0x7f1a9c06a4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4001b70 tx=0x7f1ab4009040 comp rx=0 tx=0).stop 2024-03-30T13:12:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ab80c98b0 msgr2=0x7f1ab80b79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ab80c98b0 0x7f1ab80b79d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800e7c0 tx=0x7f1aa800ec80 comp rx=0 tx=0).stop 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 shutdown_connections 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1a9c067ff0 0x7f1a9c06a4b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1ab80c9d30 0x7f1ab80b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ab80c98b0 0x7f1ab80b79d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 --2- 172.21.15.43:0/1793791979 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1aa4004d50 0x7f1ab8159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 >> 172.21.15.43:0/1793791979 conn(0x7f1ab80b6410 msgr2=0x7f1ab80b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 shutdown_connections 2024-03-30T13:12:28.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.027+0000 7f1abfee8640 1 -- 172.21.15.43:0/1793791979 wait complete. 2024-03-30T13:12:28.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:28.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f8055a6e640 1 Processor -- start 2024-03-30T13:12:28.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f8055a6e640 1 -- start start 2024-03-30T13:12:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f8055a6e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f805015e600 0x7f80500b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f8055a6e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f80500d1560 0x7f80500d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f8055a6e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80500d3e60 con 0x7f80500d1100 2024-03-30T13:12:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f8055a6e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f80500b7450 con 0x7f80500d1560 2024-03-30T13:12:28.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f8055a6e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f80500d4140 con 0x7f805015e600 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f80500d1560 0x7f80500d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f80500d1100 0x7f80500b8160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41004/0 (socket says 172.21.15.43:41004) 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f80500d1560 0x7f80500d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52836/0 (socket says 172.21.15.43:52836) 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804effd640 1 -- 172.21.15.43:0/4247320374 learned_addr learned my addr 172.21.15.43:0/4247320374 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804f7fe640 1 --2- 172.21.15.43:0/4247320374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f805015e600 0x7f80500b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804ffff640 1 -- 172.21.15.43:0/4247320374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f805015e600 msgr2=0x7f80500b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804ffff640 1 --2- 172.21.15.43:0/4247320374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f805015e600 0x7f80500b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804ffff640 1 -- 172.21.15.43:0/4247320374 >> v1:172.21.15.43:6789/0 conn(0x7f80500d1100 legacy=0x7f80500b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.183+0000 7f804ffff640 1 -- 172.21.15.43:0/4247320374 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80500b7680 con 0x7f80500d1560 2024-03-30T13:12:28.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 --2- 172.21.15.43:0/4247320374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80500d1560 0x7f80500d3610 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8048009d90 tx=0x7f804801c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320fb0db800a2899 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804e7fc640 1 -- 172.21.15.43:0/4247320374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480b2070 con 0x7f80500d1560 2024-03-30T13:12:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804e7fc640 1 -- 172.21.15.43:0/4247320374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80480a5820 con 0x7f80500d1560 2024-03-30T13:12:28.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804e7fc640 1 -- 172.21.15.43:0/4247320374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480a5bf0 con 0x7f80500d1560 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- 172.21.15.43:0/4247320374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80500d1560 msgr2=0x7f80500d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4247320374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80500d1560 0x7f80500d3610 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8048009d90 tx=0x7f804801c800 comp rx=0 tx=0).stop 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- 172.21.15.43:0/4247320374 shutdown_connections 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4247320374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f80500d1560 0x7f80500d3610 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4247320374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f805015e600 0x7f80500b9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- 172.21.15.43:0/4247320374 >> 172.21.15.43:0/4247320374 conn(0x7f80500b6230 msgr2=0x7f80500b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- 172.21.15.43:0/4247320374 shutdown_connections 2024-03-30T13:12:28.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- 172.21.15.43:0/4247320374 wait complete. 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 Processor -- start 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- start start 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80500d1100 0x7f80500ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80500d1560 0x7f8050159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f805015e600 0x7f8050157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f80500d19f0 con 0x7f805015e600 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f80500b7c40 con 0x7f80500d1100 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8055a6e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f80500d3b50 con 0x7f80500d1560 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f805015e600 0x7f8050157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f805015e600 0x7f8050157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52842/0 (socket says 172.21.15.43:52842) 2024-03-30T13:12:28.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 -- 172.21.15.43:0/4143209825 learned_addr learned my addr 172.21.15.43:0/4143209825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804effd640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80500d1560 0x7f8050159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804f7fe640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80500d1100 0x7f80500ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 -- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80500d1560 msgr2=0x7f8050159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80500d1560 0x7f8050159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 -- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80500d1100 msgr2=0x7f80500ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80500d1100 0x7f80500ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 -- 172.21.15.43:0/4143209825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8050161d20 con 0x7f805015e600 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804f7fe640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80500d1100 0x7f80500ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f804ffff640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f805015e600 0x7f8050157730 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f804801d0a0 tx=0x7f80480195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8037fff640 1 -- 172.21.15.43:0/4143209825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80480b2040 con 0x7f805015e600 2024-03-30T13:12:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.187+0000 7f8037fff640 1 -- 172.21.15.43:0/4143209825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80480bf070 con 0x7f805015e600 2024-03-30T13:12:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f8037fff640 1 -- 172.21.15.43:0/4143209825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8048009450 con 0x7f805015e600 2024-03-30T13:12:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8050156690 con 0x7f805015e600 2024-03-30T13:12:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8050156c50 con 0x7f805015e600 2024-03-30T13:12:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8050068890 con 0x7f805015e600 2024-03-30T13:12:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f8037fff640 1 -- 172.21.15.43:0/4143209825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f80480adc90 con 0x7f805015e600 2024-03-30T13:12:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f8037fff640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f803c067fc0 0x7f803c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f8037fff640 1 -- 172.21.15.43:0/4143209825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f8048121bd0 con 0x7f805015e600 2024-03-30T13:12:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f804f7fe640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f803c067fc0 0x7f803c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.191+0000 7f804f7fe640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f803c067fc0 0x7f803c06a480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8040009840 tx=0x7f8040008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.195+0000 7f8037fff640 1 -- 172.21.15.43:0/4143209825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8048016480 con 0x7f805015e600 2024-03-30T13:12:28.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.359+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f805015adb0 con 0x7f805015e600 2024-03-30T13:12:28.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8037fff640 1 -- 172.21.15.43:0/4143209825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16931 (secure 0 0 0) 0x7f80480ab340 con 0x7f805015e600 2024-03-30T13:12:28.369 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:28.369 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":40,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:12:27.355989+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f803c067fc0 msgr2=0x7f803c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f803c067fc0 0x7f803c06a480 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8040009840 tx=0x7f8040008040 comp rx=0 tx=0).stop 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f805015e600 msgr2=0x7f8050157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f805015e600 0x7f8050157730 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f804801d0a0 tx=0x7f80480195a0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 shutdown_connections 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f803c067fc0 0x7f803c06a480 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f805015e600 0x7f8050157730 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f80500d1560 0x7f8050159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 --2- 172.21.15.43:0/4143209825 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f80500d1100 0x7f80500ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 >> 172.21.15.43:0/4143209825 conn(0x7f80500b6230 msgr2=0x7f80500d2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 shutdown_connections 2024-03-30T13:12:28.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.363+0000 7f8055a6e640 1 -- 172.21.15.43:0/4143209825 wait complete. 2024-03-30T13:12:28.390 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:28.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.499+0000 7f1d24e23640 1 Processor -- start 2024-03-30T13:12:28.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d24e23640 1 -- start start 2024-03-30T13:12:28.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d24e23640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1d20144270 0x7f1d20144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d24e23640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d200b2ef0 0x7f1d2015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d24e23640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d200b33f0 con 0x7f1d200b4490 2024-03-30T13:12:28.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d24e23640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f1d20144fa0 con 0x7f1d200b2ef0 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d24e23640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2015fce0 con 0x7f1d20144270 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d200b2ef0 0x7f1d2015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d200b2ef0 0x7f1d2015f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52852/0 (socket says 172.21.15.43:52852) 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 -- 172.21.15.43:0/2525195365 learned_addr learned my addr 172.21.15.43:0/2525195365 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1e59c640 1 --2- 172.21.15.43:0/2525195365 >> v2:172.21.15.113:3300/0 conn(0x7f1d20144270 0x7f1d20144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 -- 172.21.15.43:0/2525195365 >> v2:172.21.15.113:3300/0 conn(0x7f1d20144270 msgr2=0x7f1d20144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 --2- 172.21.15.43:0/2525195365 >> v2:172.21.15.113:3300/0 conn(0x7f1d20144270 0x7f1d20144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 -- 172.21.15.43:0/2525195365 >> v1:172.21.15.113:6789/0 conn(0x7f1d200b4490 legacy=0x7f1d2006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:28.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 -- 172.21.15.43:0/2525195365 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d200b36d0 con 0x7f1d200b2ef0 2024-03-30T13:12:28.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.503+0000 7f1d1dd9b640 1 --2- 172.21.15.43:0/2525195365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d200b2ef0 0x7f1d2015f7a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1d20072000 tx=0x7f1d1401e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880c16ac031ae8d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1d59a640 1 -- 172.21.15.43:0/2525195365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d140ae070 con 0x7f1d200b2ef0 2024-03-30T13:12:28.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1d59a640 1 -- 172.21.15.43:0/2525195365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d140a8690 con 0x7f1d200b2ef0 2024-03-30T13:12:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1d59a640 1 -- 172.21.15.43:0/2525195365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d140a8a60 con 0x7f1d200b2ef0 2024-03-30T13:12:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- 172.21.15.43:0/2525195365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d200b2ef0 msgr2=0x7f1d2015f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2525195365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d200b2ef0 0x7f1d2015f7a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1d20072000 tx=0x7f1d1401e6c0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- 172.21.15.43:0/2525195365 shutdown_connections 2024-03-30T13:12:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2525195365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d200b2ef0 0x7f1d2015f7a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2525195365 >> v2:172.21.15.113:3300/0 conn(0x7f1d20144270 0x7f1d20144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- 172.21.15.43:0/2525195365 >> 172.21.15.43:0/2525195365 conn(0x7f1d200cfee0 msgr2=0x7f1d200d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- 172.21.15.43:0/2525195365 shutdown_connections 2024-03-30T13:12:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- 172.21.15.43:0/2525195365 wait complete. 2024-03-30T13:12:28.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 Processor -- start 2024-03-30T13:12:28.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- start start 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d200b2ef0 0x7f1d2013ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d200b4490 0x7f1d2015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20144270 0x7f1d2015db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1d200cdd10 con 0x7f1d20144270 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2006cf90 con 0x7f1d200b2ef0 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d24e23640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1d200cde90 con 0x7f1d200b4490 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20144270 0x7f1d2015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20144270 0x7f1d2015db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52854/0 (socket says 172.21.15.43:52854) 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 -- 172.21.15.43:0/2878617846 learned_addr learned my addr 172.21.15.43:0/2878617846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1e59c640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d200b2ef0 0x7f1d2013ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1dd9b640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d200b4490 0x7f1d2015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 -- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d200b4490 msgr2=0x7f1d2015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d200b4490 0x7f1d2015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 -- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d200b2ef0 msgr2=0x7f1d2013ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d200b2ef0 0x7f1d2013ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 -- 172.21.15.43:0/2878617846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2015e070 con 0x7f1d20144270 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1dd9b640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d200b4490 0x7f1d2015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.507+0000 7f1d1ed9d640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20144270 0x7f1d2015db30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800c780 tx=0x7f1d1800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d0effd640 1 -- 172.21.15.43:0/2878617846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18004b70 con 0x7f1d20144270 2024-03-30T13:12:28.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d0effd640 1 -- 172.21.15.43:0/2878617846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18004d10 con 0x7f1d20144270 2024-03-30T13:12:28.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d0effd640 1 -- 172.21.15.43:0/2878617846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800aa90 con 0x7f1d20144270 2024-03-30T13:12:28.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2015ba60 con 0x7f1d20144270 2024-03-30T13:12:28.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2015c020 con 0x7f1d20144270 2024-03-30T13:12:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d20068890 con 0x7f1d20144270 2024-03-30T13:12:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d0effd640 1 -- 172.21.15.43:0/2878617846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1d1800ac30 con 0x7f1d20144270 2024-03-30T13:12:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d0effd640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1d00067fc0 0x7f1d0006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d0effd640 1 -- 172.21.15.43:0/2878617846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f1d18086a00 con 0x7f1d20144270 2024-03-30T13:12:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d1e59c640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1d00067fc0 0x7f1d0006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.511+0000 7f1d1e59c640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1d00067fc0 0x7f1d0006a480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000b9d0 tx=0x7f1d10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.515+0000 7f1d0effd640 1 -- 172.21.15.43:0/2878617846 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f1d18020070 con 0x7f1d20144270 2024-03-30T13:12:28.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.679+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d200722d0 con 0x7f1d20144270 2024-03-30T13:12:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.679+0000 7f1d0effd640 1 -- 172.21.15.43:0/2878617846 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+16931 (secure 0 0 0) 0x7f1d18056f80 con 0x7f1d20144270 2024-03-30T13:12:28.687 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:28.688 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":40,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:12:27.355989+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1d00067fc0 msgr2=0x7f1d0006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1d00067fc0 0x7f1d0006a480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d1000b9d0 tx=0x7f1d10008040 comp rx=0 tx=0).stop 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20144270 msgr2=0x7f1d2015db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20144270 0x7f1d2015db30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800c780 tx=0x7f1d1800cc40 comp rx=0 tx=0).stop 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 shutdown_connections 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1d00067fc0 0x7f1d0006a480 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d20144270 0x7f1d2015db30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1d200b4490 0x7f1d2015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 --2- 172.21.15.43:0/2878617846 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1d200b2ef0 0x7f1d2013ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 >> 172.21.15.43:0/2878617846 conn(0x7f1d200cfee0 msgr2=0x7f1d20067490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:28.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 shutdown_connections 2024-03-30T13:12:28.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.683+0000 7f1d24e23640 1 -- 172.21.15.43:0/2878617846 wait complete. 2024-03-30T13:12:28.706 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-30T13:12:28.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b7ca4640 1 Processor -- start 2024-03-30T13:12:28.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b7ca4640 1 -- start start 2024-03-30T13:12:28.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b7ca4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26b006c670 0x7f26b006cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b7ca4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26b014c680 con 0x7f26b014c2a0 2024-03-30T13:12:28.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b7ca4640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26b0164750 con 0x7f26b00b40b0 2024-03-30T13:12:28.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b7ca4640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f26b00b3310 con 0x7f26b006c670 2024-03-30T13:12:28.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b5a40640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f26b014c2a0 0x7f26b0164040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41020/0 (socket says 172.21.15.43:41020) 2024-03-30T13:12:28.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b5a40640 1 -- 172.21.15.43:0/1401313356 learned_addr learned my addr 172.21.15.43:0/1401313356 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b523f640 1 --2- 172.21.15.43:0/1401313356 >> v2:172.21.15.113:3300/0 conn(0x7f26b006c670 0x7f26b006cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488991927 0 0) 0x7f26a80014a0 con 0x7f26b014c2a0 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2698003530 con 0x7f26b014c2a0 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 312281479 0 0) 0x7f2698003530 con 0x7f26b014c2a0 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 >> v2:172.21.15.113:3300/0 conn(0x7f26b006c670 msgr2=0x7f26b006cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 --2- 172.21.15.43:0/1401313356 >> v2:172.21.15.113:3300/0 conn(0x7f26b006c670 0x7f26b006cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 >> v1:172.21.15.113:6789/0 conn(0x7f26b00b40b0 legacy=0x7f26b0163190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b00b35d0 con 0x7f26b014c2a0 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f26a8001bd0 con 0x7f26b014c2a0 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 >> v1:172.21.15.43:6789/0 conn(0x7f26b014c2a0 legacy=0x7f26b0164040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26980039c0 0x7f2698003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f2698004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26980051d0 0x7f269800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26b00b3310 con 0x7f2698004810 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f269800c070 con 0x7f26980051d0 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f269800c270 con 0x7f26980039c0 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b523f640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f2698004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b6241640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26980051d0 0x7f269800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b523f640 1 -- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26980039c0 msgr2=0x7f2698003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b5a40640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26980039c0 0x7f2698003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b523f640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26980039c0 0x7f2698003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b523f640 1 -- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26980051d0 msgr2=0x7f269800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:28.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b523f640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26980051d0 0x7f269800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b523f640 1 -- 172.21.15.43:0/1401313356 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2698003530 con 0x7f2698004810 2024-03-30T13:12:28.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b6241640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26980051d0 0x7f269800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:28.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.827+0000 7f26b6241640 1 -- 172.21.15.43:0/1401313356 reap_dead start 2024-03-30T13:12:28.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b523f640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f2698004c90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009ae0 tx=0x7f26a401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf011354c346ccf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40b2070 con 0x7f2698004810 2024-03-30T13:12:28.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a40a5660 con 0x7f2698004810 2024-03-30T13:12:28.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b4a3e640 1 -- 172.21.15.43:0/1401313356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a40096b0 con 0x7f2698004810 2024-03-30T13:12:28.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1401313356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 msgr2=0x7f2698004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f2698004c90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009ae0 tx=0x7f26a401ca70 comp rx=0 tx=0).stop 2024-03-30T13:12:28.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1401313356 shutdown_connections 2024-03-30T13:12:28.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1401313356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f2698004c90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1401313356 >> 172.21.15.43:0/1401313356 conn(0x7f26b00d00a0 msgr2=0x7f26b00d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:28.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1401313356 shutdown_connections 2024-03-30T13:12:28.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1401313356 wait complete. 2024-03-30T13:12:28.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 Processor -- start 2024-03-30T13:12:28.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- start start 2024-03-30T13:12:28.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f26b01fb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26b0140560 0x7f26b01409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26b015e740 0x7f26b015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26b006cb20 con 0x7f2698004810 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f26b00b3b90 con 0x7f26b015e740 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b7ca4640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f26b006cca0 con 0x7f26b0140560 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b5a40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f26b01fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b5a40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f26b01fb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52858/0 (socket says 172.21.15.43:52858) 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b5a40640 1 -- 172.21.15.43:0/1193512310 learned_addr learned my addr 172.21.15.43:0/1193512310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b523f640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26b0140560 0x7f26b01409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.831+0000 7f26b6241640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26b015e740 0x7f26b015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b5a40640 1 -- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26b0140560 msgr2=0x7f26b01409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b5a40640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26b0140560 0x7f26b01409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b5a40640 1 -- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26b015e740 msgr2=0x7f26b015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b5a40640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26b015e740 0x7f26b015eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b5a40640 1 -- 172.21.15.43:0/1193512310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b015d6a0 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b6241640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26b015e740 0x7f26b015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b523f640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26b0140560 0x7f26b01409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b5a40640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f26b01fb470 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f26a8010b40 tx=0x7f26a8005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26ae7fc640 1 -- 172.21.15.43:0/1193512310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a800ea10 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26ae7fc640 1 -- 172.21.15.43:0/1193512310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26a8011040 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b015d980 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26ae7fc640 1 -- 172.21.15.43:0/1193512310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a800edf0 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b015df40 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b0068900 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26ae7fc640 1 -- 172.21.15.43:0/1193512310 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f26a801d020 con 0x7f2698004810 2024-03-30T13:12:28.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26ae7fc640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f269c067fc0 0x7f269c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:28.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26ae7fc640 1 -- 172.21.15.43:0/1193512310 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f26a8089620 con 0x7f2698004810 2024-03-30T13:12:28.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b523f640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f269c067fc0 0x7f269c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:28.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.835+0000 7f26b523f640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f269c067fc0 0x7f269c06a480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f26a40af000 tx=0x7f26a4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:28.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:28.839+0000 7f26ae7fc640 1 -- 172.21.15.43:0/1193512310 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f26b014c680 con 0x7f2698004810 2024-03-30T13:12:29.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:29.039+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f26b014bab0 con 0x7f2698004810 2024-03-30T13:12:30.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.023+0000 7f26ae7fc640 1 -- 172.21.15.43:0/1193512310 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f26a8059ba0 con 0x7f2698004810 2024-03-30T13:12:30.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.023+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f269c067fc0 msgr2=0x7f269c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:30.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.023+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f269c067fc0 0x7f269c06a480 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f26a40af000 tx=0x7f26a4008040 comp rx=0 tx=0).stop 2024-03-30T13:12:30.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.023+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 msgr2=0x7f26b01fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:30.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.023+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f26b01fb470 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f26a8010b40 tx=0x7f26a8005b30 comp rx=0 tx=0).stop 2024-03-30T13:12:30.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 shutdown_connections 2024-03-30T13:12:30.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f269c067fc0 0x7f269c06a480 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26b015e740 0x7f26b015eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26b0140560 0x7f26b01409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 --2- 172.21.15.43:0/1193512310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2698004810 0x7f26b01fb470 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 >> 172.21.15.43:0/1193512310 conn(0x7f26b00d00a0 msgr2=0x7f26b014ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:30.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 shutdown_connections 2024-03-30T13:12:30.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.027+0000 7f26b7ca4640 1 -- 172.21.15.43:0/1193512310 wait complete. 2024-03-30T13:12:30.053 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-30T13:12:30.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-30T13:12:30.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e2dc25640 1 Processor -- start 2024-03-30T13:12:30.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e2dc25640 1 -- start start 2024-03-30T13:12:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e2dc25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815ff40 0x7f0e2806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e2dc25640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2806d710 con 0x7f0e2806cbb0 2024-03-30T13:12:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e2dc25640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e2806d9f0 con 0x7f0e2813f140 2024-03-30T13:12:30.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e2dc25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2806d000 con 0x7f0e2815ff40 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e27fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0e2806cbb0 0x7f0e280ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41026/0 (socket says 172.21.15.43:41026) 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e27fff640 1 -- 172.21.15.43:0/4150480866 learned_addr learned my addr 172.21.15.43:0/4150480866 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e1ffff640 1 --2- 172.21.15.43:0/4150480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815ff40 0x7f0e2806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e26ffd640 1 -- 172.21.15.43:0/4150480866 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383072880 0 0) 0x7f0e2806d710 con 0x7f0e2806cbb0 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e26ffd640 1 -- 172.21.15.43:0/4150480866 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e14003530 con 0x7f0e2806cbb0 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e26ffd640 1 -- 172.21.15.43:0/4150480866 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909907431 0 0) 0x7f0e2806d9f0 con 0x7f0e2813f140 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e26ffd640 1 -- 172.21.15.43:0/4150480866 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e2806d710 con 0x7f0e2813f140 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e1ffff640 1 -- 172.21.15.43:0/4150480866 >> v1:172.21.15.113:6789/0 conn(0x7f0e2813f140 legacy=0x7f0e281606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e1ffff640 1 -- 172.21.15.43:0/4150480866 >> v1:172.21.15.43:6789/0 conn(0x7f0e2806cbb0 legacy=0x7f0e280ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:30.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.179+0000 7f0e1ffff640 1 -- 172.21.15.43:0/4150480866 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e2806dcd0 con 0x7f0e2815ff40 2024-03-30T13:12:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e1ffff640 1 --2- 172.21.15.43:0/4150480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815ff40 0x7f0e2806c670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097f0 tx=0x7f0e1001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbf057db507185eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e26ffd640 1 -- 172.21.15.43:0/4150480866 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100b0070 con 0x7f0e2815ff40 2024-03-30T13:12:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e26ffd640 1 -- 172.21.15.43:0/4150480866 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e100ab600 con 0x7f0e2815ff40 2024-03-30T13:12:30.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e26ffd640 1 -- 172.21.15.43:0/4150480866 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e100ab9b0 con 0x7f0e2815ff40 2024-03-30T13:12:30.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 -- 172.21.15.43:0/4150480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815ff40 msgr2=0x7f0e2806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:30.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/4150480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815ff40 0x7f0e2806c670 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0e100097f0 tx=0x7f0e1001c4e0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 -- 172.21.15.43:0/4150480866 shutdown_connections 2024-03-30T13:12:30.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/4150480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2815ff40 0x7f0e2806c670 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 -- 172.21.15.43:0/4150480866 >> 172.21.15.43:0/4150480866 conn(0x7f0e280d00a0 msgr2=0x7f0e280d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:30.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 -- 172.21.15.43:0/4150480866 shutdown_connections 2024-03-30T13:12:30.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 -- 172.21.15.43:0/4150480866 wait complete. 2024-03-30T13:12:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 Processor -- start 2024-03-30T13:12:30.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.183+0000 7f0e2dc25640 1 -- start start 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806cbb0 0x7f0e281388e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2813f140 0x7f0e28154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2815ff40 0x7f0e281551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e280cfc30 con 0x7f0e2815ff40 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e280cfab0 con 0x7f0e2806cbb0 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e2816a000 con 0x7f0e2813f140 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2815ff40 0x7f0e281551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2815ff40 0x7f0e281551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52866/0 (socket says 172.21.15.43:52866) 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 -- 172.21.15.43:0/1517394749 learned_addr learned my addr 172.21.15.43:0/1517394749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e1ffff640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2813f140 0x7f0e28154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e277fe640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806cbb0 0x7f0e281388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:30.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 -- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2813f140 msgr2=0x7f0e28154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2813f140 0x7f0e28154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 -- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806cbb0 msgr2=0x7f0e281388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806cbb0 0x7f0e281388e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 -- 172.21.15.43:0/1517394749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e281557d0 con 0x7f0e2815ff40 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e277fe640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806cbb0 0x7f0e281388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e27fff640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2815ff40 0x7f0e281551c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000ca80 tx=0x7f0e2000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e24ff9640 1 -- 172.21.15.43:0/1517394749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20017070 con 0x7f0e2815ff40 2024-03-30T13:12:30.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e24ff9640 1 -- 172.21.15.43:0/1517394749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e2000ac30 con 0x7f0e2815ff40 2024-03-30T13:12:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e28155ab0 con 0x7f0e2815ff40 2024-03-30T13:12:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e24ff9640 1 -- 172.21.15.43:0/1517394749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e20012400 con 0x7f0e2815ff40 2024-03-30T13:12:30.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e28156070 con 0x7f0e2815ff40 2024-03-30T13:12:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e28068900 con 0x7f0e2815ff40 2024-03-30T13:12:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e24ff9640 1 -- 172.21.15.43:0/1517394749 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0e2001a070 con 0x7f0e2815ff40 2024-03-30T13:12:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e24ff9640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e08067ff0 0x7f0e0806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e24ff9640 1 -- 172.21.15.43:0/1517394749 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f0e20086320 con 0x7f0e2815ff40 2024-03-30T13:12:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.187+0000 7f0e277fe640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e08067ff0 0x7f0e0806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:30.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.191+0000 7f0e277fe640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e08067ff0 0x7f0e0806a4b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001020 tx=0x7f0e18009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:30.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.191+0000 7f0e24ff9640 1 -- 172.21.15.43:0/1517394749 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0e200562f0 con 0x7f0e2815ff40 2024-03-30T13:12:30.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:30.387+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0e2815b950 con 0x7f0e2815ff40 2024-03-30T13:12:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.023+0000 7f0e24ff9640 1 -- 172.21.15.43:0/1517394749 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f0e2005a5f0 con 0x7f0e2815ff40 2024-03-30T13:12:31.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e08067ff0 msgr2=0x7f0e0806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e08067ff0 0x7f0e0806a4b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0e18001020 tx=0x7f0e18009040 comp rx=0 tx=0).stop 2024-03-30T13:12:31.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2815ff40 msgr2=0x7f0e281551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2815ff40 0x7f0e281551c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000ca80 tx=0x7f0e2000cf40 comp rx=0 tx=0).stop 2024-03-30T13:12:31.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 shutdown_connections 2024-03-30T13:12:31.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e08067ff0 0x7f0e0806a4b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e2815ff40 0x7f0e281551c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e2813f140 0x7f0e28154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 --2- 172.21.15.43:0/1517394749 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e2806cbb0 0x7f0e281388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 >> 172.21.15.43:0/1517394749 conn(0x7f0e280d00a0 msgr2=0x7f0e2815ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 shutdown_connections 2024-03-30T13:12:31.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.027+0000 7f0e2dc25640 1 -- 172.21.15.43:0/1517394749 wait complete. 2024-03-30T13:12:31.135 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-30T13:12:31.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f5d351640 1 Processor -- start 2024-03-30T13:12:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f5d351640 1 -- start start 2024-03-30T13:12:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f5d351640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0f5815e600 0x7f0f580b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f5d351640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 0x7f0f580d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f5d351640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f580d3e60 con 0x7f0f580b8160 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f5d351640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0f580b7450 con 0x7f0f5815e600 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f5d351640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0f580d1aa0 con 0x7f0f580d1100 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f577fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0f580b8160 0x7f0f580b85f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34482/0 (socket says 172.21.15.43:34482) 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f577fe640 1 -- 172.21.15.43:0/2346717666 learned_addr learned my addr 172.21.15.43:0/2346717666 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/2346717666 >> v2:172.21.15.113:3300/0 conn(0x7f0f5815e600 0x7f0f580b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f4ffff640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 0x7f0f580d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2285318458 0 0) 0x7f0f580d3e60 con 0x7f0f580b8160 2024-03-30T13:12:31.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f44003530 con 0x7f0f580b8160 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2083004181 0 0) 0x7f0f44003530 con 0x7f0f580b8160 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 msgr2=0x7f0f580d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 0x7f0f580d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 >> v2:172.21.15.113:3300/0 conn(0x7f0f5815e600 msgr2=0x7f0f580b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 --2- 172.21.15.43:0/2346717666 >> v2:172.21.15.113:3300/0 conn(0x7f0f5815e600 0x7f0f580b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f580b7680 con 0x7f0f580b8160 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f0f50001a00 con 0x7f0f580b8160 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 >> v1:172.21.15.43:6789/0 conn(0x7f0f580b8160 legacy=0x7f0f580b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f44003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 0x7f0f44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f44004d50 0x7f0f440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4400b9d0 con 0x7f0f440039c0 2024-03-30T13:12:31.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f4400bb50 con 0x7f0f44004d50 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0f4400bcd0 con 0x7f0f580d1100 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f44003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f577fe640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f44004d50 0x7f0f440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f4ffff640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 0x7f0f44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f56ffd640 1 -- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 msgr2=0x7f0f44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f580d1100 0x7f0f44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f56ffd640 1 -- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f44004d50 msgr2=0x7f0f440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f44004d50 0x7f0f440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.263+0000 7f0f56ffd640 1 -- 172.21.15.43:0/2346717666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f44003530 con 0x7f0f440039c0 2024-03-30T13:12:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f44003de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009ae0 tx=0x7f0f4801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2518725c3bd4dae server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480096b0 con 0x7f0f440039c0 2024-03-30T13:12:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f480a6490 con 0x7f0f440039c0 2024-03-30T13:12:31.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f567fc640 1 -- 172.21.15.43:0/2346717666 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480a6880 con 0x7f0f440039c0 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f5d351640 1 -- 172.21.15.43:0/2346717666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 msgr2=0x7f0f44003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f5d351640 1 --2- 172.21.15.43:0/2346717666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f44003de0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009ae0 tx=0x7f0f4801cab0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f577fe640 1 -- 172.21.15.43:0/2346717666 reap_dead start 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f5d351640 1 -- 172.21.15.43:0/2346717666 shutdown_connections 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f5d351640 1 -- 172.21.15.43:0/2346717666 >> 172.21.15.43:0/2346717666 conn(0x7f0f580b6230 msgr2=0x7f0f580b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f5d351640 1 -- 172.21.15.43:0/2346717666 shutdown_connections 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f5d351640 1 -- 172.21.15.43:0/2346717666 wait complete. 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.267+0000 7f0f5d351640 1 Processor -- start 2024-03-30T13:12:31.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 -- start start 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f5815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f58161f10 0x7f0f58155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f581638e0 0x7f0f58163d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58168fc0 con 0x7f0f440039c0 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58169140 con 0x7f0f581638e0 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0f581692c0 con 0x7f0f58161f10 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f5815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f5815c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34816/0 (socket says 172.21.15.43:34816) 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 -- 172.21.15.43:0/3307622737 learned_addr learned my addr 172.21.15.43:0/3307622737 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f577fe640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f581638e0 0x7f0f58163d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f4ffff640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f58161f10 0x7f0f58155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 -- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f58161f10 msgr2=0x7f0f58155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f58161f10 0x7f0f58155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 -- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f581638e0 msgr2=0x7f0f58163d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f581638e0 0x7f0f58163d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 -- 172.21.15.43:0/3307622737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5806c2d0 con 0x7f0f440039c0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f577fe640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f581638e0 0x7f0f58163d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f56ffd640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f5815c870 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0f4801d020 tx=0x7f0f44005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f4f7fe640 1 -- 172.21.15.43:0/3307622737 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480a6c70 con 0x7f0f440039c0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f4f7fe640 1 -- 172.21.15.43:0/3307622737 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f480a6e10 con 0x7f0f440039c0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f58164280 con 0x7f0f440039c0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f4f7fe640 1 -- 172.21.15.43:0/3307622737 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f480a76b0 con 0x7f0f440039c0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581626f0 con 0x7f0f440039c0 2024-03-30T13:12:31.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f58068890 con 0x7f0f440039c0 2024-03-30T13:12:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f4f7fe640 1 -- 172.21.15.43:0/3307622737 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0f480a7850 con 0x7f0f440039c0 2024-03-30T13:12:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f4f7fe640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f38067ff0 0x7f0f3806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.271+0000 7f0f4f7fe640 1 -- 172.21.15.43:0/3307622737 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f0f48120b80 con 0x7f0f440039c0 2024-03-30T13:12:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.275+0000 7f0f4ffff640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f38067ff0 0x7f0f3806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.275+0000 7f0f4ffff640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f38067ff0 0x7f0f3806a4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f58157c80 tx=0x7f0f40008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.279+0000 7f0f4f7fe640 1 -- 172.21.15.43:0/3307622737 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0f480ac020 con 0x7f0f440039c0 2024-03-30T13:12:31.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.471+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f581579d0 con 0x7f0f440039c0 2024-03-30T13:12:31.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f4f7fe640 1 -- 172.21.15.43:0/3307622737 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7f0f480167c0 con 0x7f0f440039c0 2024-03-30T13:12:31.482 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:31.482 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4271726137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4271726137},{"type":"v1","addr":"172.21.15.43:6834","nonce":4271726137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1748034672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1748034672},{"type":"v1","addr":"172.21.15.43:6835","nonce":1748034672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3254700273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3254700273},{"type":"v1","addr":"172.21.15.113:6835","nonce":3254700273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:12:27.356973+0000","modified":"2024-03-30T13:12:30.394858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4880":{"gid":4880,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1104336007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1104336007},{"type":"v1","addr":"172.21.15.113:6837","nonce":1104336007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4880,"qdb_cluster":[4880]},"id":2}]} 2024-03-30T13:12:31.482 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f38067ff0 msgr2=0x7f0f3806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f38067ff0 0x7f0f3806a4b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0f58157c80 tx=0x7f0f40008040 comp rx=0 tx=0).stop 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 msgr2=0x7f0f5815c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f5815c870 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0f4801d020 tx=0x7f0f44005840 comp rx=0 tx=0).stop 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 shutdown_connections 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0f38067ff0 0x7f0f3806a4b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0f581638e0 0x7f0f58163d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0f58161f10 0x7f0f58155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 --2- 172.21.15.43:0/3307622737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0f440039c0 0x7f0f5815c870 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 >> 172.21.15.43:0/3307622737 conn(0x7f0f580b6230 msgr2=0x7f0f5806bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 shutdown_connections 2024-03-30T13:12:31.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.475+0000 7f0f5d351640 1 -- 172.21.15.43:0/3307622737 wait complete. 2024-03-30T13:12:31.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:12:31.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.603+0000 7fdd6f9f2640 1 Processor -- start 2024-03-30T13:12:31.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.603+0000 7fdd6f9f2640 1 -- start start 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd680b2ef0 0x7fdd68144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdd6815f7a0 0x7fdd6806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd680b4ba0 con 0x7fdd68144270 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fdd68144fa0 con 0x7fdd680b2ef0 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdd680b33f0 con 0x7fdd6815f7a0 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd680b2ef0 0x7fdd68144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd680b2ef0 0x7fdd68144650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34826/0 (socket says 172.21.15.43:34826) 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 -- 172.21.15.43:0/1494103368 learned_addr learned my addr 172.21.15.43:0/1494103368 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6df8f640 1 --2- 172.21.15.43:0/1494103368 >> v2:172.21.15.113:3300/0 conn(0x7fdd6815f7a0 0x7fdd6806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 -- 172.21.15.43:0/1494103368 >> v2:172.21.15.113:3300/0 conn(0x7fdd6815f7a0 msgr2=0x7fdd6806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:31.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 --2- 172.21.15.43:0/1494103368 >> v2:172.21.15.113:3300/0 conn(0x7fdd6815f7a0 0x7fdd6806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 -- 172.21.15.43:0/1494103368 >> v1:172.21.15.113:6789/0 conn(0x7fdd68144270 legacy=0x7fdd680b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 -- 172.21.15.43:0/1494103368 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd680b36d0 con 0x7fdd680b2ef0 2024-03-30T13:12:31.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6cf8d640 1 --2- 172.21.15.43:0/1494103368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd68144650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdd680b5430 tx=0x7fdd5801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410f6c10b2c2f79a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd5ffff640 1 -- 172.21.15.43:0/1494103368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580ab600 con 0x7fdd680b2ef0 2024-03-30T13:12:31.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd5ffff640 1 -- 172.21.15.43:0/1494103368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd580b0070 con 0x7fdd680b2ef0 2024-03-30T13:12:31.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd5ffff640 1 -- 172.21.15.43:0/1494103368 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580ab9c0 con 0x7fdd680b2ef0 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/1494103368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 msgr2=0x7fdd68144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/1494103368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd68144650 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdd680b5430 tx=0x7fdd5801c4f0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/1494103368 shutdown_connections 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/1494103368 >> v2:172.21.15.113:3300/0 conn(0x7fdd6815f7a0 0x7fdd6806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/1494103368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd68144650 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/1494103368 >> 172.21.15.43:0/1494103368 conn(0x7fdd680cfee0 msgr2=0x7fdd680d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.607+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/1494103368 shutdown_connections 2024-03-30T13:12:31.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/1494103368 wait complete. 2024-03-30T13:12:31.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 Processor -- start 2024-03-30T13:12:31.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- start start 2024-03-30T13:12:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd6814b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd68144270 0x7fdd6815d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd6815f7a0 0x7fdd6815d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdd680cdd10 con 0x7fdd680b2ef0 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6815fbb0 con 0x7fdd68144270 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdd680cde90 con 0x7fdd6815f7a0 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd6814b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd6814b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34842/0 (socket says 172.21.15.43:34842) 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 -- 172.21.15.43:0/2655776632 learned_addr learned my addr 172.21.15.43:0/2655776632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6cf8d640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd68144270 0x7fdd6815d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6df8f640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd6815f7a0 0x7fdd6815d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 -- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd6815f7a0 msgr2=0x7fdd6815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd6815f7a0 0x7fdd6815d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 -- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd68144270 msgr2=0x7fdd6815d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd68144270 0x7fdd6815d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 -- 172.21.15.43:0/2655776632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd68141e30 con 0x7fdd680b2ef0 2024-03-30T13:12:31.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6cf8d640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd68144270 0x7fdd6815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6d78e640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd6814b930 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c3f0 tx=0x7fdd6400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd5dffb640 1 -- 172.21.15.43:0/2655776632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64017070 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd5dffb640 1 -- 172.21.15.43:0/2655776632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd640052d0 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd5dffb640 1 -- 172.21.15.43:0/2655776632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64003840 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd68142110 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd681426d0 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.611+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd68068890 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.615+0000 7fdd5dffb640 1 -- 172.21.15.43:0/2655776632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdd6401a070 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.615+0000 7fdd5dffb640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd4c067fc0 0x7fdd4c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.615+0000 7fdd5dffb640 1 -- 172.21.15.43:0/2655776632 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fdd64085d00 con 0x7fdd680b2ef0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.615+0000 7fdd6cf8d640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd4c067fc0 0x7fdd4c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.615+0000 7fdd6cf8d640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd4c067fc0 0x7fdd4c06a480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdd580af000 tx=0x7fdd58008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.619+0000 7fdd5dffb640 1 -- 172.21.15.43:0/2655776632 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdd64021020 con 0x7fdd680b2ef0 2024-03-30T13:12:31.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.779+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd680722d0 con 0x7fdd680b2ef0 2024-03-30T13:12:31.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.779+0000 7fdd5dffb640 1 -- 172.21.15.43:0/2655776632 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16918 (secure 0 0 0) 0x7fdd64005480 con 0x7fdd680b2ef0 2024-03-30T13:12:31.788 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:31.788 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":41,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:12:29.009425+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:12:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd4c067fc0 msgr2=0x7fdd4c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd4c067fc0 0x7fdd4c06a480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdd580af000 tx=0x7fdd58008040 comp rx=0 tx=0).stop 2024-03-30T13:12:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 msgr2=0x7fdd6814b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd6814b930 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c3f0 tx=0x7fdd6400c8b0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 shutdown_connections 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdd4c067fc0 0x7fdd4c06a480 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdd6815f7a0 0x7fdd6815d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdd68144270 0x7fdd6815d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 --2- 172.21.15.43:0/2655776632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680b2ef0 0x7fdd6814b930 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 >> 172.21.15.43:0/2655776632 conn(0x7fdd680cfee0 msgr2=0x7fdd680ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 shutdown_connections 2024-03-30T13:12:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.783+0000 7fdd6f9f2640 1 -- 172.21.15.43:0/2655776632 wait complete. 2024-03-30T13:12:31.807 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-30T13:12:31.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:12:31.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce9c128640 1 Processor -- start 2024-03-30T13:12:31.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce9c128640 1 -- start start 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce9c128640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce9415e400 0x7fce940b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce9c128640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce940b08a0 con 0x7fce940ad8e0 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce9c128640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce9406f3e0 con 0x7fce9406de90 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce9c128640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fce9406f190 con 0x7fce9415e400 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce99ec4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce9415e400 0x7fce940b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce99ec4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce9415e400 0x7fce940b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34858/0 (socket says 172.21.15.43:34858) 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce99ec4640 1 -- 172.21.15.43:0/2298654149 learned_addr learned my addr 172.21.15.43:0/2298654149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce996c3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fce940ad8e0 0x7fce9406d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34488/0 (socket says 172.21.15.43:34488) 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce99ec4640 1 -- 172.21.15.43:0/2298654149 >> v1:172.21.15.113:6789/0 conn(0x7fce9406de90 legacy=0x7fce940b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce99ec4640 1 -- 172.21.15.43:0/2298654149 >> v1:172.21.15.43:6789/0 conn(0x7fce940ad8e0 legacy=0x7fce9406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce99ec4640 1 -- 172.21.15.43:0/2298654149 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9406f6c0 con 0x7fce9415e400 2024-03-30T13:12:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.911+0000 7fce98ec2640 1 -- 172.21.15.43:0/2298654149 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4082416561 0 0) 0x7fce940b08a0 con 0x7fce940ad8e0 2024-03-30T13:12:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce99ec4640 1 --2- 172.21.15.43:0/2298654149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce940b0360 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fce90009870 tx=0x7fce9001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7e1919e34d470c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce98ec2640 1 -- 172.21.15.43:0/2298654149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900ab650 con 0x7fce9415e400 2024-03-30T13:12:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce98ec2640 1 -- 172.21.15.43:0/2298654149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce900b0070 con 0x7fce9415e400 2024-03-30T13:12:31.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce98ec2640 1 -- 172.21.15.43:0/2298654149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce900aba10 con 0x7fce9415e400 2024-03-30T13:12:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- 172.21.15.43:0/2298654149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 msgr2=0x7fce940b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 --2- 172.21.15.43:0/2298654149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce940b0360 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fce90009870 tx=0x7fce9001c560 comp rx=0 tx=0).stop 2024-03-30T13:12:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- 172.21.15.43:0/2298654149 shutdown_connections 2024-03-30T13:12:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 --2- 172.21.15.43:0/2298654149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce940b0360 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- 172.21.15.43:0/2298654149 >> 172.21.15.43:0/2298654149 conn(0x7fce94072500 msgr2=0x7fce94072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- 172.21.15.43:0/2298654149 shutdown_connections 2024-03-30T13:12:31.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- 172.21.15.43:0/2298654149 wait complete. 2024-03-30T13:12:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 Processor -- start 2024-03-30T13:12:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- start start 2024-03-30T13:12:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9406de90 0x7fce94150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce940ad8e0 0x7fce940cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce941523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce940affe0 con 0x7fce9415e400 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce940afe60 con 0x7fce9406de90 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9c128640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fce9416bf30 con 0x7fce940ad8e0 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce941523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce941523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34868/0 (socket says 172.21.15.43:34868) 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 -- 172.21.15.43:0/3443015338 learned_addr learned my addr 172.21.15.43:0/3443015338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce99ec4640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9406de90 0x7fce94150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce996c3640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce940ad8e0 0x7fce940cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 -- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce940ad8e0 msgr2=0x7fce940cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce940ad8e0 0x7fce940cd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 -- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9406de90 msgr2=0x7fce94150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9406de90 0x7fce94150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:31.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 -- 172.21.15.43:0/3443015338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce940cdc30 con 0x7fce9415e400 2024-03-30T13:12:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.915+0000 7fce9a6c5640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce941523f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c730 tx=0x7fce8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce8a7fc640 1 -- 172.21.15.43:0/3443015338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c003b80 con 0x7fce9415e400 2024-03-30T13:12:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce8a7fc640 1 -- 172.21.15.43:0/3443015338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce8c003d20 con 0x7fce9415e400 2024-03-30T13:12:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce940cdf10 con 0x7fce9415e400 2024-03-30T13:12:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce8a7fc640 1 -- 172.21.15.43:0/3443015338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8c005310 con 0x7fce9415e400 2024-03-30T13:12:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce940ce4d0 con 0x7fce9415e400 2024-03-30T13:12:31.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce94068890 con 0x7fce9415e400 2024-03-30T13:12:31.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce8a7fc640 1 -- 172.21.15.43:0/3443015338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fce8c0056c0 con 0x7fce9415e400 2024-03-30T13:12:31.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce8a7fc640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce78067fc0 0x7fce7806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:31.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce8a7fc640 1 -- 172.21.15.43:0/3443015338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fce8c086740 con 0x7fce9415e400 2024-03-30T13:12:31.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce99ec4640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce78067fc0 0x7fce7806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:31.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.919+0000 7fce99ec4640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce78067fc0 0x7fce7806a480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fce900af000 tx=0x7fce900a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:31.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:31.923+0000 7fce8a7fc640 1 -- 172.21.15.43:0/3443015338 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fce8c056d60 con 0x7fce9415e400 2024-03-30T13:12:32.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.075+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce9406ef70 con 0x7fce78067fc0 2024-03-30T13:12:32.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.095+0000 7fce8a7fc640 1 -- 172.21.15.43:0/3443015338 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246895 (secure 0 0 0) 0x7fce9406ef70 con 0x7fce78067fc0 2024-03-30T13:12:32.105 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:32.106 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce78067fc0 msgr2=0x7fce7806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce78067fc0 0x7fce7806a480 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fce900af000 tx=0x7fce900a7000 comp rx=0 tx=0).stop 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 msgr2=0x7fce941523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce941523f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fce8c00c730 tx=0x7fce8c00cbf0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 shutdown_connections 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce78067fc0 0x7fce7806a480 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce9415e400 0x7fce941523f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce940ad8e0 0x7fce940cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 --2- 172.21.15.43:0/3443015338 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce9406de90 0x7fce94150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 >> 172.21.15.43:0/3443015338 conn(0x7fce94072500 msgr2=0x7fce940b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 shutdown_connections 2024-03-30T13:12:32.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.099+0000 7fce9c128640 1 -- 172.21.15.43:0/3443015338 wait complete. 2024-03-30T13:12:32.121 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-03-30T13:12:31.871688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241592,"kb_used_data":4620,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733128,"statfs":{"total":773094113280,"available":772846723072,"internally_reserved":0,"allocated":4730880,"data_stored":2666595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.902330"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984672+0000","last_change":"2024-03-30T13:12:26.984672+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984672+0000","last_undegraded":"2024-03-30T13:12:26.984672+0000","last_fullsized":"2024-03-30T13:12:26.984672+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995194+0000","last_change":"2024-03-30T13:12:26.995194+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995194+0000","last_undegraded":"2024-03-30T13:12:26.995194+0000","last_fullsized":"2024-03-30T13:12:26.995194+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.998240+0000","last_change":"2024-03-30T13:12:26.998240+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.998240+0000","last_undegraded":"2024-03-30T13:12:26.998240+0000","last_fullsized":"2024-03-30T13:12:26.998240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992828+0000","last_change":"2024-03-30T13:12:26.992828+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992828+0000","last_undegraded":"2024-03-30T13:12:26.992828+0000","last_fullsized":"2024-03-30T13:12:26.992828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995055+0000","last_change":"2024-03-30T13:12:26.995055+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995055+0000","last_undegraded":"2024-03-30T13:12:26.995055+0000","last_fullsized":"2024-03-30T13:12:26.995055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984826+0000","last_change":"2024-03-30T13:12:26.984826+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984826+0000","last_undegraded":"2024-03-30T13:12:26.984826+0000","last_fullsized":"2024-03-30T13:12:26.984826+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.006045+0000","last_change":"2024-03-30T13:12:27.006045+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.006045+0000","last_undegraded":"2024-03-30T13:12:27.006045+0000","last_fullsized":"2024-03-30T13:12:27.006045+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988102+0000","last_change":"2024-03-30T13:12:26.988102+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988102+0000","last_undegraded":"2024-03-30T13:12:26.988102+0000","last_fullsized":"2024-03-30T13:12:26.988102+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983703+0000","last_change":"2024-03-30T13:12:26.983703+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983703+0000","last_undegraded":"2024-03-30T13:12:26.983703+0000","last_fullsized":"2024-03-30T13:12:26.983703+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983223+0000","last_change":"2024-03-30T13:12:26.983223+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983223+0000","last_undegraded":"2024-03-30T13:12:26.983223+0000","last_fullsized":"2024-03-30T13:12:26.983223+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991905+0000","last_change":"2024-03-30T13:12:26.991905+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991905+0000","last_undegraded":"2024-03-30T13:12:26.991905+0000","last_fullsized":"2024-03-30T13:12:26.991905+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984222+0000","last_change":"2024-03-30T13:12:26.984222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984222+0000","last_undegraded":"2024-03-30T13:12:26.984222+0000","last_fullsized":"2024-03-30T13:12:26.984222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983094+0000","last_change":"2024-03-30T13:12:26.983094+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983094+0000","last_undegraded":"2024-03-30T13:12:26.983094+0000","last_fullsized":"2024-03-30T13:12:26.983094+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980561+0000","last_change":"2024-03-30T13:12:26.980561+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980561+0000","last_undegraded":"2024-03-30T13:12:26.980561+0000","last_fullsized":"2024-03-30T13:12:26.980561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984611+0000","last_change":"2024-03-30T13:12:26.984611+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984611+0000","last_undegraded":"2024-03-30T13:12:26.984611+0000","last_fullsized":"2024-03-30T13:12:26.984611+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979796+0000","last_change":"2024-03-30T13:12:26.979796+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979796+0000","last_undegraded":"2024-03-30T13:12:26.979796+0000","last_fullsized":"2024-03-30T13:12:26.979796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980818+0000","last_change":"2024-03-30T13:12:26.980818+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980818+0000","last_undegraded":"2024-03-30T13:12:26.980818+0000","last_fullsized":"2024-03-30T13:12:26.980818+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028469+0000","last_change":"2024-03-30T13:12:28.035833+0000","last_active":"2024-03-30T13:12:29.028469+0000","last_peered":"2024-03-30T13:12:29.028469+0000","last_clean":"2024-03-30T13:12:29.028469+0000","last_became_active":"2024-03-30T13:12:28.035506+0000","last_became_peered":"2024-03-30T13:12:28.035506+0000","last_unstale":"2024-03-30T13:12:29.028469+0000","last_undegraded":"2024-03-30T13:12:29.028469+0000","last_fullsized":"2024-03-30T13:12:29.028469+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:18:21.093109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982539+0000","last_change":"2024-03-30T13:12:26.982539+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982539+0000","last_undegraded":"2024-03-30T13:12:26.982539+0000","last_fullsized":"2024-03-30T13:12:26.982539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982370+0000","last_change":"2024-03-30T13:12:26.982370+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982370+0000","last_undegraded":"2024-03-30T13:12:26.982370+0000","last_fullsized":"2024-03-30T13:12:26.982370+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984680+0000","last_change":"2024-03-30T13:12:26.984680+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984680+0000","last_undegraded":"2024-03-30T13:12:26.984680+0000","last_fullsized":"2024-03-30T13:12:26.984680+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980169+0000","last_change":"2024-03-30T13:12:26.980169+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980169+0000","last_undegraded":"2024-03-30T13:12:26.980169+0000","last_fullsized":"2024-03-30T13:12:26.980169+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.003382+0000","last_change":"2024-03-30T13:12:27.003382+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.003382+0000","last_undegraded":"2024-03-30T13:12:27.003382+0000","last_fullsized":"2024-03-30T13:12:27.003382+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985367+0000","last_change":"2024-03-30T13:12:26.985367+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985367+0000","last_undegraded":"2024-03-30T13:12:26.985367+0000","last_fullsized":"2024-03-30T13:12:26.985367+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.001705+0000","last_change":"2024-03-30T13:12:27.001705+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.001705+0000","last_undegraded":"2024-03-30T13:12:27.001705+0000","last_fullsized":"2024-03-30T13:12:27.001705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982634+0000","last_change":"2024-03-30T13:12:26.982634+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982634+0000","last_undegraded":"2024-03-30T13:12:26.982634+0000","last_fullsized":"2024-03-30T13:12:26.982634+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982446+0000","last_change":"2024-03-30T13:12:26.982446+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982446+0000","last_undegraded":"2024-03-30T13:12:26.982446+0000","last_fullsized":"2024-03-30T13:12:26.982446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.977730+0000","last_change":"2024-03-30T13:12:26.977730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.977730+0000","last_undegraded":"2024-03-30T13:12:26.977730+0000","last_fullsized":"2024-03-30T13:12:26.977730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979106+0000","last_change":"2024-03-30T13:12:26.979106+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979106+0000","last_undegraded":"2024-03-30T13:12:26.979106+0000","last_fullsized":"2024-03-30T13:12:26.979106+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982305+0000","last_change":"2024-03-30T13:12:26.982305+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982305+0000","last_undegraded":"2024-03-30T13:12:26.982305+0000","last_fullsized":"2024-03-30T13:12:26.982305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.999479+0000","last_change":"2024-03-30T13:12:26.999479+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.999479+0000","last_undegraded":"2024-03-30T13:12:26.999479+0000","last_fullsized":"2024-03-30T13:12:26.999479+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978170+0000","last_change":"2024-03-30T13:12:26.978170+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978170+0000","last_undegraded":"2024-03-30T13:12:26.978170+0000","last_fullsized":"2024-03-30T13:12:26.978170+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.995591+0000","last_change":"2024-03-30T13:12:26.994626+0000","last_active":"2024-03-30T13:12:26.995591+0000","last_peered":"2024-03-30T13:12:26.995591+0000","last_clean":"2024-03-30T13:12:26.995591+0000","last_became_active":"2024-03-30T13:12:26.994312+0000","last_became_peered":"2024-03-30T13:12:26.994312+0000","last_unstale":"2024-03-30T13:12:26.995591+0000","last_undegraded":"2024-03-30T13:12:26.995591+0000","last_fullsized":"2024-03-30T13:12:26.995591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:46:32.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.987730+0000","last_change":"2024-03-30T13:12:26.987730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.987730+0000","last_undegraded":"2024-03-30T13:12:26.987730+0000","last_fullsized":"2024-03-30T13:12:26.987730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025150+0000","last_change":"2024-03-30T13:12:27.020872+0000","last_active":"2024-03-30T13:12:27.025150+0000","last_peered":"2024-03-30T13:12:27.025150+0000","last_clean":"2024-03-30T13:12:27.025150+0000","last_became_active":"2024-03-30T13:12:27.020255+0000","last_became_peered":"2024-03-30T13:12:27.020255+0000","last_unstale":"2024-03-30T13:12:27.025150+0000","last_undegraded":"2024-03-30T13:12:27.025150+0000","last_fullsized":"2024-03-30T13:12:27.025150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979656+0000","last_change":"2024-03-30T13:12:26.979656+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979656+0000","last_undegraded":"2024-03-30T13:12:26.979656+0000","last_fullsized":"2024-03-30T13:12:26.979656+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026909+0000","last_change":"2024-03-30T13:12:27.024163+0000","last_active":"2024-03-30T13:12:27.026909+0000","last_peered":"2024-03-30T13:12:27.026909+0000","last_clean":"2024-03-30T13:12:27.026909+0000","last_became_active":"2024-03-30T13:12:27.023530+0000","last_became_peered":"2024-03-30T13:12:27.023530+0000","last_unstale":"2024-03-30T13:12:27.026909+0000","last_undegraded":"2024-03-30T13:12:27.026909+0000","last_fullsized":"2024-03-30T13:12:27.026909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:58:35.953478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.996306+0000","last_change":"2024-03-30T13:12:26.996306+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.996306+0000","last_undegraded":"2024-03-30T13:12:26.996306+0000","last_fullsized":"2024-03-30T13:12:26.996306+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.018387+0000","last_change":"2024-03-30T13:12:27.017703+0000","last_active":"2024-03-30T13:12:27.018387+0000","last_peered":"2024-03-30T13:12:27.018387+0000","last_clean":"2024-03-30T13:12:27.018387+0000","last_became_active":"2024-03-30T13:12:27.017417+0000","last_became_peered":"2024-03-30T13:12:27.017417+0000","last_unstale":"2024-03-30T13:12:27.018387+0000","last_undegraded":"2024-03-30T13:12:27.018387+0000","last_fullsized":"2024-03-30T13:12:27.018387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993401+0000","last_change":"2024-03-30T13:12:26.993401+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993401+0000","last_undegraded":"2024-03-30T13:12:26.993401+0000","last_fullsized":"2024-03-30T13:12:26.993401+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.005997+0000","last_change":"2024-03-30T13:12:27.005524+0000","last_active":"2024-03-30T13:12:27.005997+0000","last_peered":"2024-03-30T13:12:27.005997+0000","last_clean":"2024-03-30T13:12:27.005997+0000","last_became_active":"2024-03-30T13:12:27.005130+0000","last_became_peered":"2024-03-30T13:12:27.005130+0000","last_unstale":"2024-03-30T13:12:27.005997+0000","last_undegraded":"2024-03-30T13:12:27.005997+0000","last_fullsized":"2024-03-30T13:12:27.005997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983481+0000","last_change":"2024-03-30T13:12:26.983481+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983481+0000","last_undegraded":"2024-03-30T13:12:26.983481+0000","last_fullsized":"2024-03-30T13:12:26.983481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000683+0000","last_change":"2024-03-30T13:12:26.999973+0000","last_active":"2024-03-30T13:12:27.000683+0000","last_peered":"2024-03-30T13:12:27.000683+0000","last_clean":"2024-03-30T13:12:27.000683+0000","last_became_active":"2024-03-30T13:12:26.999607+0000","last_became_peered":"2024-03-30T13:12:26.999607+0000","last_unstale":"2024-03-30T13:12:27.000683+0000","last_undegraded":"2024-03-30T13:12:27.000683+0000","last_fullsized":"2024-03-30T13:12:27.000683+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:14:13.186516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978581+0000","last_change":"2024-03-30T13:12:26.978581+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978581+0000","last_undegraded":"2024-03-30T13:12:26.978581+0000","last_fullsized":"2024-03-30T13:12:26.978581+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.020714+0000","last_change":"2024-03-30T13:12:27.009633+0000","last_active":"2024-03-30T13:12:27.020714+0000","last_peered":"2024-03-30T13:12:27.020714+0000","last_clean":"2024-03-30T13:12:27.020714+0000","last_became_active":"2024-03-30T13:12:27.008472+0000","last_became_peered":"2024-03-30T13:12:27.008472+0000","last_unstale":"2024-03-30T13:12:27.020714+0000","last_undegraded":"2024-03-30T13:12:27.020714+0000","last_fullsized":"2024-03-30T13:12:27.020714+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:12.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992927+0000","last_change":"2024-03-30T13:12:26.991064+0000","last_active":"2024-03-30T13:12:26.992927+0000","last_peered":"2024-03-30T13:12:26.992927+0000","last_clean":"2024-03-30T13:12:26.992927+0000","last_became_active":"2024-03-30T13:12:26.990695+0000","last_became_peered":"2024-03-30T13:12:26.990695+0000","last_unstale":"2024-03-30T13:12:26.992927+0000","last_undegraded":"2024-03-30T13:12:26.992927+0000","last_fullsized":"2024-03-30T13:12:26.992927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:31:17.730101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993222+0000","last_change":"2024-03-30T13:12:26.993222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993222+0000","last_undegraded":"2024-03-30T13:12:26.993222+0000","last_fullsized":"2024-03-30T13:12:26.993222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021848+0000","last_change":"2024-03-30T13:12:27.014670+0000","last_active":"2024-03-30T13:12:27.021848+0000","last_peered":"2024-03-30T13:12:27.021848+0000","last_clean":"2024-03-30T13:12:27.021848+0000","last_became_active":"2024-03-30T13:12:27.010212+0000","last_became_peered":"2024-03-30T13:12:27.010212+0000","last_unstale":"2024-03-30T13:12:27.021848+0000","last_undegraded":"2024-03-30T13:12:27.021848+0000","last_fullsized":"2024-03-30T13:12:27.021848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:21.640047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985404+0000","last_change":"2024-03-30T13:12:26.985404+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985404+0000","last_undegraded":"2024-03-30T13:12:26.985404+0000","last_fullsized":"2024-03-30T13:12:26.985404+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.014887+0000","last_change":"2024-03-30T13:12:27.014223+0000","last_active":"2024-03-30T13:12:27.014887+0000","last_peered":"2024-03-30T13:12:27.014887+0000","last_clean":"2024-03-30T13:12:27.014887+0000","last_became_active":"2024-03-30T13:12:27.013881+0000","last_became_peered":"2024-03-30T13:12:27.013881+0000","last_unstale":"2024-03-30T13:12:27.014887+0000","last_undegraded":"2024-03-30T13:12:27.014887+0000","last_fullsized":"2024-03-30T13:12:27.014887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:55.961184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993559+0000","last_change":"2024-03-30T13:12:26.993559+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993559+0000","last_undegraded":"2024-03-30T13:12:26.993559+0000","last_fullsized":"2024-03-30T13:12:26.993559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009586+0000","last_change":"2024-03-30T13:12:27.008286+0000","last_active":"2024-03-30T13:12:27.009586+0000","last_peered":"2024-03-30T13:12:27.009586+0000","last_clean":"2024-03-30T13:12:27.009586+0000","last_became_active":"2024-03-30T13:12:27.007966+0000","last_became_peered":"2024-03-30T13:12:27.007966+0000","last_unstale":"2024-03-30T13:12:27.009586+0000","last_undegraded":"2024-03-30T13:12:27.009586+0000","last_fullsized":"2024-03-30T13:12:27.009586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:15:11.139750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983463+0000","last_change":"2024-03-30T13:12:26.983463+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983463+0000","last_undegraded":"2024-03-30T13:12:26.983463+0000","last_fullsized":"2024-03-30T13:12:26.983463+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022912+0000","last_change":"2024-03-30T13:12:27.019139+0000","last_active":"2024-03-30T13:12:27.022912+0000","last_peered":"2024-03-30T13:12:27.022912+0000","last_clean":"2024-03-30T13:12:27.022912+0000","last_became_active":"2024-03-30T13:12:27.017110+0000","last_became_peered":"2024-03-30T13:12:27.017110+0000","last_unstale":"2024-03-30T13:12:27.022912+0000","last_undegraded":"2024-03-30T13:12:27.022912+0000","last_fullsized":"2024-03-30T13:12:27.022912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992123+0000","last_change":"2024-03-30T13:12:26.992123+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992123+0000","last_undegraded":"2024-03-30T13:12:26.992123+0000","last_fullsized":"2024-03-30T13:12:26.992123+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026968+0000","last_change":"2024-03-30T13:12:27.024711+0000","last_active":"2024-03-30T13:12:27.026968+0000","last_peered":"2024-03-30T13:12:27.026968+0000","last_clean":"2024-03-30T13:12:27.026968+0000","last_became_active":"2024-03-30T13:12:27.024207+0000","last_became_peered":"2024-03-30T13:12:27.024207+0000","last_unstale":"2024-03-30T13:12:27.026968+0000","last_undegraded":"2024-03-30T13:12:27.026968+0000","last_fullsized":"2024-03-30T13:12:27.026968+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:24.223034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991563+0000","last_change":"2024-03-30T13:12:26.991563+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991563+0000","last_undegraded":"2024-03-30T13:12:26.991563+0000","last_fullsized":"2024-03-30T13:12:26.991563+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021842+0000","last_change":"2024-03-30T13:12:27.019279+0000","last_active":"2024-03-30T13:12:27.021842+0000","last_peered":"2024-03-30T13:12:27.021842+0000","last_clean":"2024-03-30T13:12:27.021842+0000","last_became_active":"2024-03-30T13:12:27.013568+0000","last_became_peered":"2024-03-30T13:12:27.013568+0000","last_unstale":"2024-03-30T13:12:27.021842+0000","last_undegraded":"2024-03-30T13:12:27.021842+0000","last_fullsized":"2024-03-30T13:12:27.021842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983986+0000","last_change":"2024-03-30T13:12:26.983986+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983986+0000","last_undegraded":"2024-03-30T13:12:26.983986+0000","last_fullsized":"2024-03-30T13:12:26.983986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.990320+0000","last_change":"2024-03-30T13:12:26.989155+0000","last_active":"2024-03-30T13:12:26.990320+0000","last_peered":"2024-03-30T13:12:26.990320+0000","last_clean":"2024-03-30T13:12:26.990320+0000","last_became_active":"2024-03-30T13:12:26.988891+0000","last_became_peered":"2024-03-30T13:12:26.988891+0000","last_unstale":"2024-03-30T13:12:26.990320+0000","last_undegraded":"2024-03-30T13:12:26.990320+0000","last_fullsized":"2024-03-30T13:12:26.990320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:13:02.680729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990026+0000","last_change":"2024-03-30T13:12:26.990026+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990026+0000","last_undegraded":"2024-03-30T13:12:26.990026+0000","last_fullsized":"2024-03-30T13:12:26.990026+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.993878+0000","last_change":"2024-03-30T13:12:26.993198+0000","last_active":"2024-03-30T13:12:26.993878+0000","last_peered":"2024-03-30T13:12:26.993878+0000","last_clean":"2024-03-30T13:12:26.993878+0000","last_became_active":"2024-03-30T13:12:26.992905+0000","last_became_peered":"2024-03-30T13:12:26.992905+0000","last_unstale":"2024-03-30T13:12:26.993878+0000","last_undegraded":"2024-03-30T13:12:26.993878+0000","last_fullsized":"2024-03-30T13:12:26.993878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:41:04.966244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986595+0000","last_change":"2024-03-30T13:12:26.986595+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986595+0000","last_undegraded":"2024-03-30T13:12:26.986595+0000","last_fullsized":"2024-03-30T13:12:26.986595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992428+0000","last_change":"2024-03-30T13:12:26.987766+0000","last_active":"2024-03-30T13:12:26.992428+0000","last_peered":"2024-03-30T13:12:26.992428+0000","last_clean":"2024-03-30T13:12:26.992428+0000","last_became_active":"2024-03-30T13:12:26.987470+0000","last_became_peered":"2024-03-30T13:12:26.987470+0000","last_unstale":"2024-03-30T13:12:26.992428+0000","last_undegraded":"2024-03-30T13:12:26.992428+0000","last_fullsized":"2024-03-30T13:12:26.992428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:59:24.699504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982849+0000","last_change":"2024-03-30T13:12:26.982849+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982849+0000","last_undegraded":"2024-03-30T13:12:26.982849+0000","last_fullsized":"2024-03-30T13:12:26.982849+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009935+0000","last_change":"2024-03-30T13:12:27.008741+0000","last_active":"2024-03-30T13:12:27.009935+0000","last_peered":"2024-03-30T13:12:27.009935+0000","last_clean":"2024-03-30T13:12:27.009935+0000","last_became_active":"2024-03-30T13:12:27.008147+0000","last_became_peered":"2024-03-30T13:12:27.008147+0000","last_unstale":"2024-03-30T13:12:27.009935+0000","last_undegraded":"2024-03-30T13:12:27.009935+0000","last_fullsized":"2024-03-30T13:12:27.009935+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:30:49.564675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983151+0000","last_change":"2024-03-30T13:12:26.983151+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983151+0000","last_undegraded":"2024-03-30T13:12:26.983151+0000","last_fullsized":"2024-03-30T13:12:26.983151+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012018+0000","last_change":"2024-03-30T13:12:27.005991+0000","last_active":"2024-03-30T13:12:27.012018+0000","last_peered":"2024-03-30T13:12:27.012018+0000","last_clean":"2024-03-30T13:12:27.012018+0000","last_became_active":"2024-03-30T13:12:27.003764+0000","last_became_peered":"2024-03-30T13:12:27.003764+0000","last_unstale":"2024-03-30T13:12:27.012018+0000","last_undegraded":"2024-03-30T13:12:27.012018+0000","last_fullsized":"2024-03-30T13:12:27.012018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:50:21.528288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028802+0000","last_change":"2024-03-30T13:12:28.040607+0000","last_active":"2024-03-30T13:12:29.028802+0000","last_peered":"2024-03-30T13:12:29.028802+0000","last_clean":"2024-03-30T13:12:29.028802+0000","last_became_active":"2024-03-30T13:12:28.040303+0000","last_became_peered":"2024-03-30T13:12:28.040303+0000","last_unstale":"2024-03-30T13:12:29.028802+0000","last_undegraded":"2024-03-30T13:12:29.028802+0000","last_fullsized":"2024-03-30T13:12:29.028802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:53:52.056944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028248+0000","last_change":"2024-03-30T13:12:26.991406+0000","last_active":"2024-03-30T13:12:29.028248+0000","last_peered":"2024-03-30T13:12:29.028248+0000","last_clean":"2024-03-30T13:12:29.028248+0000","last_became_active":"2024-03-30T13:12:26.990643+0000","last_became_peered":"2024-03-30T13:12:26.990643+0000","last_unstale":"2024-03-30T13:12:29.028248+0000","last_undegraded":"2024-03-30T13:12:29.028248+0000","last_fullsized":"2024-03-30T13:12:29.028248+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:58:57.574756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978686+0000","last_change":"2024-03-30T13:12:26.978686+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978686+0000","last_undegraded":"2024-03-30T13:12:26.978686+0000","last_fullsized":"2024-03-30T13:12:26.978686+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028428+0000","last_change":"2024-03-30T13:12:27.004641+0000","last_active":"2024-03-30T13:12:29.028428+0000","last_peered":"2024-03-30T13:12:29.028428+0000","last_clean":"2024-03-30T13:12:29.028428+0000","last_became_active":"2024-03-30T13:12:27.002667+0000","last_became_peered":"2024-03-30T13:12:27.002667+0000","last_unstale":"2024-03-30T13:12:29.028428+0000","last_undegraded":"2024-03-30T13:12:29.028428+0000","last_fullsized":"2024-03-30T13:12:29.028428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:07:39.525496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984248+0000","last_change":"2024-03-30T13:12:26.984248+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984248+0000","last_undegraded":"2024-03-30T13:12:26.984248+0000","last_fullsized":"2024-03-30T13:12:26.984248+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025614+0000","last_change":"2024-03-30T13:12:27.021918+0000","last_active":"2024-03-30T13:12:27.025614+0000","last_peered":"2024-03-30T13:12:27.025614+0000","last_clean":"2024-03-30T13:12:27.025614+0000","last_became_active":"2024-03-30T13:12:27.019605+0000","last_became_peered":"2024-03-30T13:12:27.019605+0000","last_unstale":"2024-03-30T13:12:27.025614+0000","last_undegraded":"2024-03-30T13:12:27.025614+0000","last_fullsized":"2024-03-30T13:12:27.025614+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:32:56.857217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984409+0000","last_change":"2024-03-30T13:12:26.984409+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984409+0000","last_undegraded":"2024-03-30T13:12:26.984409+0000","last_fullsized":"2024-03-30T13:12:26.984409+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992790+0000","last_change":"2024-03-30T13:12:26.989095+0000","last_active":"2024-03-30T13:12:26.992790+0000","last_peered":"2024-03-30T13:12:26.992790+0000","last_clean":"2024-03-30T13:12:26.992790+0000","last_became_active":"2024-03-30T13:12:26.988760+0000","last_became_peered":"2024-03-30T13:12:26.988760+0000","last_unstale":"2024-03-30T13:12:26.992790+0000","last_undegraded":"2024-03-30T13:12:26.992790+0000","last_fullsized":"2024-03-30T13:12:26.992790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:25:15.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984702+0000","last_change":"2024-03-30T13:12:26.984702+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984702+0000","last_undegraded":"2024-03-30T13:12:26.984702+0000","last_fullsized":"2024-03-30T13:12:26.984702+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026212+0000","last_change":"2024-03-30T13:12:27.024227+0000","last_active":"2024-03-30T13:12:27.026212+0000","last_peered":"2024-03-30T13:12:27.026212+0000","last_clean":"2024-03-30T13:12:27.026212+0000","last_became_active":"2024-03-30T13:12:27.023036+0000","last_became_peered":"2024-03-30T13:12:27.023036+0000","last_unstale":"2024-03-30T13:12:27.026212+0000","last_undegraded":"2024-03-30T13:12:27.026212+0000","last_fullsized":"2024-03-30T13:12:27.026212+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:15.489606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.989414+0000","last_change":"2024-03-30T13:12:26.989414+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.989414+0000","last_undegraded":"2024-03-30T13:12:26.989414+0000","last_fullsized":"2024-03-30T13:12:26.989414+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000874+0000","last_change":"2024-03-30T13:12:26.999648+0000","last_active":"2024-03-30T13:12:27.000874+0000","last_peered":"2024-03-30T13:12:27.000874+0000","last_clean":"2024-03-30T13:12:27.000874+0000","last_became_active":"2024-03-30T13:12:26.999374+0000","last_became_peered":"2024-03-30T13:12:26.999374+0000","last_unstale":"2024-03-30T13:12:27.000874+0000","last_undegraded":"2024-03-30T13:12:27.000874+0000","last_fullsized":"2024-03-30T13:12:27.000874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:58:38.997896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981991+0000","last_change":"2024-03-30T13:12:26.981991+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981991+0000","last_undegraded":"2024-03-30T13:12:26.981991+0000","last_fullsized":"2024-03-30T13:12:26.981991+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012967+0000","last_change":"2024-03-30T13:12:27.007320+0000","last_active":"2024-03-30T13:12:27.012967+0000","last_peered":"2024-03-30T13:12:27.012967+0000","last_clean":"2024-03-30T13:12:27.012967+0000","last_became_active":"2024-03-30T13:12:27.006579+0000","last_became_peered":"2024-03-30T13:12:27.006579+0000","last_unstale":"2024-03-30T13:12:27.012967+0000","last_undegraded":"2024-03-30T13:12:27.012967+0000","last_fullsized":"2024-03-30T13:12:27.012967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:42.922450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028489+0000","last_change":"2024-03-30T13:12:28.036105+0000","last_active":"2024-03-30T13:12:29.028489+0000","last_peered":"2024-03-30T13:12:29.028489+0000","last_clean":"2024-03-30T13:12:29.028489+0000","last_became_active":"2024-03-30T13:12:28.035011+0000","last_became_peered":"2024-03-30T13:12:28.035011+0000","last_unstale":"2024-03-30T13:12:29.028489+0000","last_undegraded":"2024-03-30T13:12:29.028489+0000","last_fullsized":"2024-03-30T13:12:29.028489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:23:31.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.019402+0000","last_change":"2024-03-30T13:12:27.017279+0000","last_active":"2024-03-30T13:12:27.019402+0000","last_peered":"2024-03-30T13:12:27.019402+0000","last_clean":"2024-03-30T13:12:27.019402+0000","last_became_active":"2024-03-30T13:12:27.016328+0000","last_became_peered":"2024-03-30T13:12:27.016328+0000","last_unstale":"2024-03-30T13:12:27.019402+0000","last_undegraded":"2024-03-30T13:12:27.019402+0000","last_fullsized":"2024-03-30T13:12:27.019402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:20.310096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986444+0000","last_change":"2024-03-30T13:12:26.986444+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986444+0000","last_undegraded":"2024-03-30T13:12:26.986444+0000","last_fullsized":"2024-03-30T13:12:26.986444+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025780+0000","last_change":"2024-03-30T13:12:27.022469+0000","last_active":"2024-03-30T13:12:27.025780+0000","last_peered":"2024-03-30T13:12:27.025780+0000","last_clean":"2024-03-30T13:12:27.025780+0000","last_became_active":"2024-03-30T13:12:27.021653+0000","last_became_peered":"2024-03-30T13:12:27.021653+0000","last_unstale":"2024-03-30T13:12:27.025780+0000","last_undegraded":"2024-03-30T13:12:27.025780+0000","last_fullsized":"2024-03-30T13:12:27.025780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990485+0000","last_change":"2024-03-30T13:12:26.990485+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990485+0000","last_undegraded":"2024-03-30T13:12:26.990485+0000","last_fullsized":"2024-03-30T13:12:26.990485+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.006219+0000","last_change":"2024-03-30T13:12:27.005433+0000","last_active":"2024-03-30T13:12:27.006219+0000","last_peered":"2024-03-30T13:12:27.006219+0000","last_clean":"2024-03-30T13:12:27.006219+0000","last_became_active":"2024-03-30T13:12:27.005116+0000","last_became_peered":"2024-03-30T13:12:27.005116+0000","last_unstale":"2024-03-30T13:12:27.006219+0000","last_undegraded":"2024-03-30T13:12:27.006219+0000","last_fullsized":"2024-03-30T13:12:27.006219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978019+0000","last_change":"2024-03-30T13:12:26.978019+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978019+0000","last_undegraded":"2024-03-30T13:12:26.978019+0000","last_fullsized":"2024-03-30T13:12:26.978019+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.999282+0000","last_change":"2024-03-30T13:12:26.998465+0000","last_active":"2024-03-30T13:12:26.999282+0000","last_peered":"2024-03-30T13:12:26.999282+0000","last_clean":"2024-03-30T13:12:26.999282+0000","last_became_active":"2024-03-30T13:12:26.998168+0000","last_became_peered":"2024-03-30T13:12:26.998168+0000","last_unstale":"2024-03-30T13:12:26.999282+0000","last_undegraded":"2024-03-30T13:12:26.999282+0000","last_fullsized":"2024-03-30T13:12:26.999282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:52:53.995529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981851+0000","last_change":"2024-03-30T13:12:26.981851+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981851+0000","last_undegraded":"2024-03-30T13:12:26.981851+0000","last_fullsized":"2024-03-30T13:12:26.981851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022219+0000","last_change":"2024-03-30T13:12:27.009963+0000","last_active":"2024-03-30T13:12:27.022219+0000","last_peered":"2024-03-30T13:12:27.022219+0000","last_clean":"2024-03-30T13:12:27.022219+0000","last_became_active":"2024-03-30T13:12:27.007546+0000","last_became_peered":"2024-03-30T13:12:27.007546+0000","last_unstale":"2024-03-30T13:12:27.022219+0000","last_undegraded":"2024-03-30T13:12:27.022219+0000","last_fullsized":"2024-03-30T13:12:27.022219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:51.709095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980231+0000","last_change":"2024-03-30T13:12:26.980231+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980231+0000","last_undegraded":"2024-03-30T13:12:26.980231+0000","last_fullsized":"2024-03-30T13:12:26.980231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.002506+0000","last_change":"2024-03-30T13:12:27.002016+0000","last_active":"2024-03-30T13:12:27.002506+0000","last_peered":"2024-03-30T13:12:27.002506+0000","last_clean":"2024-03-30T13:12:27.002506+0000","last_became_active":"2024-03-30T13:12:27.001703+0000","last_became_peered":"2024-03-30T13:12:27.001703+0000","last_unstale":"2024-03-30T13:12:27.002506+0000","last_undegraded":"2024-03-30T13:12:27.002506+0000","last_fullsized":"2024-03-30T13:12:27.002506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":200,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.976182+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:12:26.976182+0000","last_peered":"2024-03-30T13:12:26.976182+0000","last_clean":"2024-03-30T13:12:26.976182+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:12:26.976182+0000","last_undegraded":"2024-03-30T13:12:26.976182+0000","last_fullsized":"2024-03-30T13:12:26.976182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":775962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":790403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:12:32.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:12:32.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b81e8640 1 Processor -- start 2024-03-30T13:12:32.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b81e8640 1 -- start start 2024-03-30T13:12:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b81e8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 0x7f56b00cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b81e8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b00ccef0 con 0x7f56b006ca50 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b81e8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56b00b3710 con 0x7f56b006c670 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b81e8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f56b00b34c0 con 0x7f56b00b41f0 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b6785640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f56b006ca50 0x7f56b00ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34502/0 (socket says 172.21.15.43:34502) 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b6785640 1 -- 172.21.15.43:0/2655643690 learned_addr learned my addr 172.21.15.43:0/2655643690 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5783640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 0x7f56b00cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1376578413 0 0) 0x7f56b00ccef0 con 0x7f56b006ca50 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f569c003530 con 0x7f56b006ca50 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2262584531 0 0) 0x7f569c003530 con 0x7f56b006ca50 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 msgr2=0x7f56b00cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 0x7f56b00cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 >> v1:172.21.15.113:6789/0 conn(0x7f56b006c670 legacy=0x7f56b006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b00b39f0 con 0x7f56b006ca50 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f56a8001a00 con 0x7f56b006ca50 2024-03-30T13:12:32.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 >> v1:172.21.15.43:6789/0 conn(0x7f56b006ca50 legacy=0x7f56b00ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f569c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f569c0043a0 0x7f569c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 0x7f569c0050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56b00b34c0 con 0x7f569c0039f0 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f569c00ba00 con 0x7f569c0043a0 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f569c00bc00 con 0x7f56b00b41f0 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5f84640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f569c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5783640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f569c0043a0 0x7f569c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b6785640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 0x7f569c0050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5f84640 1 -- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 msgr2=0x7f569c0050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5f84640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b00b41f0 0x7f569c0050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5f84640 1 -- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f569c0043a0 msgr2=0x7f569c004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5f84640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f569c0043a0 0x7f569c004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.267+0000 7f56b5f84640 1 -- 172.21.15.43:0/2655643690 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f569c003530 con 0x7f569c0039f0 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b5f84640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f569c003dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001860 tx=0x7f56ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcb24ce3c90166b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0b0070 con 0x7f569c0039f0 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac0ab600 con 0x7f569c0039f0 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b4f82640 1 -- 172.21.15.43:0/2655643690 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac001460 con 0x7f569c0039f0 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- 172.21.15.43:0/2655643690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 msgr2=0x7f569c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 --2- 172.21.15.43:0/2655643690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f569c003dd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f56ac001860 tx=0x7f56ac01c520 comp rx=0 tx=0).stop 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b6785640 1 -- 172.21.15.43:0/2655643690 reap_dead start 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- 172.21.15.43:0/2655643690 shutdown_connections 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- 172.21.15.43:0/2655643690 >> 172.21.15.43:0/2655643690 conn(0x7f56b00cfee0 msgr2=0x7f56b00d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:32.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- 172.21.15.43:0/2655643690 shutdown_connections 2024-03-30T13:12:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- 172.21.15.43:0/2655643690 wait complete. 2024-03-30T13:12:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 Processor -- start 2024-03-30T13:12:32.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- start start 2024-03-30T13:12:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f56b0144010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b0141bf0 0x7f56b014d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b0161ec0 0x7f56b0162350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0168070 con 0x7f569c0039f0 2024-03-30T13:12:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f56b0167ef0 con 0x7f56b0141bf0 2024-03-30T13:12:32.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b81e8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f56b01681f0 con 0x7f56b0161ec0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b5f84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f56b0144010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b5f84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f56b0144010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34890/0 (socket says 172.21.15.43:34890) 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b6785640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b0161ec0 0x7f56b0162350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b5f84640 1 -- 172.21.15.43:0/884590418 learned_addr learned my addr 172.21.15.43:0/884590418 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b5783640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b0141bf0 0x7f56b014d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.271+0000 7f56b5f84640 1 -- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b0161ec0 msgr2=0x7f56b0162350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b5f84640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b0161ec0 0x7f56b0162350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b5f84640 1 -- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b0141bf0 msgr2=0x7f56b014d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b5f84640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b0141bf0 0x7f56b014d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b5f84640 1 -- 172.21.15.43:0/884590418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b0144a50 con 0x7f569c0039f0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b5f84640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f56b0144010 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0ae040 tx=0x7f569c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56a67fc640 1 -- 172.21.15.43:0/884590418 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac0b0040 con 0x7f569c0039f0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56a67fc640 1 -- 172.21.15.43:0/884590418 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56ac0bb070 con 0x7f569c0039f0 2024-03-30T13:12:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56a67fc640 1 -- 172.21.15.43:0/884590418 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac005070 con 0x7f569c0039f0 2024-03-30T13:12:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b0144770 con 0x7f569c0039f0 2024-03-30T13:12:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b013eea0 con 0x7f569c0039f0 2024-03-30T13:12:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b0068890 con 0x7f569c0039f0 2024-03-30T13:12:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56a67fc640 1 -- 172.21.15.43:0/884590418 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f56ac005210 con 0x7f569c0039f0 2024-03-30T13:12:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56a67fc640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5694067ff0 0x7f569406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56a67fc640 1 -- 172.21.15.43:0/884590418 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f56ac120540 con 0x7f569c0039f0 2024-03-30T13:12:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b5783640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5694067ff0 0x7f569406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.275+0000 7f56b5783640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5694067ff0 0x7f569406a4b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009a00 tx=0x7f56a0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:32.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.279+0000 7f56a67fc640 1 -- 172.21.15.43:0/884590418 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f56ac016230 con 0x7f569c0039f0 2024-03-30T13:12:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.435+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56b0162890 con 0x7f5694067ff0 2024-03-30T13:12:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56a67fc640 1 -- 172.21.15.43:0/884590418 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246895 (secure 0 0 0) 0x7f56a00149d0 con 0x7f5694067ff0 2024-03-30T13:12:32.462 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:12:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5694067ff0 msgr2=0x7f569406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5694067ff0 0x7f569406a4b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009a00 tx=0x7f56a0008040 comp rx=0 tx=0).stop 2024-03-30T13:12:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 msgr2=0x7f56b0144010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f56b0144010 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0ae040 tx=0x7f569c005840 comp rx=0 tx=0).stop 2024-03-30T13:12:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 shutdown_connections 2024-03-30T13:12:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5694067ff0 0x7f569406a4b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f56b0161ec0 0x7f56b0162350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f56b0141bf0 0x7f56b014d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 --2- 172.21.15.43:0/884590418 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f569c0039f0 0x7f56b0144010 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 >> 172.21.15.43:0/884590418 conn(0x7f56b00cfee0 msgr2=0x7f56b0067370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 shutdown_connections 2024-03-30T13:12:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.455+0000 7f56b81e8640 1 -- 172.21.15.43:0/884590418 wait complete. 2024-03-30T13:12:32.482 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-03-30T13:12:31.871688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241592,"kb_used_data":4620,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733128,"statfs":{"total":773094113280,"available":772846723072,"internally_reserved":0,"allocated":4730880,"data_stored":2666595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.902330"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984672+0000","last_change":"2024-03-30T13:12:26.984672+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984672+0000","last_undegraded":"2024-03-30T13:12:26.984672+0000","last_fullsized":"2024-03-30T13:12:26.984672+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995194+0000","last_change":"2024-03-30T13:12:26.995194+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995194+0000","last_undegraded":"2024-03-30T13:12:26.995194+0000","last_fullsized":"2024-03-30T13:12:26.995194+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.998240+0000","last_change":"2024-03-30T13:12:26.998240+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.998240+0000","last_undegraded":"2024-03-30T13:12:26.998240+0000","last_fullsized":"2024-03-30T13:12:26.998240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992828+0000","last_change":"2024-03-30T13:12:26.992828+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992828+0000","last_undegraded":"2024-03-30T13:12:26.992828+0000","last_fullsized":"2024-03-30T13:12:26.992828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995055+0000","last_change":"2024-03-30T13:12:26.995055+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995055+0000","last_undegraded":"2024-03-30T13:12:26.995055+0000","last_fullsized":"2024-03-30T13:12:26.995055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984826+0000","last_change":"2024-03-30T13:12:26.984826+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984826+0000","last_undegraded":"2024-03-30T13:12:26.984826+0000","last_fullsized":"2024-03-30T13:12:26.984826+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.006045+0000","last_change":"2024-03-30T13:12:27.006045+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.006045+0000","last_undegraded":"2024-03-30T13:12:27.006045+0000","last_fullsized":"2024-03-30T13:12:27.006045+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988102+0000","last_change":"2024-03-30T13:12:26.988102+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988102+0000","last_undegraded":"2024-03-30T13:12:26.988102+0000","last_fullsized":"2024-03-30T13:12:26.988102+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983703+0000","last_change":"2024-03-30T13:12:26.983703+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983703+0000","last_undegraded":"2024-03-30T13:12:26.983703+0000","last_fullsized":"2024-03-30T13:12:26.983703+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983223+0000","last_change":"2024-03-30T13:12:26.983223+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983223+0000","last_undegraded":"2024-03-30T13:12:26.983223+0000","last_fullsized":"2024-03-30T13:12:26.983223+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991905+0000","last_change":"2024-03-30T13:12:26.991905+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991905+0000","last_undegraded":"2024-03-30T13:12:26.991905+0000","last_fullsized":"2024-03-30T13:12:26.991905+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984222+0000","last_change":"2024-03-30T13:12:26.984222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984222+0000","last_undegraded":"2024-03-30T13:12:26.984222+0000","last_fullsized":"2024-03-30T13:12:26.984222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983094+0000","last_change":"2024-03-30T13:12:26.983094+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983094+0000","last_undegraded":"2024-03-30T13:12:26.983094+0000","last_fullsized":"2024-03-30T13:12:26.983094+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980561+0000","last_change":"2024-03-30T13:12:26.980561+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980561+0000","last_undegraded":"2024-03-30T13:12:26.980561+0000","last_fullsized":"2024-03-30T13:12:26.980561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984611+0000","last_change":"2024-03-30T13:12:26.984611+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984611+0000","last_undegraded":"2024-03-30T13:12:26.984611+0000","last_fullsized":"2024-03-30T13:12:26.984611+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979796+0000","last_change":"2024-03-30T13:12:26.979796+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979796+0000","last_undegraded":"2024-03-30T13:12:26.979796+0000","last_fullsized":"2024-03-30T13:12:26.979796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980818+0000","last_change":"2024-03-30T13:12:26.980818+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980818+0000","last_undegraded":"2024-03-30T13:12:26.980818+0000","last_fullsized":"2024-03-30T13:12:26.980818+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028469+0000","last_change":"2024-03-30T13:12:28.035833+0000","last_active":"2024-03-30T13:12:29.028469+0000","last_peered":"2024-03-30T13:12:29.028469+0000","last_clean":"2024-03-30T13:12:29.028469+0000","last_became_active":"2024-03-30T13:12:28.035506+0000","last_became_peered":"2024-03-30T13:12:28.035506+0000","last_unstale":"2024-03-30T13:12:29.028469+0000","last_undegraded":"2024-03-30T13:12:29.028469+0000","last_fullsized":"2024-03-30T13:12:29.028469+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:18:21.093109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982539+0000","last_change":"2024-03-30T13:12:26.982539+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982539+0000","last_undegraded":"2024-03-30T13:12:26.982539+0000","last_fullsized":"2024-03-30T13:12:26.982539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982370+0000","last_change":"2024-03-30T13:12:26.982370+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982370+0000","last_undegraded":"2024-03-30T13:12:26.982370+0000","last_fullsized":"2024-03-30T13:12:26.982370+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984680+0000","last_change":"2024-03-30T13:12:26.984680+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984680+0000","last_undegraded":"2024-03-30T13:12:26.984680+0000","last_fullsized":"2024-03-30T13:12:26.984680+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980169+0000","last_change":"2024-03-30T13:12:26.980169+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980169+0000","last_undegraded":"2024-03-30T13:12:26.980169+0000","last_fullsized":"2024-03-30T13:12:26.980169+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.003382+0000","last_change":"2024-03-30T13:12:27.003382+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.003382+0000","last_undegraded":"2024-03-30T13:12:27.003382+0000","last_fullsized":"2024-03-30T13:12:27.003382+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985367+0000","last_change":"2024-03-30T13:12:26.985367+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985367+0000","last_undegraded":"2024-03-30T13:12:26.985367+0000","last_fullsized":"2024-03-30T13:12:26.985367+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.001705+0000","last_change":"2024-03-30T13:12:27.001705+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.001705+0000","last_undegraded":"2024-03-30T13:12:27.001705+0000","last_fullsized":"2024-03-30T13:12:27.001705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982634+0000","last_change":"2024-03-30T13:12:26.982634+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982634+0000","last_undegraded":"2024-03-30T13:12:26.982634+0000","last_fullsized":"2024-03-30T13:12:26.982634+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982446+0000","last_change":"2024-03-30T13:12:26.982446+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982446+0000","last_undegraded":"2024-03-30T13:12:26.982446+0000","last_fullsized":"2024-03-30T13:12:26.982446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.977730+0000","last_change":"2024-03-30T13:12:26.977730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.977730+0000","last_undegraded":"2024-03-30T13:12:26.977730+0000","last_fullsized":"2024-03-30T13:12:26.977730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979106+0000","last_change":"2024-03-30T13:12:26.979106+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979106+0000","last_undegraded":"2024-03-30T13:12:26.979106+0000","last_fullsized":"2024-03-30T13:12:26.979106+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982305+0000","last_change":"2024-03-30T13:12:26.982305+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982305+0000","last_undegraded":"2024-03-30T13:12:26.982305+0000","last_fullsized":"2024-03-30T13:12:26.982305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.999479+0000","last_change":"2024-03-30T13:12:26.999479+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.999479+0000","last_undegraded":"2024-03-30T13:12:26.999479+0000","last_fullsized":"2024-03-30T13:12:26.999479+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978170+0000","last_change":"2024-03-30T13:12:26.978170+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978170+0000","last_undegraded":"2024-03-30T13:12:26.978170+0000","last_fullsized":"2024-03-30T13:12:26.978170+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.995591+0000","last_change":"2024-03-30T13:12:26.994626+0000","last_active":"2024-03-30T13:12:26.995591+0000","last_peered":"2024-03-30T13:12:26.995591+0000","last_clean":"2024-03-30T13:12:26.995591+0000","last_became_active":"2024-03-30T13:12:26.994312+0000","last_became_peered":"2024-03-30T13:12:26.994312+0000","last_unstale":"2024-03-30T13:12:26.995591+0000","last_undegraded":"2024-03-30T13:12:26.995591+0000","last_fullsized":"2024-03-30T13:12:26.995591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:46:32.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.987730+0000","last_change":"2024-03-30T13:12:26.987730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.987730+0000","last_undegraded":"2024-03-30T13:12:26.987730+0000","last_fullsized":"2024-03-30T13:12:26.987730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025150+0000","last_change":"2024-03-30T13:12:27.020872+0000","last_active":"2024-03-30T13:12:27.025150+0000","last_peered":"2024-03-30T13:12:27.025150+0000","last_clean":"2024-03-30T13:12:27.025150+0000","last_became_active":"2024-03-30T13:12:27.020255+0000","last_became_peered":"2024-03-30T13:12:27.020255+0000","last_unstale":"2024-03-30T13:12:27.025150+0000","last_undegraded":"2024-03-30T13:12:27.025150+0000","last_fullsized":"2024-03-30T13:12:27.025150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979656+0000","last_change":"2024-03-30T13:12:26.979656+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979656+0000","last_undegraded":"2024-03-30T13:12:26.979656+0000","last_fullsized":"2024-03-30T13:12:26.979656+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026909+0000","last_change":"2024-03-30T13:12:27.024163+0000","last_active":"2024-03-30T13:12:27.026909+0000","last_peered":"2024-03-30T13:12:27.026909+0000","last_clean":"2024-03-30T13:12:27.026909+0000","last_became_active":"2024-03-30T13:12:27.023530+0000","last_became_peered":"2024-03-30T13:12:27.023530+0000","last_unstale":"2024-03-30T13:12:27.026909+0000","last_undegraded":"2024-03-30T13:12:27.026909+0000","last_fullsized":"2024-03-30T13:12:27.026909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:58:35.953478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.996306+0000","last_change":"2024-03-30T13:12:26.996306+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.996306+0000","last_undegraded":"2024-03-30T13:12:26.996306+0000","last_fullsized":"2024-03-30T13:12:26.996306+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.018387+0000","last_change":"2024-03-30T13:12:27.017703+0000","last_active":"2024-03-30T13:12:27.018387+0000","last_peered":"2024-03-30T13:12:27.018387+0000","last_clean":"2024-03-30T13:12:27.018387+0000","last_became_active":"2024-03-30T13:12:27.017417+0000","last_became_peered":"2024-03-30T13:12:27.017417+0000","last_unstale":"2024-03-30T13:12:27.018387+0000","last_undegraded":"2024-03-30T13:12:27.018387+0000","last_fullsized":"2024-03-30T13:12:27.018387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993401+0000","last_change":"2024-03-30T13:12:26.993401+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993401+0000","last_undegraded":"2024-03-30T13:12:26.993401+0000","last_fullsized":"2024-03-30T13:12:26.993401+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.005997+0000","last_change":"2024-03-30T13:12:27.005524+0000","last_active":"2024-03-30T13:12:27.005997+0000","last_peered":"2024-03-30T13:12:27.005997+0000","last_clean":"2024-03-30T13:12:27.005997+0000","last_became_active":"2024-03-30T13:12:27.005130+0000","last_became_peered":"2024-03-30T13:12:27.005130+0000","last_unstale":"2024-03-30T13:12:27.005997+0000","last_undegraded":"2024-03-30T13:12:27.005997+0000","last_fullsized":"2024-03-30T13:12:27.005997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983481+0000","last_change":"2024-03-30T13:12:26.983481+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983481+0000","last_undegraded":"2024-03-30T13:12:26.983481+0000","last_fullsized":"2024-03-30T13:12:26.983481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000683+0000","last_change":"2024-03-30T13:12:26.999973+0000","last_active":"2024-03-30T13:12:27.000683+0000","last_peered":"2024-03-30T13:12:27.000683+0000","last_clean":"2024-03-30T13:12:27.000683+0000","last_became_active":"2024-03-30T13:12:26.999607+0000","last_became_peered":"2024-03-30T13:12:26.999607+0000","last_unstale":"2024-03-30T13:12:27.000683+0000","last_undegraded":"2024-03-30T13:12:27.000683+0000","last_fullsized":"2024-03-30T13:12:27.000683+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:14:13.186516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978581+0000","last_change":"2024-03-30T13:12:26.978581+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978581+0000","last_undegraded":"2024-03-30T13:12:26.978581+0000","last_fullsized":"2024-03-30T13:12:26.978581+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.020714+0000","last_change":"2024-03-30T13:12:27.009633+0000","last_active":"2024-03-30T13:12:27.020714+0000","last_peered":"2024-03-30T13:12:27.020714+0000","last_clean":"2024-03-30T13:12:27.020714+0000","last_became_active":"2024-03-30T13:12:27.008472+0000","last_became_peered":"2024-03-30T13:12:27.008472+0000","last_unstale":"2024-03-30T13:12:27.020714+0000","last_undegraded":"2024-03-30T13:12:27.020714+0000","last_fullsized":"2024-03-30T13:12:27.020714+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:12.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992927+0000","last_change":"2024-03-30T13:12:26.991064+0000","last_active":"2024-03-30T13:12:26.992927+0000","last_peered":"2024-03-30T13:12:26.992927+0000","last_clean":"2024-03-30T13:12:26.992927+0000","last_became_active":"2024-03-30T13:12:26.990695+0000","last_became_peered":"2024-03-30T13:12:26.990695+0000","last_unstale":"2024-03-30T13:12:26.992927+0000","last_undegraded":"2024-03-30T13:12:26.992927+0000","last_fullsized":"2024-03-30T13:12:26.992927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:31:17.730101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993222+0000","last_change":"2024-03-30T13:12:26.993222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993222+0000","last_undegraded":"2024-03-30T13:12:26.993222+0000","last_fullsized":"2024-03-30T13:12:26.993222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021848+0000","last_change":"2024-03-30T13:12:27.014670+0000","last_active":"2024-03-30T13:12:27.021848+0000","last_peered":"2024-03-30T13:12:27.021848+0000","last_clean":"2024-03-30T13:12:27.021848+0000","last_became_active":"2024-03-30T13:12:27.010212+0000","last_became_peered":"2024-03-30T13:12:27.010212+0000","last_unstale":"2024-03-30T13:12:27.021848+0000","last_undegraded":"2024-03-30T13:12:27.021848+0000","last_fullsized":"2024-03-30T13:12:27.021848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:21.640047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985404+0000","last_change":"2024-03-30T13:12:26.985404+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985404+0000","last_undegraded":"2024-03-30T13:12:26.985404+0000","last_fullsized":"2024-03-30T13:12:26.985404+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.014887+0000","last_change":"2024-03-30T13:12:27.014223+0000","last_active":"2024-03-30T13:12:27.014887+0000","last_peered":"2024-03-30T13:12:27.014887+0000","last_clean":"2024-03-30T13:12:27.014887+0000","last_became_active":"2024-03-30T13:12:27.013881+0000","last_became_peered":"2024-03-30T13:12:27.013881+0000","last_unstale":"2024-03-30T13:12:27.014887+0000","last_undegraded":"2024-03-30T13:12:27.014887+0000","last_fullsized":"2024-03-30T13:12:27.014887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:55.961184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993559+0000","last_change":"2024-03-30T13:12:26.993559+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993559+0000","last_undegraded":"2024-03-30T13:12:26.993559+0000","last_fullsized":"2024-03-30T13:12:26.993559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009586+0000","last_change":"2024-03-30T13:12:27.008286+0000","last_active":"2024-03-30T13:12:27.009586+0000","last_peered":"2024-03-30T13:12:27.009586+0000","last_clean":"2024-03-30T13:12:27.009586+0000","last_became_active":"2024-03-30T13:12:27.007966+0000","last_became_peered":"2024-03-30T13:12:27.007966+0000","last_unstale":"2024-03-30T13:12:27.009586+0000","last_undegraded":"2024-03-30T13:12:27.009586+0000","last_fullsized":"2024-03-30T13:12:27.009586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:15:11.139750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983463+0000","last_change":"2024-03-30T13:12:26.983463+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983463+0000","last_undegraded":"2024-03-30T13:12:26.983463+0000","last_fullsized":"2024-03-30T13:12:26.983463+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022912+0000","last_change":"2024-03-30T13:12:27.019139+0000","last_active":"2024-03-30T13:12:27.022912+0000","last_peered":"2024-03-30T13:12:27.022912+0000","last_clean":"2024-03-30T13:12:27.022912+0000","last_became_active":"2024-03-30T13:12:27.017110+0000","last_became_peered":"2024-03-30T13:12:27.017110+0000","last_unstale":"2024-03-30T13:12:27.022912+0000","last_undegraded":"2024-03-30T13:12:27.022912+0000","last_fullsized":"2024-03-30T13:12:27.022912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992123+0000","last_change":"2024-03-30T13:12:26.992123+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992123+0000","last_undegraded":"2024-03-30T13:12:26.992123+0000","last_fullsized":"2024-03-30T13:12:26.992123+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026968+0000","last_change":"2024-03-30T13:12:27.024711+0000","last_active":"2024-03-30T13:12:27.026968+0000","last_peered":"2024-03-30T13:12:27.026968+0000","last_clean":"2024-03-30T13:12:27.026968+0000","last_became_active":"2024-03-30T13:12:27.024207+0000","last_became_peered":"2024-03-30T13:12:27.024207+0000","last_unstale":"2024-03-30T13:12:27.026968+0000","last_undegraded":"2024-03-30T13:12:27.026968+0000","last_fullsized":"2024-03-30T13:12:27.026968+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:24.223034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991563+0000","last_change":"2024-03-30T13:12:26.991563+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991563+0000","last_undegraded":"2024-03-30T13:12:26.991563+0000","last_fullsized":"2024-03-30T13:12:26.991563+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021842+0000","last_change":"2024-03-30T13:12:27.019279+0000","last_active":"2024-03-30T13:12:27.021842+0000","last_peered":"2024-03-30T13:12:27.021842+0000","last_clean":"2024-03-30T13:12:27.021842+0000","last_became_active":"2024-03-30T13:12:27.013568+0000","last_became_peered":"2024-03-30T13:12:27.013568+0000","last_unstale":"2024-03-30T13:12:27.021842+0000","last_undegraded":"2024-03-30T13:12:27.021842+0000","last_fullsized":"2024-03-30T13:12:27.021842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983986+0000","last_change":"2024-03-30T13:12:26.983986+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983986+0000","last_undegraded":"2024-03-30T13:12:26.983986+0000","last_fullsized":"2024-03-30T13:12:26.983986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.990320+0000","last_change":"2024-03-30T13:12:26.989155+0000","last_active":"2024-03-30T13:12:26.990320+0000","last_peered":"2024-03-30T13:12:26.990320+0000","last_clean":"2024-03-30T13:12:26.990320+0000","last_became_active":"2024-03-30T13:12:26.988891+0000","last_became_peered":"2024-03-30T13:12:26.988891+0000","last_unstale":"2024-03-30T13:12:26.990320+0000","last_undegraded":"2024-03-30T13:12:26.990320+0000","last_fullsized":"2024-03-30T13:12:26.990320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:13:02.680729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990026+0000","last_change":"2024-03-30T13:12:26.990026+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990026+0000","last_undegraded":"2024-03-30T13:12:26.990026+0000","last_fullsized":"2024-03-30T13:12:26.990026+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.993878+0000","last_change":"2024-03-30T13:12:26.993198+0000","last_active":"2024-03-30T13:12:26.993878+0000","last_peered":"2024-03-30T13:12:26.993878+0000","last_clean":"2024-03-30T13:12:26.993878+0000","last_became_active":"2024-03-30T13:12:26.992905+0000","last_became_peered":"2024-03-30T13:12:26.992905+0000","last_unstale":"2024-03-30T13:12:26.993878+0000","last_undegraded":"2024-03-30T13:12:26.993878+0000","last_fullsized":"2024-03-30T13:12:26.993878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:41:04.966244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986595+0000","last_change":"2024-03-30T13:12:26.986595+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986595+0000","last_undegraded":"2024-03-30T13:12:26.986595+0000","last_fullsized":"2024-03-30T13:12:26.986595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992428+0000","last_change":"2024-03-30T13:12:26.987766+0000","last_active":"2024-03-30T13:12:26.992428+0000","last_peered":"2024-03-30T13:12:26.992428+0000","last_clean":"2024-03-30T13:12:26.992428+0000","last_became_active":"2024-03-30T13:12:26.987470+0000","last_became_peered":"2024-03-30T13:12:26.987470+0000","last_unstale":"2024-03-30T13:12:26.992428+0000","last_undegraded":"2024-03-30T13:12:26.992428+0000","last_fullsized":"2024-03-30T13:12:26.992428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:59:24.699504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982849+0000","last_change":"2024-03-30T13:12:26.982849+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982849+0000","last_undegraded":"2024-03-30T13:12:26.982849+0000","last_fullsized":"2024-03-30T13:12:26.982849+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009935+0000","last_change":"2024-03-30T13:12:27.008741+0000","last_active":"2024-03-30T13:12:27.009935+0000","last_peered":"2024-03-30T13:12:27.009935+0000","last_clean":"2024-03-30T13:12:27.009935+0000","last_became_active":"2024-03-30T13:12:27.008147+0000","last_became_peered":"2024-03-30T13:12:27.008147+0000","last_unstale":"2024-03-30T13:12:27.009935+0000","last_undegraded":"2024-03-30T13:12:27.009935+0000","last_fullsized":"2024-03-30T13:12:27.009935+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:30:49.564675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983151+0000","last_change":"2024-03-30T13:12:26.983151+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983151+0000","last_undegraded":"2024-03-30T13:12:26.983151+0000","last_fullsized":"2024-03-30T13:12:26.983151+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012018+0000","last_change":"2024-03-30T13:12:27.005991+0000","last_active":"2024-03-30T13:12:27.012018+0000","last_peered":"2024-03-30T13:12:27.012018+0000","last_clean":"2024-03-30T13:12:27.012018+0000","last_became_active":"2024-03-30T13:12:27.003764+0000","last_became_peered":"2024-03-30T13:12:27.003764+0000","last_unstale":"2024-03-30T13:12:27.012018+0000","last_undegraded":"2024-03-30T13:12:27.012018+0000","last_fullsized":"2024-03-30T13:12:27.012018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:50:21.528288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028802+0000","last_change":"2024-03-30T13:12:28.040607+0000","last_active":"2024-03-30T13:12:29.028802+0000","last_peered":"2024-03-30T13:12:29.028802+0000","last_clean":"2024-03-30T13:12:29.028802+0000","last_became_active":"2024-03-30T13:12:28.040303+0000","last_became_peered":"2024-03-30T13:12:28.040303+0000","last_unstale":"2024-03-30T13:12:29.028802+0000","last_undegraded":"2024-03-30T13:12:29.028802+0000","last_fullsized":"2024-03-30T13:12:29.028802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:53:52.056944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028248+0000","last_change":"2024-03-30T13:12:26.991406+0000","last_active":"2024-03-30T13:12:29.028248+0000","last_peered":"2024-03-30T13:12:29.028248+0000","last_clean":"2024-03-30T13:12:29.028248+0000","last_became_active":"2024-03-30T13:12:26.990643+0000","last_became_peered":"2024-03-30T13:12:26.990643+0000","last_unstale":"2024-03-30T13:12:29.028248+0000","last_undegraded":"2024-03-30T13:12:29.028248+0000","last_fullsized":"2024-03-30T13:12:29.028248+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:58:57.574756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978686+0000","last_change":"2024-03-30T13:12:26.978686+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978686+0000","last_undegraded":"2024-03-30T13:12:26.978686+0000","last_fullsized":"2024-03-30T13:12:26.978686+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028428+0000","last_change":"2024-03-30T13:12:27.004641+0000","last_active":"2024-03-30T13:12:29.028428+0000","last_peered":"2024-03-30T13:12:29.028428+0000","last_clean":"2024-03-30T13:12:29.028428+0000","last_became_active":"2024-03-30T13:12:27.002667+0000","last_became_peered":"2024-03-30T13:12:27.002667+0000","last_unstale":"2024-03-30T13:12:29.028428+0000","last_undegraded":"2024-03-30T13:12:29.028428+0000","last_fullsized":"2024-03-30T13:12:29.028428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:07:39.525496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984248+0000","last_change":"2024-03-30T13:12:26.984248+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984248+0000","last_undegraded":"2024-03-30T13:12:26.984248+0000","last_fullsized":"2024-03-30T13:12:26.984248+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025614+0000","last_change":"2024-03-30T13:12:27.021918+0000","last_active":"2024-03-30T13:12:27.025614+0000","last_peered":"2024-03-30T13:12:27.025614+0000","last_clean":"2024-03-30T13:12:27.025614+0000","last_became_active":"2024-03-30T13:12:27.019605+0000","last_became_peered":"2024-03-30T13:12:27.019605+0000","last_unstale":"2024-03-30T13:12:27.025614+0000","last_undegraded":"2024-03-30T13:12:27.025614+0000","last_fullsized":"2024-03-30T13:12:27.025614+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:32:56.857217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984409+0000","last_change":"2024-03-30T13:12:26.984409+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984409+0000","last_undegraded":"2024-03-30T13:12:26.984409+0000","last_fullsized":"2024-03-30T13:12:26.984409+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992790+0000","last_change":"2024-03-30T13:12:26.989095+0000","last_active":"2024-03-30T13:12:26.992790+0000","last_peered":"2024-03-30T13:12:26.992790+0000","last_clean":"2024-03-30T13:12:26.992790+0000","last_became_active":"2024-03-30T13:12:26.988760+0000","last_became_peered":"2024-03-30T13:12:26.988760+0000","last_unstale":"2024-03-30T13:12:26.992790+0000","last_undegraded":"2024-03-30T13:12:26.992790+0000","last_fullsized":"2024-03-30T13:12:26.992790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:25:15.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984702+0000","last_change":"2024-03-30T13:12:26.984702+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984702+0000","last_undegraded":"2024-03-30T13:12:26.984702+0000","last_fullsized":"2024-03-30T13:12:26.984702+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026212+0000","last_change":"2024-03-30T13:12:27.024227+0000","last_active":"2024-03-30T13:12:27.026212+0000","last_peered":"2024-03-30T13:12:27.026212+0000","last_clean":"2024-03-30T13:12:27.026212+0000","last_became_active":"2024-03-30T13:12:27.023036+0000","last_became_peered":"2024-03-30T13:12:27.023036+0000","last_unstale":"2024-03-30T13:12:27.026212+0000","last_undegraded":"2024-03-30T13:12:27.026212+0000","last_fullsized":"2024-03-30T13:12:27.026212+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:15.489606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.989414+0000","last_change":"2024-03-30T13:12:26.989414+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.989414+0000","last_undegraded":"2024-03-30T13:12:26.989414+0000","last_fullsized":"2024-03-30T13:12:26.989414+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000874+0000","last_change":"2024-03-30T13:12:26.999648+0000","last_active":"2024-03-30T13:12:27.000874+0000","last_peered":"2024-03-30T13:12:27.000874+0000","last_clean":"2024-03-30T13:12:27.000874+0000","last_became_active":"2024-03-30T13:12:26.999374+0000","last_became_peered":"2024-03-30T13:12:26.999374+0000","last_unstale":"2024-03-30T13:12:27.000874+0000","last_undegraded":"2024-03-30T13:12:27.000874+0000","last_fullsized":"2024-03-30T13:12:27.000874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:58:38.997896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981991+0000","last_change":"2024-03-30T13:12:26.981991+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981991+0000","last_undegraded":"2024-03-30T13:12:26.981991+0000","last_fullsized":"2024-03-30T13:12:26.981991+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012967+0000","last_change":"2024-03-30T13:12:27.007320+0000","last_active":"2024-03-30T13:12:27.012967+0000","last_peered":"2024-03-30T13:12:27.012967+0000","last_clean":"2024-03-30T13:12:27.012967+0000","last_became_active":"2024-03-30T13:12:27.006579+0000","last_became_peered":"2024-03-30T13:12:27.006579+0000","last_unstale":"2024-03-30T13:12:27.012967+0000","last_undegraded":"2024-03-30T13:12:27.012967+0000","last_fullsized":"2024-03-30T13:12:27.012967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:42.922450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028489+0000","last_change":"2024-03-30T13:12:28.036105+0000","last_active":"2024-03-30T13:12:29.028489+0000","last_peered":"2024-03-30T13:12:29.028489+0000","last_clean":"2024-03-30T13:12:29.028489+0000","last_became_active":"2024-03-30T13:12:28.035011+0000","last_became_peered":"2024-03-30T13:12:28.035011+0000","last_unstale":"2024-03-30T13:12:29.028489+0000","last_undegraded":"2024-03-30T13:12:29.028489+0000","last_fullsized":"2024-03-30T13:12:29.028489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:23:31.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.019402+0000","last_change":"2024-03-30T13:12:27.017279+0000","last_active":"2024-03-30T13:12:27.019402+0000","last_peered":"2024-03-30T13:12:27.019402+0000","last_clean":"2024-03-30T13:12:27.019402+0000","last_became_active":"2024-03-30T13:12:27.016328+0000","last_became_peered":"2024-03-30T13:12:27.016328+0000","last_unstale":"2024-03-30T13:12:27.019402+0000","last_undegraded":"2024-03-30T13:12:27.019402+0000","last_fullsized":"2024-03-30T13:12:27.019402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:20.310096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986444+0000","last_change":"2024-03-30T13:12:26.986444+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986444+0000","last_undegraded":"2024-03-30T13:12:26.986444+0000","last_fullsized":"2024-03-30T13:12:26.986444+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025780+0000","last_change":"2024-03-30T13:12:27.022469+0000","last_active":"2024-03-30T13:12:27.025780+0000","last_peered":"2024-03-30T13:12:27.025780+0000","last_clean":"2024-03-30T13:12:27.025780+0000","last_became_active":"2024-03-30T13:12:27.021653+0000","last_became_peered":"2024-03-30T13:12:27.021653+0000","last_unstale":"2024-03-30T13:12:27.025780+0000","last_undegraded":"2024-03-30T13:12:27.025780+0000","last_fullsized":"2024-03-30T13:12:27.025780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990485+0000","last_change":"2024-03-30T13:12:26.990485+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990485+0000","last_undegraded":"2024-03-30T13:12:26.990485+0000","last_fullsized":"2024-03-30T13:12:26.990485+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.006219+0000","last_change":"2024-03-30T13:12:27.005433+0000","last_active":"2024-03-30T13:12:27.006219+0000","last_peered":"2024-03-30T13:12:27.006219+0000","last_clean":"2024-03-30T13:12:27.006219+0000","last_became_active":"2024-03-30T13:12:27.005116+0000","last_became_peered":"2024-03-30T13:12:27.005116+0000","last_unstale":"2024-03-30T13:12:27.006219+0000","last_undegraded":"2024-03-30T13:12:27.006219+0000","last_fullsized":"2024-03-30T13:12:27.006219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978019+0000","last_change":"2024-03-30T13:12:26.978019+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978019+0000","last_undegraded":"2024-03-30T13:12:26.978019+0000","last_fullsized":"2024-03-30T13:12:26.978019+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.999282+0000","last_change":"2024-03-30T13:12:26.998465+0000","last_active":"2024-03-30T13:12:26.999282+0000","last_peered":"2024-03-30T13:12:26.999282+0000","last_clean":"2024-03-30T13:12:26.999282+0000","last_became_active":"2024-03-30T13:12:26.998168+0000","last_became_peered":"2024-03-30T13:12:26.998168+0000","last_unstale":"2024-03-30T13:12:26.999282+0000","last_undegraded":"2024-03-30T13:12:26.999282+0000","last_fullsized":"2024-03-30T13:12:26.999282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:52:53.995529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981851+0000","last_change":"2024-03-30T13:12:26.981851+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981851+0000","last_undegraded":"2024-03-30T13:12:26.981851+0000","last_fullsized":"2024-03-30T13:12:26.981851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022219+0000","last_change":"2024-03-30T13:12:27.009963+0000","last_active":"2024-03-30T13:12:27.022219+0000","last_peered":"2024-03-30T13:12:27.022219+0000","last_clean":"2024-03-30T13:12:27.022219+0000","last_became_active":"2024-03-30T13:12:27.007546+0000","last_became_peered":"2024-03-30T13:12:27.007546+0000","last_unstale":"2024-03-30T13:12:27.022219+0000","last_undegraded":"2024-03-30T13:12:27.022219+0000","last_fullsized":"2024-03-30T13:12:27.022219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:51.709095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980231+0000","last_change":"2024-03-30T13:12:26.980231+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980231+0000","last_undegraded":"2024-03-30T13:12:26.980231+0000","last_fullsized":"2024-03-30T13:12:26.980231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.002506+0000","last_change":"2024-03-30T13:12:27.002016+0000","last_active":"2024-03-30T13:12:27.002506+0000","last_peered":"2024-03-30T13:12:27.002506+0000","last_clean":"2024-03-30T13:12:27.002506+0000","last_became_active":"2024-03-30T13:12:27.001703+0000","last_became_peered":"2024-03-30T13:12:27.001703+0000","last_unstale":"2024-03-30T13:12:27.002506+0000","last_undegraded":"2024-03-30T13:12:27.002506+0000","last_fullsized":"2024-03-30T13:12:27.002506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":200,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.976182+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:12:26.976182+0000","last_peered":"2024-03-30T13:12:26.976182+0000","last_clean":"2024-03-30T13:12:26.976182+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:12:26.976182+0000","last_undegraded":"2024-03-30T13:12:26.976182+0000","last_fullsized":"2024-03-30T13:12:26.976182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":775962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":790403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:12:32.486 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:12:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f08678b5640 1 Processor -- start 2024-03-30T13:12:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f08678b5640 1 -- start start 2024-03-30T13:12:32.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f08678b5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0860147e10 0x7f0860071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f08678b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0860162b90 0x7f086006e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f08678b5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0860072070 con 0x7f086006ec50 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f08678b5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f086006e280 con 0x7f0860162b90 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f08678b5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f086006dc30 con 0x7f0860147e10 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0864e50640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0860162b90 0x7f086006e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0864e50640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0860162b90 0x7f086006e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34892/0 (socket says 172.21.15.43:34892) 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0864e50640 1 -- 172.21.15.43:0/3393262486 learned_addr learned my addr 172.21.15.43:0/3393262486 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0865651640 1 --2- 172.21.15.43:0/3393262486 >> v2:172.21.15.113:3300/0 conn(0x7f0860147e10 0x7f0860071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0864e50640 1 -- 172.21.15.43:0/3393262486 >> v2:172.21.15.113:3300/0 conn(0x7f0860147e10 msgr2=0x7f0860071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0864e50640 1 --2- 172.21.15.43:0/3393262486 >> v2:172.21.15.113:3300/0 conn(0x7f0860147e10 0x7f0860071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0864e50640 1 -- 172.21.15.43:0/3393262486 >> v1:172.21.15.113:6789/0 conn(0x7f086006ec50 legacy=0x7f086006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.631+0000 7f0864e50640 1 -- 172.21.15.43:0/3393262486 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086006df10 con 0x7f0860162b90 2024-03-30T13:12:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f0864e50640 1 --2- 172.21.15.43:0/3393262486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086006e710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f08600d0f70 tx=0x7f085c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4af9ea498f5d62c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f0857fff640 1 -- 172.21.15.43:0/3393262486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c0b1070 con 0x7f0860162b90 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f0857fff640 1 -- 172.21.15.43:0/3393262486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f085c0ac600 con 0x7f0860162b90 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f0857fff640 1 -- 172.21.15.43:0/3393262486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c0ac9d0 con 0x7f0860162b90 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 -- 172.21.15.43:0/3393262486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 msgr2=0x7f086006e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 --2- 172.21.15.43:0/3393262486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086006e710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f08600d0f70 tx=0x7f085c01c490 comp rx=0 tx=0).stop 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 -- 172.21.15.43:0/3393262486 shutdown_connections 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 --2- 172.21.15.43:0/3393262486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086006e710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 --2- 172.21.15.43:0/3393262486 >> v2:172.21.15.113:3300/0 conn(0x7f0860147e10 0x7f0860071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 -- 172.21.15.43:0/3393262486 >> 172.21.15.43:0/3393262486 conn(0x7f08600cfee0 msgr2=0x7f08600d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 -- 172.21.15.43:0/3393262486 shutdown_connections 2024-03-30T13:12:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 -- 172.21.15.43:0/3393262486 wait complete. 2024-03-30T13:12:32.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 Processor -- start 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.635+0000 7f08678b5640 1 -- start start 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086006ec50 0x7f086015f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0860147e10 0x7f0860140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f086006cf00 con 0x7f0860162b90 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f086006cd80 con 0x7f0860147e10 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f086016b0d0 con 0x7f086006ec50 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086015d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34894/0 (socket says 172.21.15.43:34894) 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 -- 172.21.15.43:0/3730419980 learned_addr learned my addr 172.21.15.43:0/3730419980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865651640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086006ec50 0x7f086015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0864e50640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0860147e10 0x7f0860140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 -- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086006ec50 msgr2=0x7f086015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086006ec50 0x7f086015f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 -- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0860147e10 msgr2=0x7f0860140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0860147e10 0x7f0860140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 -- 172.21.15.43:0/3730419980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086015dcf0 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865651640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086006ec50 0x7f086015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0864e50640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0860147e10 0x7f0860140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865e52640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086015d7b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f085800c710 tx=0x7f085800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0855ffb640 1 -- 172.21.15.43:0/3730419980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858016020 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0855ffb640 1 -- 172.21.15.43:0/3730419980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0858006d60 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086015fa60 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0855ffb640 1 -- 172.21.15.43:0/3730419980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0858003bc0 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0860160020 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0860068890 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0855ffb640 1 -- 172.21.15.43:0/3730419980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f085800a850 con 0x7f0860162b90 2024-03-30T13:12:32.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0855ffb640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0844067fc0 0x7f084406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0855ffb640 1 -- 172.21.15.43:0/3730419980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f0858086310 con 0x7f0860162b90 2024-03-30T13:12:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.639+0000 7f0865651640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0844067fc0 0x7f084406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:32.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.643+0000 7f0865651640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0844067fc0 0x7f084406a480 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08601fa890 tx=0x7f0850008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.643+0000 7f0855ffb640 1 -- 172.21.15.43:0/3730419980 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0858020020 con 0x7f0860162b90 2024-03-30T13:12:32.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.795+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08600d0460 con 0x7f0844067fc0 2024-03-30T13:12:32.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.815+0000 7f0855ffb640 1 -- 172.21.15.43:0/3730419980 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246895 (secure 0 0 0) 0x7f08600d0460 con 0x7f0844067fc0 2024-03-30T13:12:32.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:12:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0844067fc0 msgr2=0x7f084406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0844067fc0 0x7f084406a480 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f08601fa890 tx=0x7f0850008040 comp rx=0 tx=0).stop 2024-03-30T13:12:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 msgr2=0x7f086015d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086015d7b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f085800c710 tx=0x7f085800cbd0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 shutdown_connections 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0844067fc0 0x7f084406a480 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0860162b90 0x7f086015d7b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0860147e10 0x7f0860140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 --2- 172.21.15.43:0/3730419980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f086006ec50 0x7f086015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 >> 172.21.15.43:0/3730419980 conn(0x7f08600cfee0 msgr2=0x7f08600afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 shutdown_connections 2024-03-30T13:12:32.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.819+0000 7f08678b5640 1 -- 172.21.15.43:0/3730419980 wait complete. 2024-03-30T13:12:32.841 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-03-30T13:12:31.871688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241592,"kb_used_data":4620,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733128,"statfs":{"total":773094113280,"available":772846723072,"internally_reserved":0,"allocated":4730880,"data_stored":2666595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.902330"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984672+0000","last_change":"2024-03-30T13:12:26.984672+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984672+0000","last_undegraded":"2024-03-30T13:12:26.984672+0000","last_fullsized":"2024-03-30T13:12:26.984672+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995194+0000","last_change":"2024-03-30T13:12:26.995194+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995194+0000","last_undegraded":"2024-03-30T13:12:26.995194+0000","last_fullsized":"2024-03-30T13:12:26.995194+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.998240+0000","last_change":"2024-03-30T13:12:26.998240+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.998240+0000","last_undegraded":"2024-03-30T13:12:26.998240+0000","last_fullsized":"2024-03-30T13:12:26.998240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992828+0000","last_change":"2024-03-30T13:12:26.992828+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992828+0000","last_undegraded":"2024-03-30T13:12:26.992828+0000","last_fullsized":"2024-03-30T13:12:26.992828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995055+0000","last_change":"2024-03-30T13:12:26.995055+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995055+0000","last_undegraded":"2024-03-30T13:12:26.995055+0000","last_fullsized":"2024-03-30T13:12:26.995055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984826+0000","last_change":"2024-03-30T13:12:26.984826+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984826+0000","last_undegraded":"2024-03-30T13:12:26.984826+0000","last_fullsized":"2024-03-30T13:12:26.984826+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.006045+0000","last_change":"2024-03-30T13:12:27.006045+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.006045+0000","last_undegraded":"2024-03-30T13:12:27.006045+0000","last_fullsized":"2024-03-30T13:12:27.006045+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988102+0000","last_change":"2024-03-30T13:12:26.988102+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988102+0000","last_undegraded":"2024-03-30T13:12:26.988102+0000","last_fullsized":"2024-03-30T13:12:26.988102+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983703+0000","last_change":"2024-03-30T13:12:26.983703+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983703+0000","last_undegraded":"2024-03-30T13:12:26.983703+0000","last_fullsized":"2024-03-30T13:12:26.983703+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983223+0000","last_change":"2024-03-30T13:12:26.983223+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983223+0000","last_undegraded":"2024-03-30T13:12:26.983223+0000","last_fullsized":"2024-03-30T13:12:26.983223+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991905+0000","last_change":"2024-03-30T13:12:26.991905+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991905+0000","last_undegraded":"2024-03-30T13:12:26.991905+0000","last_fullsized":"2024-03-30T13:12:26.991905+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984222+0000","last_change":"2024-03-30T13:12:26.984222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984222+0000","last_undegraded":"2024-03-30T13:12:26.984222+0000","last_fullsized":"2024-03-30T13:12:26.984222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983094+0000","last_change":"2024-03-30T13:12:26.983094+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983094+0000","last_undegraded":"2024-03-30T13:12:26.983094+0000","last_fullsized":"2024-03-30T13:12:26.983094+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980561+0000","last_change":"2024-03-30T13:12:26.980561+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980561+0000","last_undegraded":"2024-03-30T13:12:26.980561+0000","last_fullsized":"2024-03-30T13:12:26.980561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984611+0000","last_change":"2024-03-30T13:12:26.984611+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984611+0000","last_undegraded":"2024-03-30T13:12:26.984611+0000","last_fullsized":"2024-03-30T13:12:26.984611+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979796+0000","last_change":"2024-03-30T13:12:26.979796+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979796+0000","last_undegraded":"2024-03-30T13:12:26.979796+0000","last_fullsized":"2024-03-30T13:12:26.979796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980818+0000","last_change":"2024-03-30T13:12:26.980818+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980818+0000","last_undegraded":"2024-03-30T13:12:26.980818+0000","last_fullsized":"2024-03-30T13:12:26.980818+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028469+0000","last_change":"2024-03-30T13:12:28.035833+0000","last_active":"2024-03-30T13:12:29.028469+0000","last_peered":"2024-03-30T13:12:29.028469+0000","last_clean":"2024-03-30T13:12:29.028469+0000","last_became_active":"2024-03-30T13:12:28.035506+0000","last_became_peered":"2024-03-30T13:12:28.035506+0000","last_unstale":"2024-03-30T13:12:29.028469+0000","last_undegraded":"2024-03-30T13:12:29.028469+0000","last_fullsized":"2024-03-30T13:12:29.028469+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:18:21.093109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982539+0000","last_change":"2024-03-30T13:12:26.982539+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982539+0000","last_undegraded":"2024-03-30T13:12:26.982539+0000","last_fullsized":"2024-03-30T13:12:26.982539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982370+0000","last_change":"2024-03-30T13:12:26.982370+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982370+0000","last_undegraded":"2024-03-30T13:12:26.982370+0000","last_fullsized":"2024-03-30T13:12:26.982370+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984680+0000","last_change":"2024-03-30T13:12:26.984680+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984680+0000","last_undegraded":"2024-03-30T13:12:26.984680+0000","last_fullsized":"2024-03-30T13:12:26.984680+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980169+0000","last_change":"2024-03-30T13:12:26.980169+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980169+0000","last_undegraded":"2024-03-30T13:12:26.980169+0000","last_fullsized":"2024-03-30T13:12:26.980169+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.003382+0000","last_change":"2024-03-30T13:12:27.003382+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.003382+0000","last_undegraded":"2024-03-30T13:12:27.003382+0000","last_fullsized":"2024-03-30T13:12:27.003382+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985367+0000","last_change":"2024-03-30T13:12:26.985367+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985367+0000","last_undegraded":"2024-03-30T13:12:26.985367+0000","last_fullsized":"2024-03-30T13:12:26.985367+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.001705+0000","last_change":"2024-03-30T13:12:27.001705+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.001705+0000","last_undegraded":"2024-03-30T13:12:27.001705+0000","last_fullsized":"2024-03-30T13:12:27.001705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982634+0000","last_change":"2024-03-30T13:12:26.982634+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982634+0000","last_undegraded":"2024-03-30T13:12:26.982634+0000","last_fullsized":"2024-03-30T13:12:26.982634+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982446+0000","last_change":"2024-03-30T13:12:26.982446+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982446+0000","last_undegraded":"2024-03-30T13:12:26.982446+0000","last_fullsized":"2024-03-30T13:12:26.982446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.977730+0000","last_change":"2024-03-30T13:12:26.977730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.977730+0000","last_undegraded":"2024-03-30T13:12:26.977730+0000","last_fullsized":"2024-03-30T13:12:26.977730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979106+0000","last_change":"2024-03-30T13:12:26.979106+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979106+0000","last_undegraded":"2024-03-30T13:12:26.979106+0000","last_fullsized":"2024-03-30T13:12:26.979106+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982305+0000","last_change":"2024-03-30T13:12:26.982305+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982305+0000","last_undegraded":"2024-03-30T13:12:26.982305+0000","last_fullsized":"2024-03-30T13:12:26.982305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.999479+0000","last_change":"2024-03-30T13:12:26.999479+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.999479+0000","last_undegraded":"2024-03-30T13:12:26.999479+0000","last_fullsized":"2024-03-30T13:12:26.999479+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978170+0000","last_change":"2024-03-30T13:12:26.978170+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978170+0000","last_undegraded":"2024-03-30T13:12:26.978170+0000","last_fullsized":"2024-03-30T13:12:26.978170+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.995591+0000","last_change":"2024-03-30T13:12:26.994626+0000","last_active":"2024-03-30T13:12:26.995591+0000","last_peered":"2024-03-30T13:12:26.995591+0000","last_clean":"2024-03-30T13:12:26.995591+0000","last_became_active":"2024-03-30T13:12:26.994312+0000","last_became_peered":"2024-03-30T13:12:26.994312+0000","last_unstale":"2024-03-30T13:12:26.995591+0000","last_undegraded":"2024-03-30T13:12:26.995591+0000","last_fullsized":"2024-03-30T13:12:26.995591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:46:32.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.987730+0000","last_change":"2024-03-30T13:12:26.987730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.987730+0000","last_undegraded":"2024-03-30T13:12:26.987730+0000","last_fullsized":"2024-03-30T13:12:26.987730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025150+0000","last_change":"2024-03-30T13:12:27.020872+0000","last_active":"2024-03-30T13:12:27.025150+0000","last_peered":"2024-03-30T13:12:27.025150+0000","last_clean":"2024-03-30T13:12:27.025150+0000","last_became_active":"2024-03-30T13:12:27.020255+0000","last_became_peered":"2024-03-30T13:12:27.020255+0000","last_unstale":"2024-03-30T13:12:27.025150+0000","last_undegraded":"2024-03-30T13:12:27.025150+0000","last_fullsized":"2024-03-30T13:12:27.025150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979656+0000","last_change":"2024-03-30T13:12:26.979656+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979656+0000","last_undegraded":"2024-03-30T13:12:26.979656+0000","last_fullsized":"2024-03-30T13:12:26.979656+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026909+0000","last_change":"2024-03-30T13:12:27.024163+0000","last_active":"2024-03-30T13:12:27.026909+0000","last_peered":"2024-03-30T13:12:27.026909+0000","last_clean":"2024-03-30T13:12:27.026909+0000","last_became_active":"2024-03-30T13:12:27.023530+0000","last_became_peered":"2024-03-30T13:12:27.023530+0000","last_unstale":"2024-03-30T13:12:27.026909+0000","last_undegraded":"2024-03-30T13:12:27.026909+0000","last_fullsized":"2024-03-30T13:12:27.026909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:58:35.953478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.996306+0000","last_change":"2024-03-30T13:12:26.996306+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.996306+0000","last_undegraded":"2024-03-30T13:12:26.996306+0000","last_fullsized":"2024-03-30T13:12:26.996306+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.018387+0000","last_change":"2024-03-30T13:12:27.017703+0000","last_active":"2024-03-30T13:12:27.018387+0000","last_peered":"2024-03-30T13:12:27.018387+0000","last_clean":"2024-03-30T13:12:27.018387+0000","last_became_active":"2024-03-30T13:12:27.017417+0000","last_became_peered":"2024-03-30T13:12:27.017417+0000","last_unstale":"2024-03-30T13:12:27.018387+0000","last_undegraded":"2024-03-30T13:12:27.018387+0000","last_fullsized":"2024-03-30T13:12:27.018387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993401+0000","last_change":"2024-03-30T13:12:26.993401+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993401+0000","last_undegraded":"2024-03-30T13:12:26.993401+0000","last_fullsized":"2024-03-30T13:12:26.993401+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.005997+0000","last_change":"2024-03-30T13:12:27.005524+0000","last_active":"2024-03-30T13:12:27.005997+0000","last_peered":"2024-03-30T13:12:27.005997+0000","last_clean":"2024-03-30T13:12:27.005997+0000","last_became_active":"2024-03-30T13:12:27.005130+0000","last_became_peered":"2024-03-30T13:12:27.005130+0000","last_unstale":"2024-03-30T13:12:27.005997+0000","last_undegraded":"2024-03-30T13:12:27.005997+0000","last_fullsized":"2024-03-30T13:12:27.005997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983481+0000","last_change":"2024-03-30T13:12:26.983481+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983481+0000","last_undegraded":"2024-03-30T13:12:26.983481+0000","last_fullsized":"2024-03-30T13:12:26.983481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000683+0000","last_change":"2024-03-30T13:12:26.999973+0000","last_active":"2024-03-30T13:12:27.000683+0000","last_peered":"2024-03-30T13:12:27.000683+0000","last_clean":"2024-03-30T13:12:27.000683+0000","last_became_active":"2024-03-30T13:12:26.999607+0000","last_became_peered":"2024-03-30T13:12:26.999607+0000","last_unstale":"2024-03-30T13:12:27.000683+0000","last_undegraded":"2024-03-30T13:12:27.000683+0000","last_fullsized":"2024-03-30T13:12:27.000683+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:14:13.186516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978581+0000","last_change":"2024-03-30T13:12:26.978581+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978581+0000","last_undegraded":"2024-03-30T13:12:26.978581+0000","last_fullsized":"2024-03-30T13:12:26.978581+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.020714+0000","last_change":"2024-03-30T13:12:27.009633+0000","last_active":"2024-03-30T13:12:27.020714+0000","last_peered":"2024-03-30T13:12:27.020714+0000","last_clean":"2024-03-30T13:12:27.020714+0000","last_became_active":"2024-03-30T13:12:27.008472+0000","last_became_peered":"2024-03-30T13:12:27.008472+0000","last_unstale":"2024-03-30T13:12:27.020714+0000","last_undegraded":"2024-03-30T13:12:27.020714+0000","last_fullsized":"2024-03-30T13:12:27.020714+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:12.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992927+0000","last_change":"2024-03-30T13:12:26.991064+0000","last_active":"2024-03-30T13:12:26.992927+0000","last_peered":"2024-03-30T13:12:26.992927+0000","last_clean":"2024-03-30T13:12:26.992927+0000","last_became_active":"2024-03-30T13:12:26.990695+0000","last_became_peered":"2024-03-30T13:12:26.990695+0000","last_unstale":"2024-03-30T13:12:26.992927+0000","last_undegraded":"2024-03-30T13:12:26.992927+0000","last_fullsized":"2024-03-30T13:12:26.992927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:31:17.730101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993222+0000","last_change":"2024-03-30T13:12:26.993222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993222+0000","last_undegraded":"2024-03-30T13:12:26.993222+0000","last_fullsized":"2024-03-30T13:12:26.993222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021848+0000","last_change":"2024-03-30T13:12:27.014670+0000","last_active":"2024-03-30T13:12:27.021848+0000","last_peered":"2024-03-30T13:12:27.021848+0000","last_clean":"2024-03-30T13:12:27.021848+0000","last_became_active":"2024-03-30T13:12:27.010212+0000","last_became_peered":"2024-03-30T13:12:27.010212+0000","last_unstale":"2024-03-30T13:12:27.021848+0000","last_undegraded":"2024-03-30T13:12:27.021848+0000","last_fullsized":"2024-03-30T13:12:27.021848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:21.640047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985404+0000","last_change":"2024-03-30T13:12:26.985404+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985404+0000","last_undegraded":"2024-03-30T13:12:26.985404+0000","last_fullsized":"2024-03-30T13:12:26.985404+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.014887+0000","last_change":"2024-03-30T13:12:27.014223+0000","last_active":"2024-03-30T13:12:27.014887+0000","last_peered":"2024-03-30T13:12:27.014887+0000","last_clean":"2024-03-30T13:12:27.014887+0000","last_became_active":"2024-03-30T13:12:27.013881+0000","last_became_peered":"2024-03-30T13:12:27.013881+0000","last_unstale":"2024-03-30T13:12:27.014887+0000","last_undegraded":"2024-03-30T13:12:27.014887+0000","last_fullsized":"2024-03-30T13:12:27.014887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:55.961184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993559+0000","last_change":"2024-03-30T13:12:26.993559+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993559+0000","last_undegraded":"2024-03-30T13:12:26.993559+0000","last_fullsized":"2024-03-30T13:12:26.993559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009586+0000","last_change":"2024-03-30T13:12:27.008286+0000","last_active":"2024-03-30T13:12:27.009586+0000","last_peered":"2024-03-30T13:12:27.009586+0000","last_clean":"2024-03-30T13:12:27.009586+0000","last_became_active":"2024-03-30T13:12:27.007966+0000","last_became_peered":"2024-03-30T13:12:27.007966+0000","last_unstale":"2024-03-30T13:12:27.009586+0000","last_undegraded":"2024-03-30T13:12:27.009586+0000","last_fullsized":"2024-03-30T13:12:27.009586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:15:11.139750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983463+0000","last_change":"2024-03-30T13:12:26.983463+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983463+0000","last_undegraded":"2024-03-30T13:12:26.983463+0000","last_fullsized":"2024-03-30T13:12:26.983463+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022912+0000","last_change":"2024-03-30T13:12:27.019139+0000","last_active":"2024-03-30T13:12:27.022912+0000","last_peered":"2024-03-30T13:12:27.022912+0000","last_clean":"2024-03-30T13:12:27.022912+0000","last_became_active":"2024-03-30T13:12:27.017110+0000","last_became_peered":"2024-03-30T13:12:27.017110+0000","last_unstale":"2024-03-30T13:12:27.022912+0000","last_undegraded":"2024-03-30T13:12:27.022912+0000","last_fullsized":"2024-03-30T13:12:27.022912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992123+0000","last_change":"2024-03-30T13:12:26.992123+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992123+0000","last_undegraded":"2024-03-30T13:12:26.992123+0000","last_fullsized":"2024-03-30T13:12:26.992123+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026968+0000","last_change":"2024-03-30T13:12:27.024711+0000","last_active":"2024-03-30T13:12:27.026968+0000","last_peered":"2024-03-30T13:12:27.026968+0000","last_clean":"2024-03-30T13:12:27.026968+0000","last_became_active":"2024-03-30T13:12:27.024207+0000","last_became_peered":"2024-03-30T13:12:27.024207+0000","last_unstale":"2024-03-30T13:12:27.026968+0000","last_undegraded":"2024-03-30T13:12:27.026968+0000","last_fullsized":"2024-03-30T13:12:27.026968+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:24.223034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991563+0000","last_change":"2024-03-30T13:12:26.991563+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991563+0000","last_undegraded":"2024-03-30T13:12:26.991563+0000","last_fullsized":"2024-03-30T13:12:26.991563+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021842+0000","last_change":"2024-03-30T13:12:27.019279+0000","last_active":"2024-03-30T13:12:27.021842+0000","last_peered":"2024-03-30T13:12:27.021842+0000","last_clean":"2024-03-30T13:12:27.021842+0000","last_became_active":"2024-03-30T13:12:27.013568+0000","last_became_peered":"2024-03-30T13:12:27.013568+0000","last_unstale":"2024-03-30T13:12:27.021842+0000","last_undegraded":"2024-03-30T13:12:27.021842+0000","last_fullsized":"2024-03-30T13:12:27.021842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983986+0000","last_change":"2024-03-30T13:12:26.983986+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983986+0000","last_undegraded":"2024-03-30T13:12:26.983986+0000","last_fullsized":"2024-03-30T13:12:26.983986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.990320+0000","last_change":"2024-03-30T13:12:26.989155+0000","last_active":"2024-03-30T13:12:26.990320+0000","last_peered":"2024-03-30T13:12:26.990320+0000","last_clean":"2024-03-30T13:12:26.990320+0000","last_became_active":"2024-03-30T13:12:26.988891+0000","last_became_peered":"2024-03-30T13:12:26.988891+0000","last_unstale":"2024-03-30T13:12:26.990320+0000","last_undegraded":"2024-03-30T13:12:26.990320+0000","last_fullsized":"2024-03-30T13:12:26.990320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:13:02.680729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990026+0000","last_change":"2024-03-30T13:12:26.990026+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990026+0000","last_undegraded":"2024-03-30T13:12:26.990026+0000","last_fullsized":"2024-03-30T13:12:26.990026+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.993878+0000","last_change":"2024-03-30T13:12:26.993198+0000","last_active":"2024-03-30T13:12:26.993878+0000","last_peered":"2024-03-30T13:12:26.993878+0000","last_clean":"2024-03-30T13:12:26.993878+0000","last_became_active":"2024-03-30T13:12:26.992905+0000","last_became_peered":"2024-03-30T13:12:26.992905+0000","last_unstale":"2024-03-30T13:12:26.993878+0000","last_undegraded":"2024-03-30T13:12:26.993878+0000","last_fullsized":"2024-03-30T13:12:26.993878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:41:04.966244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986595+0000","last_change":"2024-03-30T13:12:26.986595+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986595+0000","last_undegraded":"2024-03-30T13:12:26.986595+0000","last_fullsized":"2024-03-30T13:12:26.986595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992428+0000","last_change":"2024-03-30T13:12:26.987766+0000","last_active":"2024-03-30T13:12:26.992428+0000","last_peered":"2024-03-30T13:12:26.992428+0000","last_clean":"2024-03-30T13:12:26.992428+0000","last_became_active":"2024-03-30T13:12:26.987470+0000","last_became_peered":"2024-03-30T13:12:26.987470+0000","last_unstale":"2024-03-30T13:12:26.992428+0000","last_undegraded":"2024-03-30T13:12:26.992428+0000","last_fullsized":"2024-03-30T13:12:26.992428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:59:24.699504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982849+0000","last_change":"2024-03-30T13:12:26.982849+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982849+0000","last_undegraded":"2024-03-30T13:12:26.982849+0000","last_fullsized":"2024-03-30T13:12:26.982849+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009935+0000","last_change":"2024-03-30T13:12:27.008741+0000","last_active":"2024-03-30T13:12:27.009935+0000","last_peered":"2024-03-30T13:12:27.009935+0000","last_clean":"2024-03-30T13:12:27.009935+0000","last_became_active":"2024-03-30T13:12:27.008147+0000","last_became_peered":"2024-03-30T13:12:27.008147+0000","last_unstale":"2024-03-30T13:12:27.009935+0000","last_undegraded":"2024-03-30T13:12:27.009935+0000","last_fullsized":"2024-03-30T13:12:27.009935+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:30:49.564675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983151+0000","last_change":"2024-03-30T13:12:26.983151+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983151+0000","last_undegraded":"2024-03-30T13:12:26.983151+0000","last_fullsized":"2024-03-30T13:12:26.983151+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012018+0000","last_change":"2024-03-30T13:12:27.005991+0000","last_active":"2024-03-30T13:12:27.012018+0000","last_peered":"2024-03-30T13:12:27.012018+0000","last_clean":"2024-03-30T13:12:27.012018+0000","last_became_active":"2024-03-30T13:12:27.003764+0000","last_became_peered":"2024-03-30T13:12:27.003764+0000","last_unstale":"2024-03-30T13:12:27.012018+0000","last_undegraded":"2024-03-30T13:12:27.012018+0000","last_fullsized":"2024-03-30T13:12:27.012018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:50:21.528288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028802+0000","last_change":"2024-03-30T13:12:28.040607+0000","last_active":"2024-03-30T13:12:29.028802+0000","last_peered":"2024-03-30T13:12:29.028802+0000","last_clean":"2024-03-30T13:12:29.028802+0000","last_became_active":"2024-03-30T13:12:28.040303+0000","last_became_peered":"2024-03-30T13:12:28.040303+0000","last_unstale":"2024-03-30T13:12:29.028802+0000","last_undegraded":"2024-03-30T13:12:29.028802+0000","last_fullsized":"2024-03-30T13:12:29.028802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:53:52.056944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028248+0000","last_change":"2024-03-30T13:12:26.991406+0000","last_active":"2024-03-30T13:12:29.028248+0000","last_peered":"2024-03-30T13:12:29.028248+0000","last_clean":"2024-03-30T13:12:29.028248+0000","last_became_active":"2024-03-30T13:12:26.990643+0000","last_became_peered":"2024-03-30T13:12:26.990643+0000","last_unstale":"2024-03-30T13:12:29.028248+0000","last_undegraded":"2024-03-30T13:12:29.028248+0000","last_fullsized":"2024-03-30T13:12:29.028248+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:58:57.574756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978686+0000","last_change":"2024-03-30T13:12:26.978686+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978686+0000","last_undegraded":"2024-03-30T13:12:26.978686+0000","last_fullsized":"2024-03-30T13:12:26.978686+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028428+0000","last_change":"2024-03-30T13:12:27.004641+0000","last_active":"2024-03-30T13:12:29.028428+0000","last_peered":"2024-03-30T13:12:29.028428+0000","last_clean":"2024-03-30T13:12:29.028428+0000","last_became_active":"2024-03-30T13:12:27.002667+0000","last_became_peered":"2024-03-30T13:12:27.002667+0000","last_unstale":"2024-03-30T13:12:29.028428+0000","last_undegraded":"2024-03-30T13:12:29.028428+0000","last_fullsized":"2024-03-30T13:12:29.028428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:07:39.525496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984248+0000","last_change":"2024-03-30T13:12:26.984248+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984248+0000","last_undegraded":"2024-03-30T13:12:26.984248+0000","last_fullsized":"2024-03-30T13:12:26.984248+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025614+0000","last_change":"2024-03-30T13:12:27.021918+0000","last_active":"2024-03-30T13:12:27.025614+0000","last_peered":"2024-03-30T13:12:27.025614+0000","last_clean":"2024-03-30T13:12:27.025614+0000","last_became_active":"2024-03-30T13:12:27.019605+0000","last_became_peered":"2024-03-30T13:12:27.019605+0000","last_unstale":"2024-03-30T13:12:27.025614+0000","last_undegraded":"2024-03-30T13:12:27.025614+0000","last_fullsized":"2024-03-30T13:12:27.025614+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:32:56.857217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984409+0000","last_change":"2024-03-30T13:12:26.984409+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984409+0000","last_undegraded":"2024-03-30T13:12:26.984409+0000","last_fullsized":"2024-03-30T13:12:26.984409+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992790+0000","last_change":"2024-03-30T13:12:26.989095+0000","last_active":"2024-03-30T13:12:26.992790+0000","last_peered":"2024-03-30T13:12:26.992790+0000","last_clean":"2024-03-30T13:12:26.992790+0000","last_became_active":"2024-03-30T13:12:26.988760+0000","last_became_peered":"2024-03-30T13:12:26.988760+0000","last_unstale":"2024-03-30T13:12:26.992790+0000","last_undegraded":"2024-03-30T13:12:26.992790+0000","last_fullsized":"2024-03-30T13:12:26.992790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:25:15.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984702+0000","last_change":"2024-03-30T13:12:26.984702+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984702+0000","last_undegraded":"2024-03-30T13:12:26.984702+0000","last_fullsized":"2024-03-30T13:12:26.984702+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026212+0000","last_change":"2024-03-30T13:12:27.024227+0000","last_active":"2024-03-30T13:12:27.026212+0000","last_peered":"2024-03-30T13:12:27.026212+0000","last_clean":"2024-03-30T13:12:27.026212+0000","last_became_active":"2024-03-30T13:12:27.023036+0000","last_became_peered":"2024-03-30T13:12:27.023036+0000","last_unstale":"2024-03-30T13:12:27.026212+0000","last_undegraded":"2024-03-30T13:12:27.026212+0000","last_fullsized":"2024-03-30T13:12:27.026212+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:15.489606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.989414+0000","last_change":"2024-03-30T13:12:26.989414+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.989414+0000","last_undegraded":"2024-03-30T13:12:26.989414+0000","last_fullsized":"2024-03-30T13:12:26.989414+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000874+0000","last_change":"2024-03-30T13:12:26.999648+0000","last_active":"2024-03-30T13:12:27.000874+0000","last_peered":"2024-03-30T13:12:27.000874+0000","last_clean":"2024-03-30T13:12:27.000874+0000","last_became_active":"2024-03-30T13:12:26.999374+0000","last_became_peered":"2024-03-30T13:12:26.999374+0000","last_unstale":"2024-03-30T13:12:27.000874+0000","last_undegraded":"2024-03-30T13:12:27.000874+0000","last_fullsized":"2024-03-30T13:12:27.000874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:58:38.997896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981991+0000","last_change":"2024-03-30T13:12:26.981991+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981991+0000","last_undegraded":"2024-03-30T13:12:26.981991+0000","last_fullsized":"2024-03-30T13:12:26.981991+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012967+0000","last_change":"2024-03-30T13:12:27.007320+0000","last_active":"2024-03-30T13:12:27.012967+0000","last_peered":"2024-03-30T13:12:27.012967+0000","last_clean":"2024-03-30T13:12:27.012967+0000","last_became_active":"2024-03-30T13:12:27.006579+0000","last_became_peered":"2024-03-30T13:12:27.006579+0000","last_unstale":"2024-03-30T13:12:27.012967+0000","last_undegraded":"2024-03-30T13:12:27.012967+0000","last_fullsized":"2024-03-30T13:12:27.012967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:42.922450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028489+0000","last_change":"2024-03-30T13:12:28.036105+0000","last_active":"2024-03-30T13:12:29.028489+0000","last_peered":"2024-03-30T13:12:29.028489+0000","last_clean":"2024-03-30T13:12:29.028489+0000","last_became_active":"2024-03-30T13:12:28.035011+0000","last_became_peered":"2024-03-30T13:12:28.035011+0000","last_unstale":"2024-03-30T13:12:29.028489+0000","last_undegraded":"2024-03-30T13:12:29.028489+0000","last_fullsized":"2024-03-30T13:12:29.028489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:23:31.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.019402+0000","last_change":"2024-03-30T13:12:27.017279+0000","last_active":"2024-03-30T13:12:27.019402+0000","last_peered":"2024-03-30T13:12:27.019402+0000","last_clean":"2024-03-30T13:12:27.019402+0000","last_became_active":"2024-03-30T13:12:27.016328+0000","last_became_peered":"2024-03-30T13:12:27.016328+0000","last_unstale":"2024-03-30T13:12:27.019402+0000","last_undegraded":"2024-03-30T13:12:27.019402+0000","last_fullsized":"2024-03-30T13:12:27.019402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:20.310096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986444+0000","last_change":"2024-03-30T13:12:26.986444+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986444+0000","last_undegraded":"2024-03-30T13:12:26.986444+0000","last_fullsized":"2024-03-30T13:12:26.986444+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025780+0000","last_change":"2024-03-30T13:12:27.022469+0000","last_active":"2024-03-30T13:12:27.025780+0000","last_peered":"2024-03-30T13:12:27.025780+0000","last_clean":"2024-03-30T13:12:27.025780+0000","last_became_active":"2024-03-30T13:12:27.021653+0000","last_became_peered":"2024-03-30T13:12:27.021653+0000","last_unstale":"2024-03-30T13:12:27.025780+0000","last_undegraded":"2024-03-30T13:12:27.025780+0000","last_fullsized":"2024-03-30T13:12:27.025780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990485+0000","last_change":"2024-03-30T13:12:26.990485+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990485+0000","last_undegraded":"2024-03-30T13:12:26.990485+0000","last_fullsized":"2024-03-30T13:12:26.990485+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.006219+0000","last_change":"2024-03-30T13:12:27.005433+0000","last_active":"2024-03-30T13:12:27.006219+0000","last_peered":"2024-03-30T13:12:27.006219+0000","last_clean":"2024-03-30T13:12:27.006219+0000","last_became_active":"2024-03-30T13:12:27.005116+0000","last_became_peered":"2024-03-30T13:12:27.005116+0000","last_unstale":"2024-03-30T13:12:27.006219+0000","last_undegraded":"2024-03-30T13:12:27.006219+0000","last_fullsized":"2024-03-30T13:12:27.006219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978019+0000","last_change":"2024-03-30T13:12:26.978019+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978019+0000","last_undegraded":"2024-03-30T13:12:26.978019+0000","last_fullsized":"2024-03-30T13:12:26.978019+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.999282+0000","last_change":"2024-03-30T13:12:26.998465+0000","last_active":"2024-03-30T13:12:26.999282+0000","last_peered":"2024-03-30T13:12:26.999282+0000","last_clean":"2024-03-30T13:12:26.999282+0000","last_became_active":"2024-03-30T13:12:26.998168+0000","last_became_peered":"2024-03-30T13:12:26.998168+0000","last_unstale":"2024-03-30T13:12:26.999282+0000","last_undegraded":"2024-03-30T13:12:26.999282+0000","last_fullsized":"2024-03-30T13:12:26.999282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:52:53.995529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981851+0000","last_change":"2024-03-30T13:12:26.981851+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981851+0000","last_undegraded":"2024-03-30T13:12:26.981851+0000","last_fullsized":"2024-03-30T13:12:26.981851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022219+0000","last_change":"2024-03-30T13:12:27.009963+0000","last_active":"2024-03-30T13:12:27.022219+0000","last_peered":"2024-03-30T13:12:27.022219+0000","last_clean":"2024-03-30T13:12:27.022219+0000","last_became_active":"2024-03-30T13:12:27.007546+0000","last_became_peered":"2024-03-30T13:12:27.007546+0000","last_unstale":"2024-03-30T13:12:27.022219+0000","last_undegraded":"2024-03-30T13:12:27.022219+0000","last_fullsized":"2024-03-30T13:12:27.022219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:51.709095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980231+0000","last_change":"2024-03-30T13:12:26.980231+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980231+0000","last_undegraded":"2024-03-30T13:12:26.980231+0000","last_fullsized":"2024-03-30T13:12:26.980231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.002506+0000","last_change":"2024-03-30T13:12:27.002016+0000","last_active":"2024-03-30T13:12:27.002506+0000","last_peered":"2024-03-30T13:12:27.002506+0000","last_clean":"2024-03-30T13:12:27.002506+0000","last_became_active":"2024-03-30T13:12:27.001703+0000","last_became_peered":"2024-03-30T13:12:27.001703+0000","last_unstale":"2024-03-30T13:12:27.002506+0000","last_undegraded":"2024-03-30T13:12:27.002506+0000","last_fullsized":"2024-03-30T13:12:27.002506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":200,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.976182+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:12:26.976182+0000","last_peered":"2024-03-30T13:12:26.976182+0000","last_clean":"2024-03-30T13:12:26.976182+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:12:26.976182+0000","last_undegraded":"2024-03-30T13:12:26.976182+0000","last_fullsized":"2024-03-30T13:12:26.976182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":775962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":790403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:12:32.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-03-30T13:12:32.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984672+0000', 'last_change': '2024-03-30T13:12:26.984672+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984672+0000', 'last_undegraded': '2024-03-30T13:12:26.984672+0000', 'last_fullsized': '2024-03-30T13:12:26.984672+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-03-30T13:12:32.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.995194+0000', 'last_change': '2024-03-30T13:12:26.995194+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.995194+0000', 'last_undegraded': '2024-03-30T13:12:26.995194+0000', 'last_fullsized': '2024-03-30T13:12:26.995194+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-03-30T13:12:32.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.988805+0000', 'last_change': '2024-03-30T13:12:26.988805+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.988805+0000', 'last_undegraded': '2024-03-30T13:12:26.988805+0000', 'last_fullsized': '2024-03-30T13:12:26.988805+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.998240+0000', 'last_change': '2024-03-30T13:12:26.998240+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.998240+0000', 'last_undegraded': '2024-03-30T13:12:26.998240+0000', 'last_fullsized': '2024-03-30T13:12:26.998240+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.992828+0000', 'last_change': '2024-03-30T13:12:26.992828+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.992828+0000', 'last_undegraded': '2024-03-30T13:12:26.992828+0000', 'last_fullsized': '2024-03-30T13:12:26.992828+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.995055+0000', 'last_change': '2024-03-30T13:12:26.995055+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.995055+0000', 'last_undegraded': '2024-03-30T13:12:26.995055+0000', 'last_fullsized': '2024-03-30T13:12:26.995055+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984826+0000', 'last_change': '2024-03-30T13:12:26.984826+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984826+0000', 'last_undegraded': '2024-03-30T13:12:26.984826+0000', 'last_fullsized': '2024-03-30T13:12:26.984826+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:27.006045+0000', 'last_change': '2024-03-30T13:12:27.006045+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:27.006045+0000', 'last_undegraded': '2024-03-30T13:12:27.006045+0000', 'last_fullsized': '2024-03-30T13:12:27.006045+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-03-30T13:12:32.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.988102+0000', 'last_change': '2024-03-30T13:12:26.988102+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.988102+0000', 'last_undegraded': '2024-03-30T13:12:26.988102+0000', 'last_fullsized': '2024-03-30T13:12:26.988102+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T13:12:32.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-03-30T13:12:32.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.983703+0000', 'last_change': '2024-03-30T13:12:26.983703+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.983703+0000', 'last_undegraded': '2024-03-30T13:12:26.983703+0000', 'last_fullsized': '2024-03-30T13:12:26.983703+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-03-30T13:12:32.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.983223+0000', 'last_change': '2024-03-30T13:12:26.983223+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.983223+0000', 'last_undegraded': '2024-03-30T13:12:26.983223+0000', 'last_fullsized': '2024-03-30T13:12:26.983223+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.991905+0000', 'last_change': '2024-03-30T13:12:26.991905+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.991905+0000', 'last_undegraded': '2024-03-30T13:12:26.991905+0000', 'last_fullsized': '2024-03-30T13:12:26.991905+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984222+0000', 'last_change': '2024-03-30T13:12:26.984222+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984222+0000', 'last_undegraded': '2024-03-30T13:12:26.984222+0000', 'last_fullsized': '2024-03-30T13:12:26.984222+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.983094+0000', 'last_change': '2024-03-30T13:12:26.983094+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.983094+0000', 'last_undegraded': '2024-03-30T13:12:26.983094+0000', 'last_fullsized': '2024-03-30T13:12:26.983094+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.980561+0000', 'last_change': '2024-03-30T13:12:26.980561+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.980561+0000', 'last_undegraded': '2024-03-30T13:12:26.980561+0000', 'last_fullsized': '2024-03-30T13:12:26.980561+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984611+0000', 'last_change': '2024-03-30T13:12:26.984611+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984611+0000', 'last_undegraded': '2024-03-30T13:12:26.984611+0000', 'last_fullsized': '2024-03-30T13:12:26.984611+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.979796+0000', 'last_change': '2024-03-30T13:12:26.979796+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.979796+0000', 'last_undegraded': '2024-03-30T13:12:26.979796+0000', 'last_fullsized': '2024-03-30T13:12:26.979796+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.980818+0000', 'last_change': '2024-03-30T13:12:26.980818+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.980818+0000', 'last_undegraded': '2024-03-30T13:12:26.980818+0000', 'last_fullsized': '2024-03-30T13:12:26.980818+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.982539+0000', 'last_change': '2024-03-30T13:12:26.982539+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.982539+0000', 'last_undegraded': '2024-03-30T13:12:26.982539+0000', 'last_fullsized': '2024-03-30T13:12:26.982539+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.982370+0000', 'last_change': '2024-03-30T13:12:26.982370+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.982370+0000', 'last_undegraded': '2024-03-30T13:12:26.982370+0000', 'last_fullsized': '2024-03-30T13:12:26.982370+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984680+0000', 'last_change': '2024-03-30T13:12:26.984680+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984680+0000', 'last_undegraded': '2024-03-30T13:12:26.984680+0000', 'last_fullsized': '2024-03-30T13:12:26.984680+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.980169+0000', 'last_change': '2024-03-30T13:12:26.980169+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.980169+0000', 'last_undegraded': '2024-03-30T13:12:26.980169+0000', 'last_fullsized': '2024-03-30T13:12:26.980169+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:27.003382+0000', 'last_change': '2024-03-30T13:12:27.003382+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:27.003382+0000', 'last_undegraded': '2024-03-30T13:12:27.003382+0000', 'last_fullsized': '2024-03-30T13:12:27.003382+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.985367+0000', 'last_change': '2024-03-30T13:12:26.985367+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.985367+0000', 'last_undegraded': '2024-03-30T13:12:26.985367+0000', 'last_fullsized': '2024-03-30T13:12:26.985367+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:27.001705+0000', 'last_change': '2024-03-30T13:12:27.001705+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:27.001705+0000', 'last_undegraded': '2024-03-30T13:12:27.001705+0000', 'last_fullsized': '2024-03-30T13:12:27.001705+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.982634+0000', 'last_change': '2024-03-30T13:12:26.982634+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.982634+0000', 'last_undegraded': '2024-03-30T13:12:26.982634+0000', 'last_fullsized': '2024-03-30T13:12:26.982634+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-03-30T13:12:32.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.982446+0000', 'last_change': '2024-03-30T13:12:26.982446+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.982446+0000', 'last_undegraded': '2024-03-30T13:12:26.982446+0000', 'last_fullsized': '2024-03-30T13:12:26.982446+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.977730+0000', 'last_change': '2024-03-30T13:12:26.977730+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.977730+0000', 'last_undegraded': '2024-03-30T13:12:26.977730+0000', 'last_fullsized': '2024-03-30T13:12:26.977730+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.979106+0000', 'last_change': '2024-03-30T13:12:26.979106+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.979106+0000', 'last_undegraded': '2024-03-30T13:12:26.979106+0000', 'last_fullsized': '2024-03-30T13:12:26.979106+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.982305+0000', 'last_change': '2024-03-30T13:12:26.982305+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.982305+0000', 'last_undegraded': '2024-03-30T13:12:26.982305+0000', 'last_fullsized': '2024-03-30T13:12:26.982305+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.999479+0000', 'last_change': '2024-03-30T13:12:26.999479+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.999479+0000', 'last_undegraded': '2024-03-30T13:12:26.999479+0000', 'last_fullsized': '2024-03-30T13:12:26.999479+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-03-30T13:12:32.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.978170+0000', 'last_change': '2024-03-30T13:12:26.978170+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.978170+0000', 'last_undegraded': '2024-03-30T13:12:26.978170+0000', 'last_fullsized': '2024-03-30T13:12:26.978170+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.987730+0000', 'last_change': '2024-03-30T13:12:26.987730+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.987730+0000', 'last_undegraded': '2024-03-30T13:12:26.987730+0000', 'last_fullsized': '2024-03-30T13:12:26.987730+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.979656+0000', 'last_change': '2024-03-30T13:12:26.979656+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.979656+0000', 'last_undegraded': '2024-03-30T13:12:26.979656+0000', 'last_fullsized': '2024-03-30T13:12:26.979656+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.996306+0000', 'last_change': '2024-03-30T13:12:26.996306+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.996306+0000', 'last_undegraded': '2024-03-30T13:12:26.996306+0000', 'last_fullsized': '2024-03-30T13:12:26.996306+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.993401+0000', 'last_change': '2024-03-30T13:12:26.993401+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.993401+0000', 'last_undegraded': '2024-03-30T13:12:26.993401+0000', 'last_fullsized': '2024-03-30T13:12:26.993401+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.983481+0000', 'last_change': '2024-03-30T13:12:26.983481+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.983481+0000', 'last_undegraded': '2024-03-30T13:12:26.983481+0000', 'last_fullsized': '2024-03-30T13:12:26.983481+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.978581+0000', 'last_change': '2024-03-30T13:12:26.978581+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.978581+0000', 'last_undegraded': '2024-03-30T13:12:26.978581+0000', 'last_fullsized': '2024-03-30T13:12:26.978581+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.988805+0000', 'last_change': '2024-03-30T13:12:26.988805+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.988805+0000', 'last_undegraded': '2024-03-30T13:12:26.988805+0000', 'last_fullsized': '2024-03-30T13:12:26.988805+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.993222+0000', 'last_change': '2024-03-30T13:12:26.993222+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.993222+0000', 'last_undegraded': '2024-03-30T13:12:26.993222+0000', 'last_fullsized': '2024-03-30T13:12:26.993222+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.985404+0000', 'last_change': '2024-03-30T13:12:26.985404+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.985404+0000', 'last_undegraded': '2024-03-30T13:12:26.985404+0000', 'last_fullsized': '2024-03-30T13:12:26.985404+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-03-30T13:12:32.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.993559+0000', 'last_change': '2024-03-30T13:12:26.993559+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.993559+0000', 'last_undegraded': '2024-03-30T13:12:26.993559+0000', 'last_fullsized': '2024-03-30T13:12:26.993559+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.983463+0000', 'last_change': '2024-03-30T13:12:26.983463+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.983463+0000', 'last_undegraded': '2024-03-30T13:12:26.983463+0000', 'last_fullsized': '2024-03-30T13:12:26.983463+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.992123+0000', 'last_change': '2024-03-30T13:12:26.992123+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.992123+0000', 'last_undegraded': '2024-03-30T13:12:26.992123+0000', 'last_fullsized': '2024-03-30T13:12:26.992123+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.991563+0000', 'last_change': '2024-03-30T13:12:26.991563+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.991563+0000', 'last_undegraded': '2024-03-30T13:12:26.991563+0000', 'last_fullsized': '2024-03-30T13:12:26.991563+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.983986+0000', 'last_change': '2024-03-30T13:12:26.983986+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.983986+0000', 'last_undegraded': '2024-03-30T13:12:26.983986+0000', 'last_fullsized': '2024-03-30T13:12:26.983986+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-03-30T13:12:32.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.990026+0000', 'last_change': '2024-03-30T13:12:26.990026+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.990026+0000', 'last_undegraded': '2024-03-30T13:12:26.990026+0000', 'last_fullsized': '2024-03-30T13:12:26.990026+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.986595+0000', 'last_change': '2024-03-30T13:12:26.986595+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.986595+0000', 'last_undegraded': '2024-03-30T13:12:26.986595+0000', 'last_fullsized': '2024-03-30T13:12:26.986595+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.982849+0000', 'last_change': '2024-03-30T13:12:26.982849+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.982849+0000', 'last_undegraded': '2024-03-30T13:12:26.982849+0000', 'last_fullsized': '2024-03-30T13:12:26.982849+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.983151+0000', 'last_change': '2024-03-30T13:12:26.983151+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.983151+0000', 'last_undegraded': '2024-03-30T13:12:26.983151+0000', 'last_fullsized': '2024-03-30T13:12:26.983151+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.978686+0000', 'last_change': '2024-03-30T13:12:26.978686+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.978686+0000', 'last_undegraded': '2024-03-30T13:12:26.978686+0000', 'last_fullsized': '2024-03-30T13:12:26.978686+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-03-30T13:12:32.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984248+0000', 'last_change': '2024-03-30T13:12:26.984248+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984248+0000', 'last_undegraded': '2024-03-30T13:12:26.984248+0000', 'last_fullsized': '2024-03-30T13:12:26.984248+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-03-30T13:12:32.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984409+0000', 'last_change': '2024-03-30T13:12:26.984409+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984409+0000', 'last_undegraded': '2024-03-30T13:12:26.984409+0000', 'last_fullsized': '2024-03-30T13:12:26.984409+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-03-30T13:12:32.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.984702+0000', 'last_change': '2024-03-30T13:12:26.984702+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.984702+0000', 'last_undegraded': '2024-03-30T13:12:26.984702+0000', 'last_fullsized': '2024-03-30T13:12:26.984702+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-30T13:12:32.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-03-30T13:12:32.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.989414+0000', 'last_change': '2024-03-30T13:12:26.989414+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.989414+0000', 'last_undegraded': '2024-03-30T13:12:26.989414+0000', 'last_fullsized': '2024-03-30T13:12:26.989414+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T13:12:32.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-03-30T13:12:32.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.981991+0000', 'last_change': '2024-03-30T13:12:26.981991+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.981991+0000', 'last_undegraded': '2024-03-30T13:12:26.981991+0000', 'last_fullsized': '2024-03-30T13:12:26.981991+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-03-30T13:12:32.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.986444+0000', 'last_change': '2024-03-30T13:12:26.986444+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.986444+0000', 'last_undegraded': '2024-03-30T13:12:26.986444+0000', 'last_fullsized': '2024-03-30T13:12:26.986444+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-30T13:12:32.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-03-30T13:12:32.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.990485+0000', 'last_change': '2024-03-30T13:12:26.990485+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.990485+0000', 'last_undegraded': '2024-03-30T13:12:26.990485+0000', 'last_fullsized': '2024-03-30T13:12:26.990485+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-30T13:12:32.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-03-30T13:12:32.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.978019+0000', 'last_change': '2024-03-30T13:12:26.978019+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.978019+0000', 'last_undegraded': '2024-03-30T13:12:26.978019+0000', 'last_fullsized': '2024-03-30T13:12:26.978019+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-03-30T13:12:32.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.981851+0000', 'last_change': '2024-03-30T13:12:26.981851+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.981851+0000', 'last_undegraded': '2024-03-30T13:12:26.981851+0000', 'last_fullsized': '2024-03-30T13:12:26.981851+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-30T13:12:32.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-03-30T13:12:32.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 39, 'state': 'creating+peering', 'last_fresh': '2024-03-30T13:12:26.980231+0000', 'last_change': '2024-03-30T13:12:26.980231+0000', 'last_active': '2024-03-30T13:12:26.967503+0000', 'last_peered': '2024-03-30T13:12:26.967503+0000', 'last_clean': '2024-03-30T13:12:26.967503+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-30T13:12:26.980231+0000', 'last_undegraded': '2024-03-30T13:12:26.980231+0000', 'last_fullsized': '2024-03-30T13:12:26.980231+0000', 'mapping_epoch': 39, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 39, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'last_clean_scrub_stamp': '2024-03-30T13:12:26.967503+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-30T13:12:32.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-30T13:12:32.964 INFO:tasks.ceph.mds.d.smithi113.stderr:2024-03-30T13:12:32.958+0000 7f68bb86e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-30T13:12:33.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba38f26640 1 Processor -- start 2024-03-30T13:12:33.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba38f26640 1 -- start start 2024-03-30T13:12:33.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba38f26640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fba34144270 0x7fba34144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba38f26640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba3415f7a0 0x7fba3406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba38f26640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba340b4ba0 con 0x7fba340b2ef0 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba38f26640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fba34144fa0 con 0x7fba3415f7a0 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba38f26640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fba340b33f0 con 0x7fba34144270 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba32d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba3415f7a0 0x7fba3406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba2bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fba340b2ef0 0x7fba340b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34506/0 (socket says 172.21.15.43:34506) 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba32d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba3415f7a0 0x7fba3406c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34904/0 (socket says 172.21.15.43:34904) 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba2bfff640 1 -- 172.21.15.43:0/2265543612 learned_addr learned my addr 172.21.15.43:0/2265543612 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba3259c640 1 --2- 172.21.15.43:0/2265543612 >> v2:172.21.15.113:3300/0 conn(0x7fba34144270 0x7fba34144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba32d9d640 1 -- 172.21.15.43:0/2265543612 >> v2:172.21.15.113:3300/0 conn(0x7fba34144270 msgr2=0x7fba34144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:33.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba32d9d640 1 --2- 172.21.15.43:0/2265543612 >> v2:172.21.15.113:3300/0 conn(0x7fba34144270 0x7fba34144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba32d9d640 1 -- 172.21.15.43:0/2265543612 >> v1:172.21.15.43:6789/0 conn(0x7fba340b2ef0 legacy=0x7fba340b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba32d9d640 1 -- 172.21.15.43:0/2265543612 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba340b36d0 con 0x7fba3415f7a0 2024-03-30T13:12:33.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:32.999+0000 7fba31d9b640 1 -- 172.21.15.43:0/2265543612 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3343511501 0 0) 0x7fba340b4ba0 con 0x7fba340b2ef0 2024-03-30T13:12:33.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba32d9d640 1 --2- 172.21.15.43:0/2265543612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406c670 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fba34072000 tx=0x7fba2c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71ce8ed198f541d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:33.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba31d9b640 1 -- 172.21.15.43:0/2265543612 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0b2070 con 0x7fba3415f7a0 2024-03-30T13:12:33.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba31d9b640 1 -- 172.21.15.43:0/2265543612 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c0a58d0 con 0x7fba3415f7a0 2024-03-30T13:12:33.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba31d9b640 1 -- 172.21.15.43:0/2265543612 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0a5ca0 con 0x7fba3415f7a0 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 -- 172.21.15.43:0/2265543612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 msgr2=0x7fba3406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 --2- 172.21.15.43:0/2265543612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406c670 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fba34072000 tx=0x7fba2c01c900 comp rx=0 tx=0).stop 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 -- 172.21.15.43:0/2265543612 shutdown_connections 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 --2- 172.21.15.43:0/2265543612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406c670 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 --2- 172.21.15.43:0/2265543612 >> v2:172.21.15.113:3300/0 conn(0x7fba34144270 0x7fba34144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 -- 172.21.15.43:0/2265543612 >> 172.21.15.43:0/2265543612 conn(0x7fba340cfee0 msgr2=0x7fba340d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 -- 172.21.15.43:0/2265543612 shutdown_connections 2024-03-30T13:12:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.003+0000 7fba38f26640 1 -- 172.21.15.43:0/2265543612 wait complete. 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 Processor -- start 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 -- start start 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba340b2ef0 0x7fba3415aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba34144270 0x7fba3406dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fba340cdd10 con 0x7fba3415f7a0 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fba3415fc30 con 0x7fba340b2ef0 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fba340cde90 con 0x7fba34144270 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34912/0 (socket says 172.21.15.43:34912) 2024-03-30T13:12:33.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 -- 172.21.15.43:0/2025691414 learned_addr learned my addr 172.21.15.43:0/2025691414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba3259c640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba340b2ef0 0x7fba3415aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba2bfff640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba34144270 0x7fba3406dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 -- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba34144270 msgr2=0x7fba3406dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba34144270 0x7fba3406dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 -- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba340b2ef0 msgr2=0x7fba3415aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba340b2ef0 0x7fba3415aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 -- 172.21.15.43:0/2025691414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3415c250 con 0x7fba3415f7a0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba3259c640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba340b2ef0 0x7fba3415aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba32d9d640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406e150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0b0040 tx=0x7fba2c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba2affd640 1 -- 172.21.15.43:0/2025691414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0b2050 con 0x7fba3415f7a0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba2affd640 1 -- 172.21.15.43:0/2025691414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c0be070 con 0x7fba3415f7a0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba2affd640 1 -- 172.21.15.43:0/2025691414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0ada90 con 0x7fba3415f7a0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3415c530 con 0x7fba3415f7a0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3415caf0 con 0x7fba3415f7a0 2024-03-30T13:12:33.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba34068890 con 0x7fba3415f7a0 2024-03-30T13:12:33.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba2affd640 1 -- 172.21.15.43:0/2025691414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fba2c0adc30 con 0x7fba3415f7a0 2024-03-30T13:12:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.007+0000 7fba2affd640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fba14067fc0 0x7fba1406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.011+0000 7fba2affd640 1 -- 172.21.15.43:0/2025691414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7fba2c1219b0 con 0x7fba3415f7a0 2024-03-30T13:12:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.011+0000 7fba3259c640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fba14067fc0 0x7fba1406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.011+0000 7fba3259c640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fba14067fc0 0x7fba1406a480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b9f0 tx=0x7fba24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:33.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.015+0000 7fba2affd640 1 -- 172.21.15.43:0/2025691414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fba2c016520 con 0x7fba3415f7a0 2024-03-30T13:12:33.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.239+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fba340d05a0 con 0x7fba3415f7a0 2024-03-30T13:12:33.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.239+0000 7fba2affd640 1 -- 172.21.15.43:0/2025691414 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1095 (secure 0 0 0) 0x7fba2c0ab340 con 0x7fba3415f7a0 2024-03-30T13:12:33.247 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:33.247 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":122,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":8,"num_up_osds":8,"osd_up_since":1711804242,"num_in_osds":8,"osd_in_since":1711804233,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":61},{"state_name":"active+clean","count":36}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":247390208,"bytes_avail":772846723072,"bytes_total":773094113280,"inactive_pgs_ratio":0.62886595726013184,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":18,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":4880}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-03-30T13:12:01.861638+0000","services":{}},"progress_events":{}} 2024-03-30T13:12:33.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fba14067fc0 msgr2=0x7fba1406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fba14067fc0 0x7fba1406a480 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fba2400b9f0 tx=0x7fba24008040 comp rx=0 tx=0).stop 2024-03-30T13:12:33.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 msgr2=0x7fba3406e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406e150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0b0040 tx=0x7fba2c0196a0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 shutdown_connections 2024-03-30T13:12:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fba14067fc0 0x7fba1406a480 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba3415f7a0 0x7fba3406e150 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fba34144270 0x7fba3406dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 --2- 172.21.15.43:0/2025691414 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fba340b2ef0 0x7fba3415aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 >> 172.21.15.43:0/2025691414 conn(0x7fba340cfee0 msgr2=0x7fba3406b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 shutdown_connections 2024-03-30T13:12:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.243+0000 7fba38f26640 1 -- 172.21.15.43:0/2025691414 wait complete. 2024-03-30T13:12:33.266 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-30T13:12:33.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:12:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.375+0000 7ff36b296640 1 Processor -- start 2024-03-30T13:12:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.375+0000 7ff36b296640 1 -- start start 2024-03-30T13:12:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff36b296640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 0x7ff364144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff36b296640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff3640b2ef0 0x7ff36415f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff36b296640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3640b33f0 con 0x7ff3640b4490 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff36b296640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff364144fa0 con 0x7ff3640b2ef0 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff36b296640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff36415fce0 con 0x7ff364144270 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff369833640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff3640b4490 0x7ff36406c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34510/0 (socket says 172.21.15.43:34510) 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff369833640 1 -- 172.21.15.43:0/379990831 learned_addr learned my addr 172.21.15.43:0/379990831 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff369032640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 0x7ff364144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff368831640 1 --2- 172.21.15.43:0/379990831 >> v2:172.21.15.113:3300/0 conn(0x7ff3640b2ef0 0x7ff36415f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2601342591 0 0) 0x7ff3640b33f0 con 0x7ff3640b4490 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff350005740 con 0x7ff3640b4490 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2330982184 0 0) 0x7ff350005740 con 0x7ff3640b4490 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 msgr2=0x7ff364144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 0x7ff364144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 >> v2:172.21.15.113:3300/0 conn(0x7ff3640b2ef0 msgr2=0x7ff36415f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 --2- 172.21.15.43:0/379990831 >> v2:172.21.15.113:3300/0 conn(0x7ff3640b2ef0 0x7ff36415f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3640b36d0 con 0x7ff3640b4490 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ff35c001a00 con 0x7ff3640b4490 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 >> v1:172.21.15.43:6789/0 conn(0x7ff3640b4490 legacy=0x7ff36406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350005bd0 0x7ff350005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff350006a20 0x7ff350006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 0x7ff3500073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff36415fce0 con 0x7ff350006a20 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff35000dbe0 con 0x7ff350005bd0 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff35000dd60 con 0x7ff364144270 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff368831640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff350006a20 0x7ff350006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff369032640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350005bd0 0x7ff350005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff369833640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 0x7ff3500073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff368831640 1 -- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 msgr2=0x7ff3500073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff368831640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff364144270 0x7ff3500073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff368831640 1 -- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350005bd0 msgr2=0x7ff350005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff368831640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff350005bd0 0x7ff350005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.379+0000 7ff368831640 1 -- 172.21.15.43:0/379990831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff350005740 con 0x7ff350006a20 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff368831640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff350006a20 0x7ff350006ea0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff354009ae0 tx=0x7ff35401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89484e81af154e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540096b0 con 0x7ff350006a20 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3540a64c0 con 0x7ff350006a20 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff35bfff640 1 -- 172.21.15.43:0/379990831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540a68b0 con 0x7ff350006a20 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- 172.21.15.43:0/379990831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff350006a20 msgr2=0x7ff350006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 --2- 172.21.15.43:0/379990831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff350006a20 0x7ff350006ea0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff354009ae0 tx=0x7ff35401cab0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff369833640 1 -- 172.21.15.43:0/379990831 reap_dead start 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- 172.21.15.43:0/379990831 shutdown_connections 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- 172.21.15.43:0/379990831 >> 172.21.15.43:0/379990831 conn(0x7ff3640cfee0 msgr2=0x7ff3640d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- 172.21.15.43:0/379990831 shutdown_connections 2024-03-30T13:12:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- 172.21.15.43:0/379990831 wait complete. 2024-03-30T13:12:33.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 Processor -- start 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- start start 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff350006a20 0x7ff3641f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff36415d6d0 0x7ff36415db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3641383d0 0x7ff364138850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff36406cd80 con 0x7ff36415d6d0 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff3640b3f70 con 0x7ff3641383d0 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff36b296640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff36406cf00 con 0x7ff350006a20 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff368831640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff36415d6d0 0x7ff36415db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff368831640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff36415d6d0 0x7ff36415db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34928/0 (socket says 172.21.15.43:34928) 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.383+0000 7ff368831640 1 -- 172.21.15.43:0/1048971722 learned_addr learned my addr 172.21.15.43:0/1048971722 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff369032640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff350006a20 0x7ff3641f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff369833640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3641383d0 0x7ff364138850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff368831640 1 -- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff350006a20 msgr2=0x7ff3641f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff368831640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff350006a20 0x7ff3641f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff368831640 1 -- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3641383d0 msgr2=0x7ff364138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff368831640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3641383d0 0x7ff364138850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff368831640 1 -- 172.21.15.43:0/1048971722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff364139130 con 0x7ff36415d6d0 2024-03-30T13:12:33.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff368831640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff36415d6d0 0x7ff36415db30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff3540b0040 tx=0x7ff350007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:33.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff359ffb640 1 -- 172.21.15.43:0/1048971722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540be020 con 0x7ff36415d6d0 2024-03-30T13:12:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff359ffb640 1 -- 172.21.15.43:0/1048971722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3540a6a80 con 0x7ff36415d6d0 2024-03-30T13:12:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff359ffb640 1 -- 172.21.15.43:0/1048971722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff354006040 con 0x7ff36415d6d0 2024-03-30T13:12:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff364139410 con 0x7ff36415d6d0 2024-03-30T13:12:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3641399d0 con 0x7ff36415d6d0 2024-03-30T13:12:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff364068890 con 0x7ff36415d6d0 2024-03-30T13:12:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff359ffb640 1 -- 172.21.15.43:0/1048971722 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff3540c1020 con 0x7ff36415d6d0 2024-03-30T13:12:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff359ffb640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff348068010 0x7ff34806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff359ffb640 1 -- 172.21.15.43:0/1048971722 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7ff354121520 con 0x7ff36415d6d0 2024-03-30T13:12:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff369032640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff348068010 0x7ff34806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.387+0000 7ff369032640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff348068010 0x7ff34806a4d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff360009b00 tx=0x7ff360009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.391+0000 7ff359ffb640 1 -- 172.21.15.43:0/1048971722 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff3540c8030 con 0x7ff36415d6d0 2024-03-30T13:12:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.547+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff3640c9f30 con 0x7ff348068010 2024-03-30T13:12:33.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.567+0000 7ff359ffb640 1 -- 172.21.15.43:0/1048971722 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246895 (secure 0 0 0) 0x7ff3640c9f30 con 0x7ff348068010 2024-03-30T13:12:33.576 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:12:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff348068010 msgr2=0x7ff34806a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff348068010 0x7ff34806a4d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff360009b00 tx=0x7ff360009190 comp rx=0 tx=0).stop 2024-03-30T13:12:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff36415d6d0 msgr2=0x7ff36415db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff36415d6d0 0x7ff36415db30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff3540b0040 tx=0x7ff350007a50 comp rx=0 tx=0).stop 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 shutdown_connections 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff348068010 0x7ff34806a4d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff3641383d0 0x7ff364138850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff36415d6d0 0x7ff36415db30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 --2- 172.21.15.43:0/1048971722 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff350006a20 0x7ff3641f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 >> 172.21.15.43:0/1048971722 conn(0x7ff3640cfee0 msgr2=0x7ff3640ce8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 shutdown_connections 2024-03-30T13:12:33.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:33.571+0000 7ff36b296640 1 -- 172.21.15.43:0/1048971722 wait complete. 2024-03-30T13:12:33.593 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":90,"stamp":"2024-03-30T13:12:31.871688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":160,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241592,"kb_used_data":4620,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733128,"statfs":{"total":773094113280,"available":772846723072,"internally_reserved":0,"allocated":4730880,"data_stored":2666595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":242535986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.902330"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984672+0000","last_change":"2024-03-30T13:12:26.984672+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984672+0000","last_undegraded":"2024-03-30T13:12:26.984672+0000","last_fullsized":"2024-03-30T13:12:26.984672+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995194+0000","last_change":"2024-03-30T13:12:26.995194+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995194+0000","last_undegraded":"2024-03-30T13:12:26.995194+0000","last_fullsized":"2024-03-30T13:12:26.995194+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.998240+0000","last_change":"2024-03-30T13:12:26.998240+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.998240+0000","last_undegraded":"2024-03-30T13:12:26.998240+0000","last_fullsized":"2024-03-30T13:12:26.998240+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992828+0000","last_change":"2024-03-30T13:12:26.992828+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992828+0000","last_undegraded":"2024-03-30T13:12:26.992828+0000","last_fullsized":"2024-03-30T13:12:26.992828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.995055+0000","last_change":"2024-03-30T13:12:26.995055+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.995055+0000","last_undegraded":"2024-03-30T13:12:26.995055+0000","last_fullsized":"2024-03-30T13:12:26.995055+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984826+0000","last_change":"2024-03-30T13:12:26.984826+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984826+0000","last_undegraded":"2024-03-30T13:12:26.984826+0000","last_fullsized":"2024-03-30T13:12:26.984826+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.006045+0000","last_change":"2024-03-30T13:12:27.006045+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.006045+0000","last_undegraded":"2024-03-30T13:12:27.006045+0000","last_fullsized":"2024-03-30T13:12:27.006045+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988102+0000","last_change":"2024-03-30T13:12:26.988102+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988102+0000","last_undegraded":"2024-03-30T13:12:26.988102+0000","last_fullsized":"2024-03-30T13:12:26.988102+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983703+0000","last_change":"2024-03-30T13:12:26.983703+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983703+0000","last_undegraded":"2024-03-30T13:12:26.983703+0000","last_fullsized":"2024-03-30T13:12:26.983703+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983223+0000","last_change":"2024-03-30T13:12:26.983223+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983223+0000","last_undegraded":"2024-03-30T13:12:26.983223+0000","last_fullsized":"2024-03-30T13:12:26.983223+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991905+0000","last_change":"2024-03-30T13:12:26.991905+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991905+0000","last_undegraded":"2024-03-30T13:12:26.991905+0000","last_fullsized":"2024-03-30T13:12:26.991905+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984222+0000","last_change":"2024-03-30T13:12:26.984222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984222+0000","last_undegraded":"2024-03-30T13:12:26.984222+0000","last_fullsized":"2024-03-30T13:12:26.984222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983094+0000","last_change":"2024-03-30T13:12:26.983094+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983094+0000","last_undegraded":"2024-03-30T13:12:26.983094+0000","last_fullsized":"2024-03-30T13:12:26.983094+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980561+0000","last_change":"2024-03-30T13:12:26.980561+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980561+0000","last_undegraded":"2024-03-30T13:12:26.980561+0000","last_fullsized":"2024-03-30T13:12:26.980561+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984611+0000","last_change":"2024-03-30T13:12:26.984611+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984611+0000","last_undegraded":"2024-03-30T13:12:26.984611+0000","last_fullsized":"2024-03-30T13:12:26.984611+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979796+0000","last_change":"2024-03-30T13:12:26.979796+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979796+0000","last_undegraded":"2024-03-30T13:12:26.979796+0000","last_fullsized":"2024-03-30T13:12:26.979796+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980818+0000","last_change":"2024-03-30T13:12:26.980818+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980818+0000","last_undegraded":"2024-03-30T13:12:26.980818+0000","last_fullsized":"2024-03-30T13:12:26.980818+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028469+0000","last_change":"2024-03-30T13:12:28.035833+0000","last_active":"2024-03-30T13:12:29.028469+0000","last_peered":"2024-03-30T13:12:29.028469+0000","last_clean":"2024-03-30T13:12:29.028469+0000","last_became_active":"2024-03-30T13:12:28.035506+0000","last_became_peered":"2024-03-30T13:12:28.035506+0000","last_unstale":"2024-03-30T13:12:29.028469+0000","last_undegraded":"2024-03-30T13:12:29.028469+0000","last_fullsized":"2024-03-30T13:12:29.028469+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:18:21.093109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982539+0000","last_change":"2024-03-30T13:12:26.982539+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982539+0000","last_undegraded":"2024-03-30T13:12:26.982539+0000","last_fullsized":"2024-03-30T13:12:26.982539+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982370+0000","last_change":"2024-03-30T13:12:26.982370+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982370+0000","last_undegraded":"2024-03-30T13:12:26.982370+0000","last_fullsized":"2024-03-30T13:12:26.982370+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984680+0000","last_change":"2024-03-30T13:12:26.984680+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984680+0000","last_undegraded":"2024-03-30T13:12:26.984680+0000","last_fullsized":"2024-03-30T13:12:26.984680+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980169+0000","last_change":"2024-03-30T13:12:26.980169+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980169+0000","last_undegraded":"2024-03-30T13:12:26.980169+0000","last_fullsized":"2024-03-30T13:12:26.980169+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.003382+0000","last_change":"2024-03-30T13:12:27.003382+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.003382+0000","last_undegraded":"2024-03-30T13:12:27.003382+0000","last_fullsized":"2024-03-30T13:12:27.003382+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985367+0000","last_change":"2024-03-30T13:12:26.985367+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985367+0000","last_undegraded":"2024-03-30T13:12:26.985367+0000","last_fullsized":"2024-03-30T13:12:26.985367+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:27.001705+0000","last_change":"2024-03-30T13:12:27.001705+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:27.001705+0000","last_undegraded":"2024-03-30T13:12:27.001705+0000","last_fullsized":"2024-03-30T13:12:27.001705+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982634+0000","last_change":"2024-03-30T13:12:26.982634+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982634+0000","last_undegraded":"2024-03-30T13:12:26.982634+0000","last_fullsized":"2024-03-30T13:12:26.982634+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982446+0000","last_change":"2024-03-30T13:12:26.982446+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982446+0000","last_undegraded":"2024-03-30T13:12:26.982446+0000","last_fullsized":"2024-03-30T13:12:26.982446+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.977730+0000","last_change":"2024-03-30T13:12:26.977730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.977730+0000","last_undegraded":"2024-03-30T13:12:26.977730+0000","last_fullsized":"2024-03-30T13:12:26.977730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979106+0000","last_change":"2024-03-30T13:12:26.979106+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979106+0000","last_undegraded":"2024-03-30T13:12:26.979106+0000","last_fullsized":"2024-03-30T13:12:26.979106+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982305+0000","last_change":"2024-03-30T13:12:26.982305+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982305+0000","last_undegraded":"2024-03-30T13:12:26.982305+0000","last_fullsized":"2024-03-30T13:12:26.982305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.999479+0000","last_change":"2024-03-30T13:12:26.999479+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.999479+0000","last_undegraded":"2024-03-30T13:12:26.999479+0000","last_fullsized":"2024-03-30T13:12:26.999479+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978170+0000","last_change":"2024-03-30T13:12:26.978170+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978170+0000","last_undegraded":"2024-03-30T13:12:26.978170+0000","last_fullsized":"2024-03-30T13:12:26.978170+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.995591+0000","last_change":"2024-03-30T13:12:26.994626+0000","last_active":"2024-03-30T13:12:26.995591+0000","last_peered":"2024-03-30T13:12:26.995591+0000","last_clean":"2024-03-30T13:12:26.995591+0000","last_became_active":"2024-03-30T13:12:26.994312+0000","last_became_peered":"2024-03-30T13:12:26.994312+0000","last_unstale":"2024-03-30T13:12:26.995591+0000","last_undegraded":"2024-03-30T13:12:26.995591+0000","last_fullsized":"2024-03-30T13:12:26.995591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:46:32.922336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.987730+0000","last_change":"2024-03-30T13:12:26.987730+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.987730+0000","last_undegraded":"2024-03-30T13:12:26.987730+0000","last_fullsized":"2024-03-30T13:12:26.987730+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025150+0000","last_change":"2024-03-30T13:12:27.020872+0000","last_active":"2024-03-30T13:12:27.025150+0000","last_peered":"2024-03-30T13:12:27.025150+0000","last_clean":"2024-03-30T13:12:27.025150+0000","last_became_active":"2024-03-30T13:12:27.020255+0000","last_became_peered":"2024-03-30T13:12:27.020255+0000","last_unstale":"2024-03-30T13:12:27.025150+0000","last_undegraded":"2024-03-30T13:12:27.025150+0000","last_fullsized":"2024-03-30T13:12:27.025150+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.979656+0000","last_change":"2024-03-30T13:12:26.979656+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.979656+0000","last_undegraded":"2024-03-30T13:12:26.979656+0000","last_fullsized":"2024-03-30T13:12:26.979656+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026909+0000","last_change":"2024-03-30T13:12:27.024163+0000","last_active":"2024-03-30T13:12:27.026909+0000","last_peered":"2024-03-30T13:12:27.026909+0000","last_clean":"2024-03-30T13:12:27.026909+0000","last_became_active":"2024-03-30T13:12:27.023530+0000","last_became_peered":"2024-03-30T13:12:27.023530+0000","last_unstale":"2024-03-30T13:12:27.026909+0000","last_undegraded":"2024-03-30T13:12:27.026909+0000","last_fullsized":"2024-03-30T13:12:27.026909+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:58:35.953478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.996306+0000","last_change":"2024-03-30T13:12:26.996306+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.996306+0000","last_undegraded":"2024-03-30T13:12:26.996306+0000","last_fullsized":"2024-03-30T13:12:26.996306+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.018387+0000","last_change":"2024-03-30T13:12:27.017703+0000","last_active":"2024-03-30T13:12:27.018387+0000","last_peered":"2024-03-30T13:12:27.018387+0000","last_clean":"2024-03-30T13:12:27.018387+0000","last_became_active":"2024-03-30T13:12:27.017417+0000","last_became_peered":"2024-03-30T13:12:27.017417+0000","last_unstale":"2024-03-30T13:12:27.018387+0000","last_undegraded":"2024-03-30T13:12:27.018387+0000","last_fullsized":"2024-03-30T13:12:27.018387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:32:06.564626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993401+0000","last_change":"2024-03-30T13:12:26.993401+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993401+0000","last_undegraded":"2024-03-30T13:12:26.993401+0000","last_fullsized":"2024-03-30T13:12:26.993401+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.005997+0000","last_change":"2024-03-30T13:12:27.005524+0000","last_active":"2024-03-30T13:12:27.005997+0000","last_peered":"2024-03-30T13:12:27.005997+0000","last_clean":"2024-03-30T13:12:27.005997+0000","last_became_active":"2024-03-30T13:12:27.005130+0000","last_became_peered":"2024-03-30T13:12:27.005130+0000","last_unstale":"2024-03-30T13:12:27.005997+0000","last_undegraded":"2024-03-30T13:12:27.005997+0000","last_fullsized":"2024-03-30T13:12:27.005997+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983481+0000","last_change":"2024-03-30T13:12:26.983481+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983481+0000","last_undegraded":"2024-03-30T13:12:26.983481+0000","last_fullsized":"2024-03-30T13:12:26.983481+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000683+0000","last_change":"2024-03-30T13:12:26.999973+0000","last_active":"2024-03-30T13:12:27.000683+0000","last_peered":"2024-03-30T13:12:27.000683+0000","last_clean":"2024-03-30T13:12:27.000683+0000","last_became_active":"2024-03-30T13:12:26.999607+0000","last_became_peered":"2024-03-30T13:12:26.999607+0000","last_unstale":"2024-03-30T13:12:27.000683+0000","last_undegraded":"2024-03-30T13:12:27.000683+0000","last_fullsized":"2024-03-30T13:12:27.000683+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:14:13.186516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978581+0000","last_change":"2024-03-30T13:12:26.978581+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978581+0000","last_undegraded":"2024-03-30T13:12:26.978581+0000","last_fullsized":"2024-03-30T13:12:26.978581+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.020714+0000","last_change":"2024-03-30T13:12:27.009633+0000","last_active":"2024-03-30T13:12:27.020714+0000","last_peered":"2024-03-30T13:12:27.020714+0000","last_clean":"2024-03-30T13:12:27.020714+0000","last_became_active":"2024-03-30T13:12:27.008472+0000","last_became_peered":"2024-03-30T13:12:27.008472+0000","last_unstale":"2024-03-30T13:12:27.020714+0000","last_undegraded":"2024-03-30T13:12:27.020714+0000","last_fullsized":"2024-03-30T13:12:27.020714+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:12.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.988805+0000","last_change":"2024-03-30T13:12:26.988805+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.988805+0000","last_undegraded":"2024-03-30T13:12:26.988805+0000","last_fullsized":"2024-03-30T13:12:26.988805+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992927+0000","last_change":"2024-03-30T13:12:26.991064+0000","last_active":"2024-03-30T13:12:26.992927+0000","last_peered":"2024-03-30T13:12:26.992927+0000","last_clean":"2024-03-30T13:12:26.992927+0000","last_became_active":"2024-03-30T13:12:26.990695+0000","last_became_peered":"2024-03-30T13:12:26.990695+0000","last_unstale":"2024-03-30T13:12:26.992927+0000","last_undegraded":"2024-03-30T13:12:26.992927+0000","last_fullsized":"2024-03-30T13:12:26.992927+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:31:17.730101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993222+0000","last_change":"2024-03-30T13:12:26.993222+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993222+0000","last_undegraded":"2024-03-30T13:12:26.993222+0000","last_fullsized":"2024-03-30T13:12:26.993222+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021848+0000","last_change":"2024-03-30T13:12:27.014670+0000","last_active":"2024-03-30T13:12:27.021848+0000","last_peered":"2024-03-30T13:12:27.021848+0000","last_clean":"2024-03-30T13:12:27.021848+0000","last_became_active":"2024-03-30T13:12:27.010212+0000","last_became_peered":"2024-03-30T13:12:27.010212+0000","last_unstale":"2024-03-30T13:12:27.021848+0000","last_undegraded":"2024-03-30T13:12:27.021848+0000","last_fullsized":"2024-03-30T13:12:27.021848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:21.640047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.985404+0000","last_change":"2024-03-30T13:12:26.985404+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.985404+0000","last_undegraded":"2024-03-30T13:12:26.985404+0000","last_fullsized":"2024-03-30T13:12:26.985404+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.014887+0000","last_change":"2024-03-30T13:12:27.014223+0000","last_active":"2024-03-30T13:12:27.014887+0000","last_peered":"2024-03-30T13:12:27.014887+0000","last_clean":"2024-03-30T13:12:27.014887+0000","last_became_active":"2024-03-30T13:12:27.013881+0000","last_became_peered":"2024-03-30T13:12:27.013881+0000","last_unstale":"2024-03-30T13:12:27.014887+0000","last_undegraded":"2024-03-30T13:12:27.014887+0000","last_fullsized":"2024-03-30T13:12:27.014887+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:55.961184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.993559+0000","last_change":"2024-03-30T13:12:26.993559+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.993559+0000","last_undegraded":"2024-03-30T13:12:26.993559+0000","last_fullsized":"2024-03-30T13:12:26.993559+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009586+0000","last_change":"2024-03-30T13:12:27.008286+0000","last_active":"2024-03-30T13:12:27.009586+0000","last_peered":"2024-03-30T13:12:27.009586+0000","last_clean":"2024-03-30T13:12:27.009586+0000","last_became_active":"2024-03-30T13:12:27.007966+0000","last_became_peered":"2024-03-30T13:12:27.007966+0000","last_unstale":"2024-03-30T13:12:27.009586+0000","last_undegraded":"2024-03-30T13:12:27.009586+0000","last_fullsized":"2024-03-30T13:12:27.009586+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:15:11.139750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983463+0000","last_change":"2024-03-30T13:12:26.983463+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983463+0000","last_undegraded":"2024-03-30T13:12:26.983463+0000","last_fullsized":"2024-03-30T13:12:26.983463+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022912+0000","last_change":"2024-03-30T13:12:27.019139+0000","last_active":"2024-03-30T13:12:27.022912+0000","last_peered":"2024-03-30T13:12:27.022912+0000","last_clean":"2024-03-30T13:12:27.022912+0000","last_became_active":"2024-03-30T13:12:27.017110+0000","last_became_peered":"2024-03-30T13:12:27.017110+0000","last_unstale":"2024-03-30T13:12:27.022912+0000","last_undegraded":"2024-03-30T13:12:27.022912+0000","last_fullsized":"2024-03-30T13:12:27.022912+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.992123+0000","last_change":"2024-03-30T13:12:26.992123+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.992123+0000","last_undegraded":"2024-03-30T13:12:26.992123+0000","last_fullsized":"2024-03-30T13:12:26.992123+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026968+0000","last_change":"2024-03-30T13:12:27.024711+0000","last_active":"2024-03-30T13:12:27.026968+0000","last_peered":"2024-03-30T13:12:27.026968+0000","last_clean":"2024-03-30T13:12:27.026968+0000","last_became_active":"2024-03-30T13:12:27.024207+0000","last_became_peered":"2024-03-30T13:12:27.024207+0000","last_unstale":"2024-03-30T13:12:27.026968+0000","last_undegraded":"2024-03-30T13:12:27.026968+0000","last_fullsized":"2024-03-30T13:12:27.026968+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:24.223034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.991563+0000","last_change":"2024-03-30T13:12:26.991563+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.991563+0000","last_undegraded":"2024-03-30T13:12:26.991563+0000","last_fullsized":"2024-03-30T13:12:26.991563+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.021842+0000","last_change":"2024-03-30T13:12:27.019279+0000","last_active":"2024-03-30T13:12:27.021842+0000","last_peered":"2024-03-30T13:12:27.021842+0000","last_clean":"2024-03-30T13:12:27.021842+0000","last_became_active":"2024-03-30T13:12:27.013568+0000","last_became_peered":"2024-03-30T13:12:27.013568+0000","last_unstale":"2024-03-30T13:12:27.021842+0000","last_undegraded":"2024-03-30T13:12:27.021842+0000","last_fullsized":"2024-03-30T13:12:27.021842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:56:53.365286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983986+0000","last_change":"2024-03-30T13:12:26.983986+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983986+0000","last_undegraded":"2024-03-30T13:12:26.983986+0000","last_fullsized":"2024-03-30T13:12:26.983986+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.990320+0000","last_change":"2024-03-30T13:12:26.989155+0000","last_active":"2024-03-30T13:12:26.990320+0000","last_peered":"2024-03-30T13:12:26.990320+0000","last_clean":"2024-03-30T13:12:26.990320+0000","last_became_active":"2024-03-30T13:12:26.988891+0000","last_became_peered":"2024-03-30T13:12:26.988891+0000","last_unstale":"2024-03-30T13:12:26.990320+0000","last_undegraded":"2024-03-30T13:12:26.990320+0000","last_fullsized":"2024-03-30T13:12:26.990320+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:13:02.680729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990026+0000","last_change":"2024-03-30T13:12:26.990026+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990026+0000","last_undegraded":"2024-03-30T13:12:26.990026+0000","last_fullsized":"2024-03-30T13:12:26.990026+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.993878+0000","last_change":"2024-03-30T13:12:26.993198+0000","last_active":"2024-03-30T13:12:26.993878+0000","last_peered":"2024-03-30T13:12:26.993878+0000","last_clean":"2024-03-30T13:12:26.993878+0000","last_became_active":"2024-03-30T13:12:26.992905+0000","last_became_peered":"2024-03-30T13:12:26.992905+0000","last_unstale":"2024-03-30T13:12:26.993878+0000","last_undegraded":"2024-03-30T13:12:26.993878+0000","last_fullsized":"2024-03-30T13:12:26.993878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:41:04.966244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986595+0000","last_change":"2024-03-30T13:12:26.986595+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986595+0000","last_undegraded":"2024-03-30T13:12:26.986595+0000","last_fullsized":"2024-03-30T13:12:26.986595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992428+0000","last_change":"2024-03-30T13:12:26.987766+0000","last_active":"2024-03-30T13:12:26.992428+0000","last_peered":"2024-03-30T13:12:26.992428+0000","last_clean":"2024-03-30T13:12:26.992428+0000","last_became_active":"2024-03-30T13:12:26.987470+0000","last_became_peered":"2024-03-30T13:12:26.987470+0000","last_unstale":"2024-03-30T13:12:26.992428+0000","last_undegraded":"2024-03-30T13:12:26.992428+0000","last_fullsized":"2024-03-30T13:12:26.992428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:59:24.699504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.982849+0000","last_change":"2024-03-30T13:12:26.982849+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.982849+0000","last_undegraded":"2024-03-30T13:12:26.982849+0000","last_fullsized":"2024-03-30T13:12:26.982849+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.009935+0000","last_change":"2024-03-30T13:12:27.008741+0000","last_active":"2024-03-30T13:12:27.009935+0000","last_peered":"2024-03-30T13:12:27.009935+0000","last_clean":"2024-03-30T13:12:27.009935+0000","last_became_active":"2024-03-30T13:12:27.008147+0000","last_became_peered":"2024-03-30T13:12:27.008147+0000","last_unstale":"2024-03-30T13:12:27.009935+0000","last_undegraded":"2024-03-30T13:12:27.009935+0000","last_fullsized":"2024-03-30T13:12:27.009935+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:30:49.564675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.983151+0000","last_change":"2024-03-30T13:12:26.983151+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.983151+0000","last_undegraded":"2024-03-30T13:12:26.983151+0000","last_fullsized":"2024-03-30T13:12:26.983151+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012018+0000","last_change":"2024-03-30T13:12:27.005991+0000","last_active":"2024-03-30T13:12:27.012018+0000","last_peered":"2024-03-30T13:12:27.012018+0000","last_clean":"2024-03-30T13:12:27.012018+0000","last_became_active":"2024-03-30T13:12:27.003764+0000","last_became_peered":"2024-03-30T13:12:27.003764+0000","last_unstale":"2024-03-30T13:12:27.012018+0000","last_undegraded":"2024-03-30T13:12:27.012018+0000","last_fullsized":"2024-03-30T13:12:27.012018+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:50:21.528288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028802+0000","last_change":"2024-03-30T13:12:28.040607+0000","last_active":"2024-03-30T13:12:29.028802+0000","last_peered":"2024-03-30T13:12:29.028802+0000","last_clean":"2024-03-30T13:12:29.028802+0000","last_became_active":"2024-03-30T13:12:28.040303+0000","last_became_peered":"2024-03-30T13:12:28.040303+0000","last_unstale":"2024-03-30T13:12:29.028802+0000","last_undegraded":"2024-03-30T13:12:29.028802+0000","last_fullsized":"2024-03-30T13:12:29.028802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:53:52.056944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028248+0000","last_change":"2024-03-30T13:12:26.991406+0000","last_active":"2024-03-30T13:12:29.028248+0000","last_peered":"2024-03-30T13:12:29.028248+0000","last_clean":"2024-03-30T13:12:29.028248+0000","last_became_active":"2024-03-30T13:12:26.990643+0000","last_became_peered":"2024-03-30T13:12:26.990643+0000","last_unstale":"2024-03-30T13:12:29.028248+0000","last_undegraded":"2024-03-30T13:12:29.028248+0000","last_fullsized":"2024-03-30T13:12:29.028248+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:58:57.574756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978686+0000","last_change":"2024-03-30T13:12:26.978686+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978686+0000","last_undegraded":"2024-03-30T13:12:26.978686+0000","last_fullsized":"2024-03-30T13:12:26.978686+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028428+0000","last_change":"2024-03-30T13:12:27.004641+0000","last_active":"2024-03-30T13:12:29.028428+0000","last_peered":"2024-03-30T13:12:29.028428+0000","last_clean":"2024-03-30T13:12:29.028428+0000","last_became_active":"2024-03-30T13:12:27.002667+0000","last_became_peered":"2024-03-30T13:12:27.002667+0000","last_unstale":"2024-03-30T13:12:29.028428+0000","last_undegraded":"2024-03-30T13:12:29.028428+0000","last_fullsized":"2024-03-30T13:12:29.028428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:07:39.525496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984248+0000","last_change":"2024-03-30T13:12:26.984248+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984248+0000","last_undegraded":"2024-03-30T13:12:26.984248+0000","last_fullsized":"2024-03-30T13:12:26.984248+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025614+0000","last_change":"2024-03-30T13:12:27.021918+0000","last_active":"2024-03-30T13:12:27.025614+0000","last_peered":"2024-03-30T13:12:27.025614+0000","last_clean":"2024-03-30T13:12:27.025614+0000","last_became_active":"2024-03-30T13:12:27.019605+0000","last_became_peered":"2024-03-30T13:12:27.019605+0000","last_unstale":"2024-03-30T13:12:27.025614+0000","last_undegraded":"2024-03-30T13:12:27.025614+0000","last_fullsized":"2024-03-30T13:12:27.025614+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:32:56.857217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984409+0000","last_change":"2024-03-30T13:12:26.984409+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984409+0000","last_undegraded":"2024-03-30T13:12:26.984409+0000","last_fullsized":"2024-03-30T13:12:26.984409+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.992790+0000","last_change":"2024-03-30T13:12:26.989095+0000","last_active":"2024-03-30T13:12:26.992790+0000","last_peered":"2024-03-30T13:12:26.992790+0000","last_clean":"2024-03-30T13:12:26.992790+0000","last_became_active":"2024-03-30T13:12:26.988760+0000","last_became_peered":"2024-03-30T13:12:26.988760+0000","last_unstale":"2024-03-30T13:12:26.992790+0000","last_undegraded":"2024-03-30T13:12:26.992790+0000","last_fullsized":"2024-03-30T13:12:26.992790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:25:15.723577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.984702+0000","last_change":"2024-03-30T13:12:26.984702+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.984702+0000","last_undegraded":"2024-03-30T13:12:26.984702+0000","last_fullsized":"2024-03-30T13:12:26.984702+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.026212+0000","last_change":"2024-03-30T13:12:27.024227+0000","last_active":"2024-03-30T13:12:27.026212+0000","last_peered":"2024-03-30T13:12:27.026212+0000","last_clean":"2024-03-30T13:12:27.026212+0000","last_became_active":"2024-03-30T13:12:27.023036+0000","last_became_peered":"2024-03-30T13:12:27.023036+0000","last_unstale":"2024-03-30T13:12:27.026212+0000","last_undegraded":"2024-03-30T13:12:27.026212+0000","last_fullsized":"2024-03-30T13:12:27.026212+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:15.489606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.989414+0000","last_change":"2024-03-30T13:12:26.989414+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.989414+0000","last_undegraded":"2024-03-30T13:12:26.989414+0000","last_fullsized":"2024-03-30T13:12:26.989414+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.000874+0000","last_change":"2024-03-30T13:12:26.999648+0000","last_active":"2024-03-30T13:12:27.000874+0000","last_peered":"2024-03-30T13:12:27.000874+0000","last_clean":"2024-03-30T13:12:27.000874+0000","last_became_active":"2024-03-30T13:12:26.999374+0000","last_became_peered":"2024-03-30T13:12:26.999374+0000","last_unstale":"2024-03-30T13:12:27.000874+0000","last_undegraded":"2024-03-30T13:12:27.000874+0000","last_fullsized":"2024-03-30T13:12:27.000874+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:58:38.997896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981991+0000","last_change":"2024-03-30T13:12:26.981991+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981991+0000","last_undegraded":"2024-03-30T13:12:26.981991+0000","last_fullsized":"2024-03-30T13:12:26.981991+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.012967+0000","last_change":"2024-03-30T13:12:27.007320+0000","last_active":"2024-03-30T13:12:27.012967+0000","last_peered":"2024-03-30T13:12:27.012967+0000","last_clean":"2024-03-30T13:12:27.012967+0000","last_became_active":"2024-03-30T13:12:27.006579+0000","last_became_peered":"2024-03-30T13:12:27.006579+0000","last_unstale":"2024-03-30T13:12:27.012967+0000","last_undegraded":"2024-03-30T13:12:27.012967+0000","last_fullsized":"2024-03-30T13:12:27.012967+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:42.922450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028489+0000","last_change":"2024-03-30T13:12:28.036105+0000","last_active":"2024-03-30T13:12:29.028489+0000","last_peered":"2024-03-30T13:12:29.028489+0000","last_clean":"2024-03-30T13:12:29.028489+0000","last_became_active":"2024-03-30T13:12:28.035011+0000","last_became_peered":"2024-03-30T13:12:28.035011+0000","last_unstale":"2024-03-30T13:12:29.028489+0000","last_undegraded":"2024-03-30T13:12:29.028489+0000","last_fullsized":"2024-03-30T13:12:29.028489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:23:31.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.019402+0000","last_change":"2024-03-30T13:12:27.017279+0000","last_active":"2024-03-30T13:12:27.019402+0000","last_peered":"2024-03-30T13:12:27.019402+0000","last_clean":"2024-03-30T13:12:27.019402+0000","last_became_active":"2024-03-30T13:12:27.016328+0000","last_became_peered":"2024-03-30T13:12:27.016328+0000","last_unstale":"2024-03-30T13:12:27.019402+0000","last_undegraded":"2024-03-30T13:12:27.019402+0000","last_fullsized":"2024-03-30T13:12:27.019402+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:20.310096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.986444+0000","last_change":"2024-03-30T13:12:26.986444+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.986444+0000","last_undegraded":"2024-03-30T13:12:26.986444+0000","last_fullsized":"2024-03-30T13:12:26.986444+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.025780+0000","last_change":"2024-03-30T13:12:27.022469+0000","last_active":"2024-03-30T13:12:27.025780+0000","last_peered":"2024-03-30T13:12:27.025780+0000","last_clean":"2024-03-30T13:12:27.025780+0000","last_became_active":"2024-03-30T13:12:27.021653+0000","last_became_peered":"2024-03-30T13:12:27.021653+0000","last_unstale":"2024-03-30T13:12:27.025780+0000","last_undegraded":"2024-03-30T13:12:27.025780+0000","last_fullsized":"2024-03-30T13:12:27.025780+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.990485+0000","last_change":"2024-03-30T13:12:26.990485+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.990485+0000","last_undegraded":"2024-03-30T13:12:26.990485+0000","last_fullsized":"2024-03-30T13:12:26.990485+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.006219+0000","last_change":"2024-03-30T13:12:27.005433+0000","last_active":"2024-03-30T13:12:27.006219+0000","last_peered":"2024-03-30T13:12:27.006219+0000","last_clean":"2024-03-30T13:12:27.006219+0000","last_became_active":"2024-03-30T13:12:27.005116+0000","last_became_peered":"2024-03-30T13:12:27.005116+0000","last_unstale":"2024-03-30T13:12:27.006219+0000","last_undegraded":"2024-03-30T13:12:27.006219+0000","last_fullsized":"2024-03-30T13:12:27.006219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.978019+0000","last_change":"2024-03-30T13:12:26.978019+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.978019+0000","last_undegraded":"2024-03-30T13:12:26.978019+0000","last_fullsized":"2024-03-30T13:12:26.978019+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.999282+0000","last_change":"2024-03-30T13:12:26.998465+0000","last_active":"2024-03-30T13:12:26.999282+0000","last_peered":"2024-03-30T13:12:26.999282+0000","last_clean":"2024-03-30T13:12:26.999282+0000","last_became_active":"2024-03-30T13:12:26.998168+0000","last_became_peered":"2024-03-30T13:12:26.998168+0000","last_unstale":"2024-03-30T13:12:26.999282+0000","last_undegraded":"2024-03-30T13:12:26.999282+0000","last_fullsized":"2024-03-30T13:12:26.999282+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:52:53.995529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.981851+0000","last_change":"2024-03-30T13:12:26.981851+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.981851+0000","last_undegraded":"2024-03-30T13:12:26.981851+0000","last_fullsized":"2024-03-30T13:12:26.981851+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.022219+0000","last_change":"2024-03-30T13:12:27.009963+0000","last_active":"2024-03-30T13:12:27.022219+0000","last_peered":"2024-03-30T13:12:27.022219+0000","last_clean":"2024-03-30T13:12:27.022219+0000","last_became_active":"2024-03-30T13:12:27.007546+0000","last_became_peered":"2024-03-30T13:12:27.007546+0000","last_unstale":"2024-03-30T13:12:27.022219+0000","last_undegraded":"2024-03-30T13:12:27.022219+0000","last_fullsized":"2024-03-30T13:12:27.022219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:51.709095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-03-30T13:12:26.980231+0000","last_change":"2024-03-30T13:12:26.980231+0000","last_active":"2024-03-30T13:12:26.967503+0000","last_peered":"2024-03-30T13:12:26.967503+0000","last_clean":"2024-03-30T13:12:26.967503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-30T13:12:26.980231+0000","last_undegraded":"2024-03-30T13:12:26.980231+0000","last_fullsized":"2024-03-30T13:12:26.980231+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:27.002506+0000","last_change":"2024-03-30T13:12:27.002016+0000","last_active":"2024-03-30T13:12:27.002506+0000","last_peered":"2024-03-30T13:12:27.002506+0000","last_clean":"2024-03-30T13:12:27.002506+0000","last_became_active":"2024-03-30T13:12:27.001703+0000","last_became_peered":"2024-03-30T13:12:27.001703+0000","last_unstale":"2024-03-30T13:12:27.002506+0000","last_undegraded":"2024-03-30T13:12:27.002506+0000","last_fullsized":"2024-03-30T13:12:27.002506+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":200,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-03-30T13:12:26.976182+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:12:26.976182+0000","last_peered":"2024-03-30T13:12:26.976182+0000","last_clean":"2024-03-30T13:12:26.976182+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:12:26.976182+0000","last_undegraded":"2024-03-30T13:12:26.976182+0000","last_fullsized":"2024-03-30T13:12:26.976182+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":185594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":775962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31600,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340240,"statfs":{"total":96636764160,"available":96604405760,"internally_reserved":0,"allocated":430080,"data_stored":178927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":790403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:12:36.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:12:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c95c3640 1 Processor -- start 2024-03-30T13:12:36.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c95c3640 1 -- start start 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c95c3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f49c40b2ef0 0x7f49c4144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c95c3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49c415f7a0 0x7f49c406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c95c3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49c40b4ba0 con 0x7f49c4144270 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c95c3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f49c4144fa0 con 0x7f49c415f7a0 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c95c3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f49c40b33f0 con 0x7f49c40b2ef0 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49c415f7a0 0x7f49c406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f49c4144270 0x7f49c40b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34522/0 (socket says 172.21.15.43:34522) 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c2ffd640 1 -- 172.21.15.43:0/4245106335 learned_addr learned my addr 172.21.15.43:0/4245106335 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49ba7fc640 1 --2- 172.21.15.43:0/4245106335 >> v2:172.21.15.113:3300/0 conn(0x7f49c40b2ef0 0x7f49c4144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:36.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c27fc640 1 -- 172.21.15.43:0/4245106335 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3430893660 0 0) 0x7f49c40b4ba0 con 0x7f49c4144270 2024-03-30T13:12:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c27fc640 1 -- 172.21.15.43:0/4245106335 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49b0005740 con 0x7f49c4144270 2024-03-30T13:12:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c37fe640 1 -- 172.21.15.43:0/4245106335 >> v2:172.21.15.113:3300/0 conn(0x7f49c40b2ef0 msgr2=0x7f49c4144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c37fe640 1 --2- 172.21.15.43:0/4245106335 >> v2:172.21.15.113:3300/0 conn(0x7f49c40b2ef0 0x7f49c4144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c37fe640 1 -- 172.21.15.43:0/4245106335 >> v1:172.21.15.43:6789/0 conn(0x7f49c4144270 legacy=0x7f49c40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49c37fe640 1 -- 172.21.15.43:0/4245106335 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c40b36d0 con 0x7f49c415f7a0 2024-03-30T13:12:36.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.727+0000 7f49ba7fc640 1 --2- 172.21.15.43:0/4245106335 >> v2:172.21.15.113:3300/0 conn(0x7f49c40b2ef0 0x7f49c4144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c37fe640 1 --2- 172.21.15.43:0/4245106335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c406c670 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f49c40b4e70 tx=0x7f49bc01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526e59be4bd3862 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c27fc640 1 -- 172.21.15.43:0/4245106335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0b2070 con 0x7f49c415f7a0 2024-03-30T13:12:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c27fc640 1 -- 172.21.15.43:0/4245106335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49bc0a58d0 con 0x7f49c415f7a0 2024-03-30T13:12:36.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c27fc640 1 -- 172.21.15.43:0/4245106335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0a5ca0 con 0x7f49c415f7a0 2024-03-30T13:12:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 -- 172.21.15.43:0/4245106335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 msgr2=0x7f49c406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 --2- 172.21.15.43:0/4245106335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c406c670 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f49c40b4e70 tx=0x7f49bc01c900 comp rx=0 tx=0).stop 2024-03-30T13:12:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 -- 172.21.15.43:0/4245106335 shutdown_connections 2024-03-30T13:12:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 --2- 172.21.15.43:0/4245106335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c406c670 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 --2- 172.21.15.43:0/4245106335 >> v2:172.21.15.113:3300/0 conn(0x7f49c40b2ef0 0x7f49c4144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 -- 172.21.15.43:0/4245106335 >> 172.21.15.43:0/4245106335 conn(0x7f49c40cfee0 msgr2=0x7f49c40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 -- 172.21.15.43:0/4245106335 shutdown_connections 2024-03-30T13:12:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 -- 172.21.15.43:0/4245106335 wait complete. 2024-03-30T13:12:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 Processor -- start 2024-03-30T13:12:36.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.731+0000 7f49c95c3640 1 -- start start 2024-03-30T13:12:36.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c40b2ef0 0x7f49c4133380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:36.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c4144270 0x7f49c415e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:36.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:36.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f49c40cdd10 con 0x7f49c415f7a0 2024-03-30T13:12:36.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f49c415fc30 con 0x7f49c40b2ef0 2024-03-30T13:12:36.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f49c40cde90 con 0x7f49c4144270 2024-03-30T13:12:36.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c415a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34938/0 (socket says 172.21.15.43:34938) 2024-03-30T13:12:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 -- 172.21.15.43:0/3194982400 learned_addr learned my addr 172.21.15.43:0/3194982400 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c2ffd640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c40b2ef0 0x7f49c4133380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49ba7fc640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c4144270 0x7f49c415e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:36.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 -- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c4144270 msgr2=0x7f49c415e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c4144270 0x7f49c415e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 -- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c40b2ef0 msgr2=0x7f49c4133380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c40b2ef0 0x7f49c4133380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 -- 172.21.15.43:0/3194982400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c41339f0 con 0x7f49c415f7a0 2024-03-30T13:12:36.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c37fe640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c415a0a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b0040 tx=0x7f49bc0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49bbfff640 1 -- 172.21.15.43:0/3194982400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0b2050 con 0x7f49c415f7a0 2024-03-30T13:12:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49bbfff640 1 -- 172.21.15.43:0/3194982400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49bc0be070 con 0x7f49c415f7a0 2024-03-30T13:12:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49bbfff640 1 -- 172.21.15.43:0/3194982400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0ada70 con 0x7f49c415f7a0 2024-03-30T13:12:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c4133cd0 con 0x7f49c415f7a0 2024-03-30T13:12:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c4134290 con 0x7f49c415f7a0 2024-03-30T13:12:36.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.735+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c40722d0 con 0x7f49c415f7a0 2024-03-30T13:12:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.739+0000 7f49bbfff640 1 -- 172.21.15.43:0/3194982400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f49bc0a5750 con 0x7f49c415f7a0 2024-03-30T13:12:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.739+0000 7f49bbfff640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49a4067fc0 0x7f49a406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.739+0000 7f49bbfff640 1 -- 172.21.15.43:0/3194982400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f49bc121910 con 0x7f49c415f7a0 2024-03-30T13:12:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.739+0000 7f49c2ffd640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49a4067fc0 0x7f49a406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.739+0000 7f49c2ffd640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49a4067fc0 0x7f49a406a480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005890 tx=0x7f49b4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.743+0000 7f49bbfff640 1 -- 172.21.15.43:0/3194982400 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f49bc016520 con 0x7f49c415f7a0 2024-03-30T13:12:36.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.903+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49c4068870 con 0x7f49a4067fc0 2024-03-30T13:12:36.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49bbfff640 1 -- 172.21.15.43:0/3194982400 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251949 (secure 0 0 0) 0x7f49c4068870 con 0x7f49a4067fc0 2024-03-30T13:12:36.930 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:36.931 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:12:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49a4067fc0 msgr2=0x7f49a406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49a4067fc0 0x7f49a406a480 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f49b4005890 tx=0x7f49b4009040 comp rx=0 tx=0).stop 2024-03-30T13:12:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 msgr2=0x7f49c415a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c415a0a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b0040 tx=0x7f49bc0196a0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 shutdown_connections 2024-03-30T13:12:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f49a4067fc0 0x7f49a406a480 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49c415f7a0 0x7f49c415a0a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49c4144270 0x7f49c415e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 --2- 172.21.15.43:0/3194982400 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49c40b2ef0 0x7f49c4133380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 >> 172.21.15.43:0/3194982400 conn(0x7f49c40cfee0 msgr2=0x7f49c406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 shutdown_connections 2024-03-30T13:12:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:36.923+0000 7f49c95c3640 1 -- 172.21.15.43:0/3194982400 wait complete. 2024-03-30T13:12:36.949 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2024-03-30T13:12:35.873152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":246024,"kb_used_data":4872,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754728696,"statfs":{"total":773094113280,"available":772842184704,"internally_reserved":0,"allocated":4988928,"data_stored":2781180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":246795826},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002740"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473992+0000","last_change":"2024-03-30T13:12:28.032532+0000","last_active":"2024-03-30T13:12:30.473992+0000","last_peered":"2024-03-30T13:12:30.473992+0000","last_clean":"2024-03-30T13:12:30.473992+0000","last_became_active":"2024-03-30T13:12:28.031495+0000","last_became_peered":"2024-03-30T13:12:28.031495+0000","last_unstale":"2024-03-30T13:12:30.473992+0000","last_undegraded":"2024-03-30T13:12:30.473992+0000","last_fullsized":"2024-03-30T13:12:30.473992+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:09:57.186980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564355+0000","last_change":"2024-03-30T13:12:28.028692+0000","last_active":"2024-03-30T13:12:30.564355+0000","last_peered":"2024-03-30T13:12:30.564355+0000","last_clean":"2024-03-30T13:12:30.564355+0000","last_became_active":"2024-03-30T13:12:28.028380+0000","last_became_peered":"2024-03-30T13:12:28.028380+0000","last_unstale":"2024-03-30T13:12:30.564355+0000","last_undegraded":"2024-03-30T13:12:30.564355+0000","last_fullsized":"2024-03-30T13:12:30.564355+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:42:06.499046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473469+0000","last_change":"2024-03-30T13:12:28.041389+0000","last_active":"2024-03-30T13:12:30.473469+0000","last_peered":"2024-03-30T13:12:30.473469+0000","last_clean":"2024-03-30T13:12:30.473469+0000","last_became_active":"2024-03-30T13:12:28.040848+0000","last_became_peered":"2024-03-30T13:12:28.040848+0000","last_unstale":"2024-03-30T13:12:30.473469+0000","last_undegraded":"2024-03-30T13:12:30.473469+0000","last_fullsized":"2024-03-30T13:12:30.473469+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:55:08.106457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473244+0000","last_change":"2024-03-30T13:12:28.040276+0000","last_active":"2024-03-30T13:12:30.473244+0000","last_peered":"2024-03-30T13:12:30.473244+0000","last_clean":"2024-03-30T13:12:30.473244+0000","last_became_active":"2024-03-30T13:12:28.040008+0000","last_became_peered":"2024-03-30T13:12:28.040008+0000","last_unstale":"2024-03-30T13:12:30.473244+0000","last_undegraded":"2024-03-30T13:12:30.473244+0000","last_fullsized":"2024-03-30T13:12:30.473244+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:31:49.551089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026465+0000","last_change":"2024-03-30T13:12:28.028715+0000","last_active":"2024-03-30T13:12:29.026465+0000","last_peered":"2024-03-30T13:12:29.026465+0000","last_clean":"2024-03-30T13:12:29.026465+0000","last_became_active":"2024-03-30T13:12:28.028477+0000","last_became_peered":"2024-03-30T13:12:28.028477+0000","last_unstale":"2024-03-30T13:12:29.026465+0000","last_undegraded":"2024-03-30T13:12:29.026465+0000","last_fullsized":"2024-03-30T13:12:29.026465+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:58:36.961839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563570+0000","last_change":"2024-03-30T13:12:28.029156+0000","last_active":"2024-03-30T13:12:30.563570+0000","last_peered":"2024-03-30T13:12:30.563570+0000","last_clean":"2024-03-30T13:12:30.563570+0000","last_became_active":"2024-03-30T13:12:28.028336+0000","last_became_peered":"2024-03-30T13:12:28.028336+0000","last_unstale":"2024-03-30T13:12:30.563570+0000","last_undegraded":"2024-03-30T13:12:30.563570+0000","last_fullsized":"2024-03-30T13:12:30.563570+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:13:03.689090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024263+0000","last_change":"2024-03-30T13:12:28.016044+0000","last_active":"2024-03-30T13:12:29.024263+0000","last_peered":"2024-03-30T13:12:29.024263+0000","last_clean":"2024-03-30T13:12:29.024263+0000","last_became_active":"2024-03-30T13:12:28.015266+0000","last_became_peered":"2024-03-30T13:12:28.015266+0000","last_unstale":"2024-03-30T13:12:29.024263+0000","last_undegraded":"2024-03-30T13:12:29.024263+0000","last_fullsized":"2024-03-30T13:12:29.024263+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:52.717456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473045+0000","last_change":"2024-03-30T13:12:28.048543+0000","last_active":"2024-03-30T13:12:30.473045+0000","last_peered":"2024-03-30T13:12:30.473045+0000","last_clean":"2024-03-30T13:12:30.473045+0000","last_became_active":"2024-03-30T13:12:28.047762+0000","last_became_peered":"2024-03-30T13:12:28.047762+0000","last_unstale":"2024-03-30T13:12:30.473045+0000","last_undegraded":"2024-03-30T13:12:30.473045+0000","last_fullsized":"2024-03-30T13:12:30.473045+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:34:23.801156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026023+0000","last_change":"2024-03-30T13:12:28.034772+0000","last_active":"2024-03-30T13:12:29.026023+0000","last_peered":"2024-03-30T13:12:29.026023+0000","last_clean":"2024-03-30T13:12:29.026023+0000","last_became_active":"2024-03-30T13:12:28.034493+0000","last_became_peered":"2024-03-30T13:12:28.034493+0000","last_unstale":"2024-03-30T13:12:29.026023+0000","last_undegraded":"2024-03-30T13:12:29.026023+0000","last_fullsized":"2024-03-30T13:12:29.026023+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:22.648408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.025496+0000","last_change":"2024-03-30T13:12:28.019994+0000","last_active":"2024-03-30T13:12:29.025496+0000","last_peered":"2024-03-30T13:12:29.025496+0000","last_clean":"2024-03-30T13:12:29.025496+0000","last_became_active":"2024-03-30T13:12:28.019682+0000","last_became_peered":"2024-03-30T13:12:28.019682+0000","last_unstale":"2024-03-30T13:12:29.025496+0000","last_undegraded":"2024-03-30T13:12:29.025496+0000","last_fullsized":"2024-03-30T13:12:29.025496+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:25.231395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.035189+0000","last_change":"2024-03-30T13:12:28.036623+0000","last_active":"2024-03-30T13:12:29.035189+0000","last_peered":"2024-03-30T13:12:29.035189+0000","last_clean":"2024-03-30T13:12:29.035189+0000","last_became_active":"2024-03-30T13:12:28.035237+0000","last_became_peered":"2024-03-30T13:12:28.035237+0000","last_unstale":"2024-03-30T13:12:29.035189+0000","last_undegraded":"2024-03-30T13:12:29.035189+0000","last_fullsized":"2024-03-30T13:12:29.035189+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:42:24.354249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.025852+0000","last_change":"2024-03-30T13:12:28.040084+0000","last_active":"2024-03-30T13:12:29.025852+0000","last_peered":"2024-03-30T13:12:29.025852+0000","last_clean":"2024-03-30T13:12:29.025852+0000","last_became_active":"2024-03-30T13:12:28.039744+0000","last_became_peered":"2024-03-30T13:12:28.039744+0000","last_unstale":"2024-03-30T13:12:29.025852+0000","last_undegraded":"2024-03-30T13:12:29.025852+0000","last_fullsized":"2024-03-30T13:12:29.025852+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:52.717456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024357+0000","last_change":"2024-03-30T13:12:28.027909+0000","last_active":"2024-03-30T13:12:29.024357+0000","last_peered":"2024-03-30T13:12:29.024357+0000","last_clean":"2024-03-30T13:12:29.024357+0000","last_became_active":"2024-03-30T13:12:28.027022+0000","last_became_peered":"2024-03-30T13:12:28.027022+0000","last_unstale":"2024-03-30T13:12:29.024357+0000","last_undegraded":"2024-03-30T13:12:29.024357+0000","last_fullsized":"2024-03-30T13:12:29.024357+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:22.648408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563738+0000","last_change":"2024-03-30T13:12:28.038195+0000","last_active":"2024-03-30T13:12:30.563738+0000","last_peered":"2024-03-30T13:12:30.563738+0000","last_clean":"2024-03-30T13:12:30.563738+0000","last_became_active":"2024-03-30T13:12:28.036826+0000","last_became_peered":"2024-03-30T13:12:28.036826+0000","last_unstale":"2024-03-30T13:12:30.563738+0000","last_undegraded":"2024-03-30T13:12:30.563738+0000","last_fullsized":"2024-03-30T13:12:30.563738+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:22:19.832916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563469+0000","last_change":"2024-03-30T13:12:28.030785+0000","last_active":"2024-03-30T13:12:30.563469+0000","last_peered":"2024-03-30T13:12:30.563469+0000","last_clean":"2024-03-30T13:12:30.563469+0000","last_became_active":"2024-03-30T13:12:28.030156+0000","last_became_peered":"2024-03-30T13:12:28.030156+0000","last_unstale":"2024-03-30T13:12:30.563469+0000","last_undegraded":"2024-03-30T13:12:30.563469+0000","last_fullsized":"2024-03-30T13:12:30.563469+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:02:48.439021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024020+0000","last_change":"2024-03-30T13:12:28.044630+0000","last_active":"2024-03-30T13:12:29.024020+0000","last_peered":"2024-03-30T13:12:29.024020+0000","last_clean":"2024-03-30T13:12:29.024020+0000","last_became_active":"2024-03-30T13:12:28.044316+0000","last_became_peered":"2024-03-30T13:12:28.044316+0000","last_unstale":"2024-03-30T13:12:29.024020+0000","last_undegraded":"2024-03-30T13:12:29.024020+0000","last_fullsized":"2024-03-30T13:12:29.024020+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:13.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565035+0000","last_change":"2024-03-30T13:12:28.022114+0000","last_active":"2024-03-30T13:12:30.565035+0000","last_peered":"2024-03-30T13:12:30.565035+0000","last_clean":"2024-03-30T13:12:30.565035+0000","last_became_active":"2024-03-30T13:12:28.021804+0000","last_became_peered":"2024-03-30T13:12:28.021804+0000","last_unstale":"2024-03-30T13:12:30.565035+0000","last_undegraded":"2024-03-30T13:12:30.565035+0000","last_fullsized":"2024-03-30T13:12:30.565035+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:41:29.248034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565436+0000","last_change":"2024-03-30T13:12:28.028847+0000","last_active":"2024-03-30T13:12:30.565436+0000","last_peered":"2024-03-30T13:12:30.565436+0000","last_clean":"2024-03-30T13:12:30.565436+0000","last_became_active":"2024-03-30T13:12:28.028472+0000","last_became_peered":"2024-03-30T13:12:28.028472+0000","last_unstale":"2024-03-30T13:12:30.565436+0000","last_undegraded":"2024-03-30T13:12:30.565436+0000","last_fullsized":"2024-03-30T13:12:30.565436+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:21:57.284421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028469+0000","last_change":"2024-03-30T13:12:28.035833+0000","last_active":"2024-03-30T13:12:29.028469+0000","last_peered":"2024-03-30T13:12:29.028469+0000","last_clean":"2024-03-30T13:12:29.028469+0000","last_became_active":"2024-03-30T13:12:28.035506+0000","last_became_peered":"2024-03-30T13:12:28.035506+0000","last_unstale":"2024-03-30T13:12:29.028469+0000","last_undegraded":"2024-03-30T13:12:29.028469+0000","last_fullsized":"2024-03-30T13:12:29.028469+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:18:21.093109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034660+0000","last_change":"2024-03-30T13:12:28.036380+0000","last_active":"2024-03-30T13:12:29.034660+0000","last_peered":"2024-03-30T13:12:29.034660+0000","last_clean":"2024-03-30T13:12:29.034660+0000","last_became_active":"2024-03-30T13:12:28.035664+0000","last_became_peered":"2024-03-30T13:12:28.035664+0000","last_unstale":"2024-03-30T13:12:29.034660+0000","last_undegraded":"2024-03-30T13:12:29.034660+0000","last_fullsized":"2024-03-30T13:12:29.034660+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:39:45.939887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034274+0000","last_change":"2024-03-30T13:12:28.038716+0000","last_active":"2024-03-30T13:12:29.034274+0000","last_peered":"2024-03-30T13:12:29.034274+0000","last_clean":"2024-03-30T13:12:29.034274+0000","last_became_active":"2024-03-30T13:12:28.038062+0000","last_became_peered":"2024-03-30T13:12:28.038062+0000","last_unstale":"2024-03-30T13:12:29.034274+0000","last_undegraded":"2024-03-30T13:12:29.034274+0000","last_fullsized":"2024-03-30T13:12:29.034274+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:30:50.573036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034264+0000","last_change":"2024-03-30T13:12:28.048592+0000","last_active":"2024-03-30T13:12:29.034264+0000","last_peered":"2024-03-30T13:12:29.034264+0000","last_clean":"2024-03-30T13:12:29.034264+0000","last_became_active":"2024-03-30T13:12:28.041261+0000","last_became_peered":"2024-03-30T13:12:28.041261+0000","last_unstale":"2024-03-30T13:12:29.034264+0000","last_undegraded":"2024-03-30T13:12:29.034264+0000","last_fullsized":"2024-03-30T13:12:29.034264+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:56.969545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564458+0000","last_change":"2024-03-30T13:12:28.037599+0000","last_active":"2024-03-30T13:12:30.564458+0000","last_peered":"2024-03-30T13:12:30.564458+0000","last_clean":"2024-03-30T13:12:30.564458+0000","last_became_active":"2024-03-30T13:12:28.036994+0000","last_became_peered":"2024-03-30T13:12:28.036994+0000","last_unstale":"2024-03-30T13:12:30.564458+0000","last_undegraded":"2024-03-30T13:12:30.564458+0000","last_fullsized":"2024-03-30T13:12:30.564458+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:12:33.770997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473140+0000","last_change":"2024-03-30T13:12:28.037698+0000","last_active":"2024-03-30T13:12:30.473140+0000","last_peered":"2024-03-30T13:12:30.473140+0000","last_clean":"2024-03-30T13:12:30.473140+0000","last_became_active":"2024-03-30T13:12:28.037108+0000","last_became_peered":"2024-03-30T13:12:28.037108+0000","last_unstale":"2024-03-30T13:12:30.473140+0000","last_undegraded":"2024-03-30T13:12:30.473140+0000","last_fullsized":"2024-03-30T13:12:30.473140+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:25:19.492965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564319+0000","last_change":"2024-03-30T13:12:28.026226+0000","last_active":"2024-03-30T13:12:30.564319+0000","last_peered":"2024-03-30T13:12:30.564319+0000","last_clean":"2024-03-30T13:12:30.564319+0000","last_became_active":"2024-03-30T13:12:28.025765+0000","last_became_peered":"2024-03-30T13:12:28.025765+0000","last_unstale":"2024-03-30T13:12:30.564319+0000","last_undegraded":"2024-03-30T13:12:30.564319+0000","last_fullsized":"2024-03-30T13:12:30.564319+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:22:37.188424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.474277+0000","last_change":"2024-03-30T13:12:28.037067+0000","last_active":"2024-03-30T13:12:30.474277+0000","last_peered":"2024-03-30T13:12:30.474277+0000","last_clean":"2024-03-30T13:12:30.474277+0000","last_became_active":"2024-03-30T13:12:28.036732+0000","last_became_peered":"2024-03-30T13:12:28.036732+0000","last_unstale":"2024-03-30T13:12:30.474277+0000","last_undegraded":"2024-03-30T13:12:30.474277+0000","last_fullsized":"2024-03-30T13:12:30.474277+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:12:33.770997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.035301+0000","last_change":"2024-03-30T13:12:28.030267+0000","last_active":"2024-03-30T13:12:29.035301+0000","last_peered":"2024-03-30T13:12:29.035301+0000","last_clean":"2024-03-30T13:12:29.035301+0000","last_became_active":"2024-03-30T13:12:28.029329+0000","last_became_peered":"2024-03-30T13:12:28.029329+0000","last_unstale":"2024-03-30T13:12:29.035301+0000","last_undegraded":"2024-03-30T13:12:29.035301+0000","last_fullsized":"2024-03-30T13:12:29.035301+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:52.518135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024987+0000","last_change":"2024-03-30T13:12:28.045987+0000","last_active":"2024-03-30T13:12:29.024987+0000","last_peered":"2024-03-30T13:12:29.024987+0000","last_clean":"2024-03-30T13:12:29.024987+0000","last_became_active":"2024-03-30T13:12:28.044851+0000","last_became_peered":"2024-03-30T13:12:28.044851+0000","last_unstale":"2024-03-30T13:12:29.024987+0000","last_undegraded":"2024-03-30T13:12:29.024987+0000","last_fullsized":"2024-03-30T13:12:29.024987+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:58:36.961839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565515+0000","last_change":"2024-03-30T13:12:28.035786+0000","last_active":"2024-03-30T13:12:30.565515+0000","last_peered":"2024-03-30T13:12:30.565515+0000","last_clean":"2024-03-30T13:12:30.565515+0000","last_became_active":"2024-03-30T13:12:28.035149+0000","last_became_peered":"2024-03-30T13:12:28.035149+0000","last_unstale":"2024-03-30T13:12:30.565515+0000","last_undegraded":"2024-03-30T13:12:30.565515+0000","last_fullsized":"2024-03-30T13:12:30.565515+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:47:15.520405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565129+0000","last_change":"2024-03-30T13:12:28.033059+0000","last_active":"2024-03-30T13:12:30.565129+0000","last_peered":"2024-03-30T13:12:30.565129+0000","last_clean":"2024-03-30T13:12:30.565129+0000","last_became_active":"2024-03-30T13:12:28.032464+0000","last_became_peered":"2024-03-30T13:12:28.032464+0000","last_unstale":"2024-03-30T13:12:30.565129+0000","last_undegraded":"2024-03-30T13:12:30.565129+0000","last_fullsized":"2024-03-30T13:12:30.565129+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:01:02.481671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024674+0000","last_change":"2024-03-30T13:12:28.032855+0000","last_active":"2024-03-30T13:12:29.024674+0000","last_peered":"2024-03-30T13:12:29.024674+0000","last_clean":"2024-03-30T13:12:29.024674+0000","last_became_active":"2024-03-30T13:12:28.032549+0000","last_became_peered":"2024-03-30T13:12:28.032549+0000","last_unstale":"2024-03-30T13:12:29.024674+0000","last_undegraded":"2024-03-30T13:12:29.024674+0000","last_fullsized":"2024-03-30T13:12:29.024674+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:15.580995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473514+0000","last_change":"2024-03-30T13:12:28.048398+0000","last_active":"2024-03-30T13:12:30.473514+0000","last_peered":"2024-03-30T13:12:30.473514+0000","last_clean":"2024-03-30T13:12:30.473514+0000","last_became_active":"2024-03-30T13:12:28.047655+0000","last_became_peered":"2024-03-30T13:12:28.047655+0000","last_unstale":"2024-03-30T13:12:30.473514+0000","last_undegraded":"2024-03-30T13:12:30.473514+0000","last_fullsized":"2024-03-30T13:12:30.473514+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:59:46.966079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565230+0000","last_change":"2024-03-30T13:12:28.023006+0000","last_active":"2024-03-30T13:12:30.565230+0000","last_peered":"2024-03-30T13:12:30.565230+0000","last_clean":"2024-03-30T13:12:30.565230+0000","last_became_active":"2024-03-30T13:12:28.021127+0000","last_became_peered":"2024-03-30T13:12:28.021127+0000","last_unstale":"2024-03-30T13:12:30.565230+0000","last_undegraded":"2024-03-30T13:12:30.565230+0000","last_fullsized":"2024-03-30T13:12:30.565230+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:52:18.859216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564258+0000","last_change":"2024-03-30T13:12:26.994626+0000","last_active":"2024-03-30T13:12:30.564258+0000","last_peered":"2024-03-30T13:12:30.564258+0000","last_clean":"2024-03-30T13:12:30.564258+0000","last_became_active":"2024-03-30T13:12:26.994312+0000","last_became_peered":"2024-03-30T13:12:26.994312+0000","last_unstale":"2024-03-30T13:12:30.564258+0000","last_undegraded":"2024-03-30T13:12:30.564258+0000","last_fullsized":"2024-03-30T13:12:30.564258+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:12.328507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563978+0000","last_change":"2024-03-30T13:12:28.025796+0000","last_active":"2024-03-30T13:12:30.563978+0000","last_peered":"2024-03-30T13:12:30.563978+0000","last_clean":"2024-03-30T13:12:30.563978+0000","last_became_active":"2024-03-30T13:12:28.025450+0000","last_became_peered":"2024-03-30T13:12:28.025450+0000","last_unstale":"2024-03-30T13:12:30.563978+0000","last_undegraded":"2024-03-30T13:12:30.563978+0000","last_fullsized":"2024-03-30T13:12:30.563978+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:30:50.573036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564555+0000","last_change":"2024-03-30T13:12:27.020872+0000","last_active":"2024-03-30T13:12:30.564555+0000","last_peered":"2024-03-30T13:12:30.564555+0000","last_clean":"2024-03-30T13:12:30.564555+0000","last_became_active":"2024-03-30T13:12:27.020255+0000","last_became_peered":"2024-03-30T13:12:27.020255+0000","last_unstale":"2024-03-30T13:12:30.564555+0000","last_undegraded":"2024-03-30T13:12:30.564555+0000","last_fullsized":"2024-03-30T13:12:30.564555+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:14.572634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565916+0000","last_change":"2024-03-30T13:12:28.021436+0000","last_active":"2024-03-30T13:12:30.565916+0000","last_peered":"2024-03-30T13:12:30.565916+0000","last_clean":"2024-03-30T13:12:30.565916+0000","last_became_active":"2024-03-30T13:12:28.021122+0000","last_became_peered":"2024-03-30T13:12:28.021122+0000","last_unstale":"2024-03-30T13:12:30.565916+0000","last_undegraded":"2024-03-30T13:12:30.565916+0000","last_fullsized":"2024-03-30T13:12:30.565916+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:03:45.522447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473654+0000","last_change":"2024-03-30T13:12:27.024163+0000","last_active":"2024-03-30T13:12:30.473654+0000","last_peered":"2024-03-30T13:12:30.473654+0000","last_clean":"2024-03-30T13:12:30.473654+0000","last_became_active":"2024-03-30T13:12:27.023530+0000","last_became_peered":"2024-03-30T13:12:27.023530+0000","last_unstale":"2024-03-30T13:12:30.473654+0000","last_undegraded":"2024-03-30T13:12:30.473654+0000","last_fullsized":"2024-03-30T13:12:30.473654+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:22:36.180063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563955+0000","last_change":"2024-03-30T13:12:28.029652+0000","last_active":"2024-03-30T13:12:30.563955+0000","last_peered":"2024-03-30T13:12:30.563955+0000","last_clean":"2024-03-30T13:12:30.563955+0000","last_became_active":"2024-03-30T13:12:28.027729+0000","last_became_peered":"2024-03-30T13:12:28.027729+0000","last_unstale":"2024-03-30T13:12:30.563955+0000","last_undegraded":"2024-03-30T13:12:30.563955+0000","last_fullsized":"2024-03-30T13:12:30.563955+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:15:12.148112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563888+0000","last_change":"2024-03-30T13:12:27.017703+0000","last_active":"2024-03-30T13:12:30.563888+0000","last_peered":"2024-03-30T13:12:30.563888+0000","last_clean":"2024-03-30T13:12:30.563888+0000","last_became_active":"2024-03-30T13:12:27.017417+0000","last_became_peered":"2024-03-30T13:12:27.017417+0000","last_unstale":"2024-03-30T13:12:30.563888+0000","last_undegraded":"2024-03-30T13:12:30.563888+0000","last_fullsized":"2024-03-30T13:12:30.563888+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:51.709095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473885+0000","last_change":"2024-03-30T13:12:28.037425+0000","last_active":"2024-03-30T13:12:30.473885+0000","last_peered":"2024-03-30T13:12:30.473885+0000","last_clean":"2024-03-30T13:12:30.473885+0000","last_became_active":"2024-03-30T13:12:28.036676+0000","last_became_peered":"2024-03-30T13:12:28.036676+0000","last_unstale":"2024-03-30T13:12:30.473885+0000","last_undegraded":"2024-03-30T13:12:30.473885+0000","last_fullsized":"2024-03-30T13:12:30.473885+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:36:54.824737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563837+0000","last_change":"2024-03-30T13:12:27.005524+0000","last_active":"2024-03-30T13:12:30.563837+0000","last_peered":"2024-03-30T13:12:30.563837+0000","last_clean":"2024-03-30T13:12:30.563837+0000","last_became_active":"2024-03-30T13:12:27.005130+0000","last_became_peered":"2024-03-30T13:12:27.005130+0000","last_unstale":"2024-03-30T13:12:30.563837+0000","last_undegraded":"2024-03-30T13:12:30.563837+0000","last_fullsized":"2024-03-30T13:12:30.563837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:21.640047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034112+0000","last_change":"2024-03-30T13:12:28.032597+0000","last_active":"2024-03-30T13:12:29.034112+0000","last_peered":"2024-03-30T13:12:29.034112+0000","last_clean":"2024-03-30T13:12:29.034112+0000","last_became_active":"2024-03-30T13:12:28.031858+0000","last_became_peered":"2024-03-30T13:12:28.031858+0000","last_unstale":"2024-03-30T13:12:29.034112+0000","last_undegraded":"2024-03-30T13:12:29.034112+0000","last_fullsized":"2024-03-30T13:12:29.034112+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:15:12.148112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.023878+0000","last_change":"2024-03-30T13:12:26.999973+0000","last_active":"2024-03-30T13:12:29.023878+0000","last_peered":"2024-03-30T13:12:29.023878+0000","last_clean":"2024-03-30T13:12:29.023878+0000","last_became_active":"2024-03-30T13:12:26.999607+0000","last_became_peered":"2024-03-30T13:12:26.999607+0000","last_unstale":"2024-03-30T13:12:29.023878+0000","last_undegraded":"2024-03-30T13:12:29.023878+0000","last_fullsized":"2024-03-30T13:12:29.023878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:50:21.528288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565544+0000","last_change":"2024-03-30T13:12:28.035367+0000","last_active":"2024-03-30T13:12:30.565544+0000","last_peered":"2024-03-30T13:12:30.565544+0000","last_clean":"2024-03-30T13:12:30.565544+0000","last_became_active":"2024-03-30T13:12:28.032369+0000","last_became_peered":"2024-03-30T13:12:28.032369+0000","last_unstale":"2024-03-30T13:12:30.565544+0000","last_undegraded":"2024-03-30T13:12:30.565544+0000","last_fullsized":"2024-03-30T13:12:30.565544+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:55:04.039804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.472785+0000","last_change":"2024-03-30T13:12:27.009633+0000","last_active":"2024-03-30T13:12:30.472785+0000","last_peered":"2024-03-30T13:12:30.472785+0000","last_clean":"2024-03-30T13:12:30.472785+0000","last_became_active":"2024-03-30T13:12:27.008472+0000","last_became_peered":"2024-03-30T13:12:27.008472+0000","last_unstale":"2024-03-30T13:12:30.472785+0000","last_undegraded":"2024-03-30T13:12:30.472785+0000","last_fullsized":"2024-03-30T13:12:30.472785+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:22:18.824554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564016+0000","last_change":"2024-03-30T13:12:28.024860+0000","last_active":"2024-03-30T13:12:30.564016+0000","last_peered":"2024-03-30T13:12:30.564016+0000","last_clean":"2024-03-30T13:12:30.564016+0000","last_became_active":"2024-03-30T13:12:28.023505+0000","last_became_peered":"2024-03-30T13:12:28.023505+0000","last_unstale":"2024-03-30T13:12:30.564016+0000","last_undegraded":"2024-03-30T13:12:30.564016+0000","last_fullsized":"2024-03-30T13:12:30.564016+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:56.969545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024034+0000","last_change":"2024-03-30T13:12:26.991064+0000","last_active":"2024-03-30T13:12:29.024034+0000","last_peered":"2024-03-30T13:12:29.024034+0000","last_clean":"2024-03-30T13:12:29.024034+0000","last_became_active":"2024-03-30T13:12:26.990695+0000","last_became_peered":"2024-03-30T13:12:26.990695+0000","last_unstale":"2024-03-30T13:12:29.024034+0000","last_undegraded":"2024-03-30T13:12:29.024034+0000","last_fullsized":"2024-03-30T13:12:29.024034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:14:34.418163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473359+0000","last_change":"2024-03-30T13:12:28.039993+0000","last_active":"2024-03-30T13:12:30.473359+0000","last_peered":"2024-03-30T13:12:30.473359+0000","last_clean":"2024-03-30T13:12:30.473359+0000","last_became_active":"2024-03-30T13:12:28.039679+0000","last_became_peered":"2024-03-30T13:12:28.039679+0000","last_unstale":"2024-03-30T13:12:30.473359+0000","last_undegraded":"2024-03-30T13:12:30.473359+0000","last_fullsized":"2024-03-30T13:12:30.473359+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:36.971723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473870+0000","last_change":"2024-03-30T13:12:27.014670+0000","last_active":"2024-03-30T13:12:30.473870+0000","last_peered":"2024-03-30T13:12:30.473870+0000","last_clean":"2024-03-30T13:12:30.473870+0000","last_became_active":"2024-03-30T13:12:27.010212+0000","last_became_peered":"2024-03-30T13:12:27.010212+0000","last_unstale":"2024-03-30T13:12:30.473870+0000","last_undegraded":"2024-03-30T13:12:30.473870+0000","last_fullsized":"2024-03-30T13:12:30.473870+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:00:54.669816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564346+0000","last_change":"2024-03-30T13:12:28.025448+0000","last_active":"2024-03-30T13:12:30.564346+0000","last_peered":"2024-03-30T13:12:30.564346+0000","last_clean":"2024-03-30T13:12:30.564346+0000","last_became_active":"2024-03-30T13:12:28.024574+0000","last_became_peered":"2024-03-30T13:12:28.024574+0000","last_unstale":"2024-03-30T13:12:30.564346+0000","last_undegraded":"2024-03-30T13:12:30.564346+0000","last_fullsized":"2024-03-30T13:12:30.564346+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:09.036154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565664+0000","last_change":"2024-03-30T13:12:27.014223+0000","last_active":"2024-03-30T13:12:30.565664+0000","last_peered":"2024-03-30T13:12:30.565664+0000","last_clean":"2024-03-30T13:12:30.565664+0000","last_became_active":"2024-03-30T13:12:27.013881+0000","last_became_peered":"2024-03-30T13:12:27.013881+0000","last_unstale":"2024-03-30T13:12:30.565664+0000","last_undegraded":"2024-03-30T13:12:30.565664+0000","last_fullsized":"2024-03-30T13:12:30.565664+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:39:44.931526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473841+0000","last_change":"2024-03-30T13:12:28.047142+0000","last_active":"2024-03-30T13:12:30.473841+0000","last_peered":"2024-03-30T13:12:30.473841+0000","last_clean":"2024-03-30T13:12:30.473841+0000","last_became_active":"2024-03-30T13:12:28.045729+0000","last_became_peered":"2024-03-30T13:12:28.045729+0000","last_unstale":"2024-03-30T13:12:30.473841+0000","last_undegraded":"2024-03-30T13:12:30.473841+0000","last_fullsized":"2024-03-30T13:12:30.473841+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:11:06.370369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564500+0000","last_change":"2024-03-30T13:12:27.008286+0000","last_active":"2024-03-30T13:12:30.564500+0000","last_peered":"2024-03-30T13:12:30.564500+0000","last_clean":"2024-03-30T13:12:30.564500+0000","last_became_active":"2024-03-30T13:12:27.007966+0000","last_became_peered":"2024-03-30T13:12:27.007966+0000","last_unstale":"2024-03-30T13:12:30.564500+0000","last_undegraded":"2024-03-30T13:12:30.564500+0000","last_fullsized":"2024-03-30T13:12:30.564500+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:08:47.384945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024066+0000","last_change":"2024-03-30T13:12:28.024846+0000","last_active":"2024-03-30T13:12:29.024066+0000","last_peered":"2024-03-30T13:12:29.024066+0000","last_clean":"2024-03-30T13:12:29.024066+0000","last_became_active":"2024-03-30T13:12:28.024158+0000","last_became_peered":"2024-03-30T13:12:28.024158+0000","last_unstale":"2024-03-30T13:12:29.024066+0000","last_undegraded":"2024-03-30T13:12:29.024066+0000","last_fullsized":"2024-03-30T13:12:29.024066+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:16.497968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473385+0000","last_change":"2024-03-30T13:12:27.019139+0000","last_active":"2024-03-30T13:12:30.473385+0000","last_peered":"2024-03-30T13:12:30.473385+0000","last_clean":"2024-03-30T13:12:30.473385+0000","last_became_active":"2024-03-30T13:12:27.017110+0000","last_became_peered":"2024-03-30T13:12:27.017110+0000","last_unstale":"2024-03-30T13:12:30.473385+0000","last_undegraded":"2024-03-30T13:12:30.473385+0000","last_fullsized":"2024-03-30T13:12:30.473385+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:54:10.909790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.474093+0000","last_change":"2024-03-30T13:12:28.048022+0000","last_active":"2024-03-30T13:12:30.474093+0000","last_peered":"2024-03-30T13:12:30.474093+0000","last_clean":"2024-03-30T13:12:30.474093+0000","last_became_active":"2024-03-30T13:12:28.045970+0000","last_became_peered":"2024-03-30T13:12:28.045970+0000","last_unstale":"2024-03-30T13:12:30.474093+0000","last_undegraded":"2024-03-30T13:12:30.474093+0000","last_fullsized":"2024-03-30T13:12:30.474093+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:02:48.439021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.472893+0000","last_change":"2024-03-30T13:12:27.024711+0000","last_active":"2024-03-30T13:12:30.472893+0000","last_peered":"2024-03-30T13:12:30.472893+0000","last_clean":"2024-03-30T13:12:30.472893+0000","last_became_active":"2024-03-30T13:12:27.024207+0000","last_became_peered":"2024-03-30T13:12:27.024207+0000","last_unstale":"2024-03-30T13:12:30.472893+0000","last_undegraded":"2024-03-30T13:12:30.472893+0000","last_fullsized":"2024-03-30T13:12:30.472893+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:48:04.905382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563353+0000","last_change":"2024-03-30T13:12:28.044126+0000","last_active":"2024-03-30T13:12:30.563353+0000","last_peered":"2024-03-30T13:12:30.563353+0000","last_clean":"2024-03-30T13:12:30.563353+0000","last_became_active":"2024-03-30T13:12:28.043860+0000","last_became_peered":"2024-03-30T13:12:28.043860+0000","last_unstale":"2024-03-30T13:12:30.563353+0000","last_undegraded":"2024-03-30T13:12:30.563353+0000","last_fullsized":"2024-03-30T13:12:30.563353+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:00:55.678177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563557+0000","last_change":"2024-03-30T13:12:27.019279+0000","last_active":"2024-03-30T13:12:30.563557+0000","last_peered":"2024-03-30T13:12:30.563557+0000","last_clean":"2024-03-30T13:12:30.563557+0000","last_became_active":"2024-03-30T13:12:27.013568+0000","last_became_peered":"2024-03-30T13:12:27.013568+0000","last_unstale":"2024-03-30T13:12:30.563557+0000","last_undegraded":"2024-03-30T13:12:30.563557+0000","last_fullsized":"2024-03-30T13:12:30.563557+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:58:21.640047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034665+0000","last_change":"2024-03-30T13:12:28.030372+0000","last_active":"2024-03-30T13:12:29.034665+0000","last_peered":"2024-03-30T13:12:29.034665+0000","last_clean":"2024-03-30T13:12:29.034665+0000","last_became_active":"2024-03-30T13:12:28.030060+0000","last_became_peered":"2024-03-30T13:12:28.030060+0000","last_unstale":"2024-03-30T13:12:29.034665+0000","last_undegraded":"2024-03-30T13:12:29.034665+0000","last_fullsized":"2024-03-30T13:12:29.034665+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:36:13.511033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.564926+0000","last_change":"2024-03-30T13:12:26.989155+0000","last_active":"2024-03-30T13:12:30.564926+0000","last_peered":"2024-03-30T13:12:30.564926+0000","last_clean":"2024-03-30T13:12:30.564926+0000","last_became_active":"2024-03-30T13:12:26.988891+0000","last_became_peered":"2024-03-30T13:12:26.988891+0000","last_unstale":"2024-03-30T13:12:30.564926+0000","last_undegraded":"2024-03-30T13:12:30.564926+0000","last_fullsized":"2024-03-30T13:12:30.564926+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:42:23.345888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026063+0000","last_change":"2024-03-30T13:12:28.039485+0000","last_active":"2024-03-30T13:12:29.026063+0000","last_peered":"2024-03-30T13:12:29.026063+0000","last_clean":"2024-03-30T13:12:29.026063+0000","last_became_active":"2024-03-30T13:12:28.039136+0000","last_became_peered":"2024-03-30T13:12:28.039136+0000","last_unstale":"2024-03-30T13:12:29.026063+0000","last_undegraded":"2024-03-30T13:12:29.026063+0000","last_fullsized":"2024-03-30T13:12:29.026063+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:15.580995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563517+0000","last_change":"2024-03-30T13:12:26.993198+0000","last_active":"2024-03-30T13:12:30.563517+0000","last_peered":"2024-03-30T13:12:30.563517+0000","last_clean":"2024-03-30T13:12:30.563517+0000","last_became_active":"2024-03-30T13:12:26.992905+0000","last_became_peered":"2024-03-30T13:12:26.992905+0000","last_unstale":"2024-03-30T13:12:30.563517+0000","last_undegraded":"2024-03-30T13:12:30.563517+0000","last_fullsized":"2024-03-30T13:12:30.563517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:15.489606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563847+0000","last_change":"2024-03-30T13:12:28.044084+0000","last_active":"2024-03-30T13:12:30.563847+0000","last_peered":"2024-03-30T13:12:30.563847+0000","last_clean":"2024-03-30T13:12:30.563847+0000","last_became_active":"2024-03-30T13:12:28.043811+0000","last_became_peered":"2024-03-30T13:12:28.043811+0000","last_unstale":"2024-03-30T13:12:30.563847+0000","last_undegraded":"2024-03-30T13:12:30.563847+0000","last_fullsized":"2024-03-30T13:12:30.563847+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:58:40.006257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026236+0000","last_change":"2024-03-30T13:12:26.987766+0000","last_active":"2024-03-30T13:12:29.026236+0000","last_peered":"2024-03-30T13:12:29.026236+0000","last_clean":"2024-03-30T13:12:29.026236+0000","last_became_active":"2024-03-30T13:12:26.987470+0000","last_became_peered":"2024-03-30T13:12:26.987470+0000","last_unstale":"2024-03-30T13:12:29.026236+0000","last_undegraded":"2024-03-30T13:12:29.026236+0000","last_fullsized":"2024-03-30T13:12:29.026236+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:39:20.310096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024895+0000","last_change":"2024-03-30T13:12:28.041679+0000","last_active":"2024-03-30T13:12:29.024895+0000","last_peered":"2024-03-30T13:12:29.024895+0000","last_clean":"2024-03-30T13:12:29.024895+0000","last_became_active":"2024-03-30T13:12:28.041373+0000","last_became_peered":"2024-03-30T13:12:28.041373+0000","last_unstale":"2024-03-30T13:12:29.024895+0000","last_undegraded":"2024-03-30T13:12:29.024895+0000","last_fullsized":"2024-03-30T13:12:29.024895+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:39.375738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565840+0000","last_change":"2024-03-30T13:12:27.008741+0000","last_active":"2024-03-30T13:12:30.565840+0000","last_peered":"2024-03-30T13:12:30.565840+0000","last_clean":"2024-03-30T13:12:30.565840+0000","last_became_active":"2024-03-30T13:12:27.008147+0000","last_became_peered":"2024-03-30T13:12:27.008147+0000","last_unstale":"2024-03-30T13:12:30.565840+0000","last_undegraded":"2024-03-30T13:12:30.565840+0000","last_fullsized":"2024-03-30T13:12:30.565840+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:51.894740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.025854+0000","last_change":"2024-03-30T13:12:28.033135+0000","last_active":"2024-03-30T13:12:29.025854+0000","last_peered":"2024-03-30T13:12:29.025854+0000","last_clean":"2024-03-30T13:12:29.025854+0000","last_became_active":"2024-03-30T13:12:28.028800+0000","last_became_peered":"2024-03-30T13:12:28.028800+0000","last_unstale":"2024-03-30T13:12:29.025854+0000","last_undegraded":"2024-03-30T13:12:29.025854+0000","last_fullsized":"2024-03-30T13:12:29.025854+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:22:19.832916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026249+0000","last_change":"2024-03-30T13:12:27.005991+0000","last_active":"2024-03-30T13:12:29.026249+0000","last_peered":"2024-03-30T13:12:29.026249+0000","last_clean":"2024-03-30T13:12:29.026249+0000","last_became_active":"2024-03-30T13:12:27.003764+0000","last_became_peered":"2024-03-30T13:12:27.003764+0000","last_unstale":"2024-03-30T13:12:29.026249+0000","last_undegraded":"2024-03-30T13:12:29.026249+0000","last_fullsized":"2024-03-30T13:12:29.026249+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:50:23.779432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028802+0000","last_change":"2024-03-30T13:12:28.040607+0000","last_active":"2024-03-30T13:12:29.028802+0000","last_peered":"2024-03-30T13:12:29.028802+0000","last_clean":"2024-03-30T13:12:29.028802+0000","last_became_active":"2024-03-30T13:12:28.040303+0000","last_became_peered":"2024-03-30T13:12:28.040303+0000","last_unstale":"2024-03-30T13:12:29.028802+0000","last_undegraded":"2024-03-30T13:12:29.028802+0000","last_fullsized":"2024-03-30T13:12:29.028802+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:53:52.056944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028248+0000","last_change":"2024-03-30T13:12:26.991406+0000","last_active":"2024-03-30T13:12:29.028248+0000","last_peered":"2024-03-30T13:12:29.028248+0000","last_clean":"2024-03-30T13:12:29.028248+0000","last_became_active":"2024-03-30T13:12:26.990643+0000","last_became_peered":"2024-03-30T13:12:26.990643+0000","last_unstale":"2024-03-30T13:12:29.028248+0000","last_undegraded":"2024-03-30T13:12:29.028248+0000","last_fullsized":"2024-03-30T13:12:29.028248+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:58:57.574756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563363+0000","last_change":"2024-03-30T13:12:28.027236+0000","last_active":"2024-03-30T13:12:30.563363+0000","last_peered":"2024-03-30T13:12:30.563363+0000","last_clean":"2024-03-30T13:12:30.563363+0000","last_became_active":"2024-03-30T13:12:28.026214+0000","last_became_peered":"2024-03-30T13:12:28.026214+0000","last_unstale":"2024-03-30T13:12:30.563363+0000","last_undegraded":"2024-03-30T13:12:30.563363+0000","last_fullsized":"2024-03-30T13:12:30.563363+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:09:57.186980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028428+0000","last_change":"2024-03-30T13:12:27.004641+0000","last_active":"2024-03-30T13:12:29.028428+0000","last_peered":"2024-03-30T13:12:29.028428+0000","last_clean":"2024-03-30T13:12:29.028428+0000","last_became_active":"2024-03-30T13:12:27.002667+0000","last_became_peered":"2024-03-30T13:12:27.002667+0000","last_unstale":"2024-03-30T13:12:29.028428+0000","last_undegraded":"2024-03-30T13:12:29.028428+0000","last_fullsized":"2024-03-30T13:12:29.028428+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:07:39.525496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473430+0000","last_change":"2024-03-30T13:12:28.046608+0000","last_active":"2024-03-30T13:12:30.473430+0000","last_peered":"2024-03-30T13:12:30.473430+0000","last_clean":"2024-03-30T13:12:30.473430+0000","last_became_active":"2024-03-30T13:12:28.046265+0000","last_became_peered":"2024-03-30T13:12:28.046265+0000","last_unstale":"2024-03-30T13:12:30.473430+0000","last_undegraded":"2024-03-30T13:12:30.473430+0000","last_fullsized":"2024-03-30T13:12:30.473430+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:27:01.959344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563463+0000","last_change":"2024-03-30T13:12:27.021918+0000","last_active":"2024-03-30T13:12:30.563463+0000","last_peered":"2024-03-30T13:12:30.563463+0000","last_clean":"2024-03-30T13:12:30.563463+0000","last_became_active":"2024-03-30T13:12:27.019605+0000","last_became_peered":"2024-03-30T13:12:27.019605+0000","last_unstale":"2024-03-30T13:12:30.563463+0000","last_undegraded":"2024-03-30T13:12:30.563463+0000","last_fullsized":"2024-03-30T13:12:30.563463+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-01T00:09:51.709095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034286+0000","last_change":"2024-03-30T13:12:28.031749+0000","last_active":"2024-03-30T13:12:29.034286+0000","last_peered":"2024-03-30T13:12:29.034286+0000","last_clean":"2024-03-30T13:12:29.034286+0000","last_became_active":"2024-03-30T13:12:28.031449+0000","last_became_peered":"2024-03-30T13:12:28.031449+0000","last_unstale":"2024-03-30T13:12:29.034286+0000","last_undegraded":"2024-03-30T13:12:29.034286+0000","last_fullsized":"2024-03-30T13:12:29.034286+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:42:06.499046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024910+0000","last_change":"2024-03-30T13:12:26.989095+0000","last_active":"2024-03-30T13:12:29.024910+0000","last_peered":"2024-03-30T13:12:29.024910+0000","last_clean":"2024-03-30T13:12:29.024910+0000","last_became_active":"2024-03-30T13:12:26.988760+0000","last_became_peered":"2024-03-30T13:12:26.988760+0000","last_unstale":"2024-03-30T13:12:29.024910+0000","last_undegraded":"2024-03-30T13:12:29.024910+0000","last_fullsized":"2024-03-30T13:12:29.024910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:42:42.922450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473102+0000","last_change":"2024-03-30T13:12:28.046772+0000","last_active":"2024-03-30T13:12:30.473102+0000","last_peered":"2024-03-30T13:12:30.473102+0000","last_clean":"2024-03-30T13:12:30.473102+0000","last_became_active":"2024-03-30T13:12:28.046371+0000","last_became_peered":"2024-03-30T13:12:28.046371+0000","last_unstale":"2024-03-30T13:12:30.473102+0000","last_undegraded":"2024-03-30T13:12:30.473102+0000","last_fullsized":"2024-03-30T13:12:30.473102+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:27:41.290838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563190+0000","last_change":"2024-03-30T13:12:27.024227+0000","last_active":"2024-03-30T13:12:30.563190+0000","last_peered":"2024-03-30T13:12:30.563190+0000","last_clean":"2024-03-30T13:12:30.563190+0000","last_became_active":"2024-03-30T13:12:27.023036+0000","last_became_peered":"2024-03-30T13:12:27.023036+0000","last_unstale":"2024-03-30T13:12:30.563190+0000","last_undegraded":"2024-03-30T13:12:30.563190+0000","last_fullsized":"2024-03-30T13:12:30.563190+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:38.367377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026174+0000","last_change":"2024-03-30T13:12:28.045779+0000","last_active":"2024-03-30T13:12:29.026174+0000","last_peered":"2024-03-30T13:12:29.026174+0000","last_clean":"2024-03-30T13:12:29.026174+0000","last_became_active":"2024-03-30T13:12:28.045503+0000","last_became_peered":"2024-03-30T13:12:28.045503+0000","last_unstale":"2024-03-30T13:12:29.026174+0000","last_undegraded":"2024-03-30T13:12:29.026174+0000","last_fullsized":"2024-03-30T13:12:29.026174+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T15:07:39.375738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565262+0000","last_change":"2024-03-30T13:12:26.999648+0000","last_active":"2024-03-30T13:12:30.565262+0000","last_peered":"2024-03-30T13:12:30.565262+0000","last_clean":"2024-03-30T13:12:30.565262+0000","last_became_active":"2024-03-30T13:12:26.999374+0000","last_became_peered":"2024-03-30T13:12:26.999374+0000","last_unstale":"2024-03-30T13:12:30.565262+0000","last_undegraded":"2024-03-30T13:12:30.565262+0000","last_fullsized":"2024-03-30T13:12:30.565262+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T19:36:12.502672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.035403+0000","last_change":"2024-03-30T13:12:28.024738+0000","last_active":"2024-03-30T13:12:29.035403+0000","last_peered":"2024-03-30T13:12:29.035403+0000","last_clean":"2024-03-30T13:12:29.035403+0000","last_became_active":"2024-03-30T13:12:28.024043+0000","last_became_peered":"2024-03-30T13:12:28.024043+0000","last_unstale":"2024-03-30T13:12:29.035403+0000","last_undegraded":"2024-03-30T13:12:29.035403+0000","last_fullsized":"2024-03-30T13:12:29.035403+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:13:52.903101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"41'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026363+0000","last_change":"2024-03-30T13:12:27.007320+0000","last_active":"2024-03-30T13:12:29.026363+0000","last_peered":"2024-03-30T13:12:29.026363+0000","last_clean":"2024-03-30T13:12:29.026363+0000","last_became_active":"2024-03-30T13:12:27.006579+0000","last_became_peered":"2024-03-30T13:12:27.006579+0000","last_unstale":"2024-03-30T13:12:29.026363+0000","last_undegraded":"2024-03-30T13:12:29.026363+0000","last_fullsized":"2024-03-30T13:12:29.026363+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T18:19:42.500630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.028489+0000","last_change":"2024-03-30T13:12:28.036105+0000","last_active":"2024-03-30T13:12:29.028489+0000","last_peered":"2024-03-30T13:12:29.028489+0000","last_clean":"2024-03-30T13:12:29.028489+0000","last_became_active":"2024-03-30T13:12:28.035011+0000","last_became_peered":"2024-03-30T13:12:28.035011+0000","last_unstale":"2024-03-30T13:12:29.028489+0000","last_undegraded":"2024-03-30T13:12:29.028489+0000","last_fullsized":"2024-03-30T13:12:29.028489+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:23:31.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.026276+0000","last_change":"2024-03-30T13:12:27.017279+0000","last_active":"2024-03-30T13:12:29.026276+0000","last_peered":"2024-03-30T13:12:29.026276+0000","last_clean":"2024-03-30T13:12:29.026276+0000","last_became_active":"2024-03-30T13:12:27.016328+0000","last_became_peered":"2024-03-30T13:12:27.016328+0000","last_unstale":"2024-03-30T13:12:29.026276+0000","last_undegraded":"2024-03-30T13:12:29.026276+0000","last_fullsized":"2024-03-30T13:12:29.026276+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:10:05.400003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.025777+0000","last_change":"2024-03-30T13:12:28.040795+0000","last_active":"2024-03-30T13:12:29.025777+0000","last_peered":"2024-03-30T13:12:29.025777+0000","last_clean":"2024-03-30T13:12:29.025777+0000","last_became_active":"2024-03-30T13:12:28.040492+0000","last_became_peered":"2024-03-30T13:12:28.040492+0000","last_unstale":"2024-03-30T13:12:29.025777+0000","last_undegraded":"2024-03-30T13:12:29.025777+0000","last_fullsized":"2024-03-30T13:12:29.025777+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:13:13.336869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.473414+0000","last_change":"2024-03-30T13:12:27.022469+0000","last_active":"2024-03-30T13:12:30.473414+0000","last_peered":"2024-03-30T13:12:30.473414+0000","last_clean":"2024-03-30T13:12:30.473414+0000","last_became_active":"2024-03-30T13:12:27.021653+0000","last_became_peered":"2024-03-30T13:12:27.021653+0000","last_unstale":"2024-03-30T13:12:30.473414+0000","last_undegraded":"2024-03-30T13:12:30.473414+0000","last_fullsized":"2024-03-30T13:12:30.473414+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:08.027793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.563615+0000","last_change":"2024-03-30T13:12:28.023180+0000","last_active":"2024-03-30T13:12:30.563615+0000","last_peered":"2024-03-30T13:12:30.563615+0000","last_clean":"2024-03-30T13:12:30.563615+0000","last_became_active":"2024-03-30T13:12:28.022915+0000","last_became_peered":"2024-03-30T13:12:28.022915+0000","last_unstale":"2024-03-30T13:12:30.563615+0000","last_undegraded":"2024-03-30T13:12:30.563615+0000","last_fullsized":"2024-03-30T13:12:30.563615+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:52:55.003890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034715+0000","last_change":"2024-03-30T13:12:27.005433+0000","last_active":"2024-03-30T13:12:29.034715+0000","last_peered":"2024-03-30T13:12:29.034715+0000","last_clean":"2024-03-30T13:12:29.034715+0000","last_became_active":"2024-03-30T13:12:27.005116+0000","last_became_peered":"2024-03-30T13:12:27.005116+0000","last_unstale":"2024-03-30T13:12:29.034715+0000","last_undegraded":"2024-03-30T13:12:29.034715+0000","last_fullsized":"2024-03-30T13:12:29.034715+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T16:58:35.953478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565633+0000","last_change":"2024-03-30T13:12:28.040218+0000","last_active":"2024-03-30T13:12:30.565633+0000","last_peered":"2024-03-30T13:12:30.565633+0000","last_clean":"2024-03-30T13:12:30.565633+0000","last_became_active":"2024-03-30T13:12:28.039540+0000","last_became_peered":"2024-03-30T13:12:28.039540+0000","last_unstale":"2024-03-30T13:12:30.565633+0000","last_undegraded":"2024-03-30T13:12:30.565633+0000","last_fullsized":"2024-03-30T13:12:30.565633+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T21:40:20.889934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565188+0000","last_change":"2024-03-30T13:12:26.998465+0000","last_active":"2024-03-30T13:12:30.565188+0000","last_peered":"2024-03-30T13:12:30.565188+0000","last_clean":"2024-03-30T13:12:30.565188+0000","last_became_active":"2024-03-30T13:12:26.998168+0000","last_became_peered":"2024-03-30T13:12:26.998168+0000","last_unstale":"2024-03-30T13:12:30.565188+0000","last_undegraded":"2024-03-30T13:12:30.565188+0000","last_fullsized":"2024-03-30T13:12:30.565188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:04:51.509774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034573+0000","last_change":"2024-03-30T13:12:28.032142+0000","last_active":"2024-03-30T13:12:29.034573+0000","last_peered":"2024-03-30T13:12:29.034573+0000","last_clean":"2024-03-30T13:12:29.034573+0000","last_became_active":"2024-03-30T13:12:28.031694+0000","last_became_peered":"2024-03-30T13:12:28.031694+0000","last_unstale":"2024-03-30T13:12:29.034573+0000","last_undegraded":"2024-03-30T13:12:29.034573+0000","last_fullsized":"2024-03-30T13:12:29.034573+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T22:08:48.393306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.474219+0000","last_change":"2024-03-30T13:12:27.009963+0000","last_active":"2024-03-30T13:12:30.474219+0000","last_peered":"2024-03-30T13:12:30.474219+0000","last_clean":"2024-03-30T13:12:30.474219+0000","last_became_active":"2024-03-30T13:12:27.007546+0000","last_became_peered":"2024-03-30T13:12:27.007546+0000","last_unstale":"2024-03-30T13:12:30.474219+0000","last_undegraded":"2024-03-30T13:12:30.474219+0000","last_fullsized":"2024-03-30T13:12:30.474219+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T13:13:02.680729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:30.565004+0000","last_change":"2024-03-30T13:12:28.029109+0000","last_active":"2024-03-30T13:12:30.565004+0000","last_peered":"2024-03-30T13:12:30.565004+0000","last_clean":"2024-03-30T13:12:30.565004+0000","last_became_active":"2024-03-30T13:12:28.028466+0000","last_became_peered":"2024-03-30T13:12:28.028466+0000","last_unstale":"2024-03-30T13:12:30.565004+0000","last_undegraded":"2024-03-30T13:12:30.565004+0000","last_fullsized":"2024-03-30T13:12:30.565004+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:26.967503+0000","last_clean_scrub_stamp":"2024-03-30T13:12:26.967503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T20:51:42.137629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.034893+0000","last_change":"2024-03-30T13:12:27.002016+0000","last_active":"2024-03-30T13:12:29.034893+0000","last_peered":"2024-03-30T13:12:29.034893+0000","last_clean":"2024-03-30T13:12:29.034893+0000","last_became_active":"2024-03-30T13:12:27.001703+0000","last_became_peered":"2024-03-30T13:12:27.001703+0000","last_unstale":"2024-03-30T13:12:29.034893+0000","last_undegraded":"2024-03-30T13:12:29.034893+0000","last_fullsized":"2024-03-30T13:12:29.034893+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:12:25.959142+0000","last_clean_scrub_stamp":"2024-03-30T13:12:25.959142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T14:28:24.223034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":204,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-03-30T13:12:29.024273+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:12:29.024273+0000","last_peered":"2024-03-30T13:12:29.024273+0000","last_clean":"2024-03-30T13:12:29.024273+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:12:29.024273+0000","last_undegraded":"2024-03-30T13:12:29.024273+0000","last_fullsized":"2024-03-30T13:12:29.024273+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":199523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":471040,"data_stored":199487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509465,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":201189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509466,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":471040,"data_stored":199543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509466,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":483328,"data_stored":201647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36388,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335452,"statfs":{"total":96636764160,"available":96599502848,"internally_reserved":0,"allocated":1064960,"data_stored":789843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509465,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31652,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340188,"statfs":{"total":96636764160,"available":96604352512,"internally_reserved":0,"allocated":475136,"data_stored":199545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1064960,"data_stored":790403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:12:36.953 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-30T13:12:36.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-30T13:12:37.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.131+0000 7fdfec2b1640 1 Processor -- start 2024-03-30T13:12:37.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.131+0000 7fdfec2b1640 1 -- start start 2024-03-30T13:12:37.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfec2b1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdfe406dac0 0x7fdfe4141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfec2b1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406c2e0 0x7fdfe4141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfec2b1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfe4142550 con 0x7fdfe415d130 2024-03-30T13:12:37.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfec2b1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe40cf830 con 0x7fdfe406dac0 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfec2b1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe4142310 con 0x7fdfe406c2e0 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfea84e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406c2e0 0x7fdfe4141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfe984c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdfe406dac0 0x7fdfe4141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfea84e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406c2e0 0x7fdfe4141dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:43934/0 (socket says 172.21.15.43:43934) 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfea84e640 1 -- 172.21.15.43:0/3175354887 learned_addr learned my addr 172.21.15.43:0/3175354887 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfea84e640 1 -- 172.21.15.43:0/3175354887 >> v2:172.21.15.113:3300/0 conn(0x7fdfe406dac0 msgr2=0x7fdfe4141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfea84e640 1 --2- 172.21.15.43:0/3175354887 >> v2:172.21.15.113:3300/0 conn(0x7fdfe406dac0 0x7fdfe4141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfea84e640 1 -- 172.21.15.43:0/3175354887 >> v1:172.21.15.113:6789/0 conn(0x7fdfe415d130 legacy=0x7fdfe406d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.135+0000 7fdfea84e640 1 -- 172.21.15.43:0/3175354887 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe406b800 con 0x7fdfe406c2e0 2024-03-30T13:12:37.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea84e640 1 --2- 172.21.15.43:0/3175354887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406c2e0 0x7fdfe4141dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009e50 tx=0x7fdfdc01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=799e821f4469809a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:37.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfe904b640 1 -- 172.21.15.43:0/3175354887 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0a57f0 con 0x7fdfe406c2e0 2024-03-30T13:12:37.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfe904b640 1 -- 172.21.15.43:0/3175354887 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfdc0a5990 con 0x7fdfe406c2e0 2024-03-30T13:12:37.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfe904b640 1 -- 172.21.15.43:0/3175354887 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0a5d60 con 0x7fdfe406c2e0 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3175354887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406c2e0 msgr2=0x7fdfe4141dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3175354887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406c2e0 0x7fdfe4141dd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc009e50 tx=0x7fdfdc01ca30 comp rx=0 tx=0).stop 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3175354887 shutdown_connections 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3175354887 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406c2e0 0x7fdfe4141dd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3175354887 >> v2:172.21.15.113:3300/0 conn(0x7fdfe406dac0 0x7fdfe4141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3175354887 >> 172.21.15.43:0/3175354887 conn(0x7fdfe40b1c60 msgr2=0x7fdfe40b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3175354887 shutdown_connections 2024-03-30T13:12:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3175354887 wait complete. 2024-03-30T13:12:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 Processor -- start 2024-03-30T13:12:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- start start 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406c2e0 0x7fdfe413e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406dac0 0x7fdfe4138420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe415d130 0x7fdfe4138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe40cf1b0 con 0x7fdfe406c2e0 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe40cf030 con 0x7fdfe415d130 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfec2b1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe40cf330 con 0x7fdfe406dac0 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406c2e0 0x7fdfe413e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406c2e0 0x7fdfe413e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34950/0 (socket says 172.21.15.43:34950) 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 -- 172.21.15.43:0/3119836417 learned_addr learned my addr 172.21.15.43:0/3119836417 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfe984c640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406dac0 0x7fdfe4138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea84e640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe415d130 0x7fdfe4138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 -- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406dac0 msgr2=0x7fdfe4138420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406dac0 0x7fdfe4138420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 -- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe415d130 msgr2=0x7fdfe4138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe415d130 0x7fdfe4138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.139+0000 7fdfea04d640 1 -- 172.21.15.43:0/3119836417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe40aa4d0 con 0x7fdfe406c2e0 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfea04d640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406c2e0 0x7fdfe413e3a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c3c0 tx=0x7fdfd400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfdaffd640 1 -- 172.21.15.43:0/3119836417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4017070 con 0x7fdfe406c2e0 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe4151a80 con 0x7fdfe406c2e0 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfdaffd640 1 -- 172.21.15.43:0/3119836417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd40052d0 con 0x7fdfe406c2e0 2024-03-30T13:12:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfdaffd640 1 -- 172.21.15.43:0/3119836417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4003840 con 0x7fdfe406c2e0 2024-03-30T13:12:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe4137940 con 0x7fdfe406c2e0 2024-03-30T13:12:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe4068900 con 0x7fdfe406c2e0 2024-03-30T13:12:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfdaffd640 1 -- 172.21.15.43:0/3119836417 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdfd401a070 con 0x7fdfe406c2e0 2024-03-30T13:12:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfdaffd640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfc8067fc0 0x7fdfc806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfdaffd640 1 -- 172.21.15.43:0/3119836417 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7fdfd40859e0 con 0x7fdfe406c2e0 2024-03-30T13:12:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfe984c640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfc8067fc0 0x7fdfc806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.143+0000 7fdfe984c640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfc8067fc0 0x7fdfc806a480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40b2550 tx=0x7fdfe0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:37.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.147+0000 7fdfdaffd640 1 -- 172.21.15.43:0/3119836417 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdfd4005480 con 0x7fdfe406c2e0 2024-03-30T13:12:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.371+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fdfe406cba0 con 0x7fdfe406c2e0 2024-03-30T13:12:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfdaffd640 1 -- 172.21.15.43:0/3119836417 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7fdfd4055fd0 con 0x7fdfe406c2e0 2024-03-30T13:12:37.382 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-03-30T13:12:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAmDwhmDh1uOhAAyQ23sd4h6j2HVM6D/hX5WA== 2024-03-30T13:12:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-03-30T13:12:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-03-30T13:12:37.382 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-30T13:12:37.383 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-03-30T13:12:37.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfc8067fc0 msgr2=0x7fdfc806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfc8067fc0 0x7fdfc806a480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40b2550 tx=0x7fdfe0008040 comp rx=0 tx=0).stop 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406c2e0 msgr2=0x7fdfe413e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406c2e0 0x7fdfe413e3a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c3c0 tx=0x7fdfd400c880 comp rx=0 tx=0).stop 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 shutdown_connections 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdfc8067fc0 0x7fdfc806a480 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdfe415d130 0x7fdfe4138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdfe406dac0 0x7fdfe4138420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 --2- 172.21.15.43:0/3119836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdfe406c2e0 0x7fdfe413e3a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 >> 172.21.15.43:0/3119836417 conn(0x7fdfe40b1c60 msgr2=0x7fdfe40b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.375+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 shutdown_connections 2024-03-30T13:12:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.379+0000 7fdfec2b1640 1 -- 172.21.15.43:0/3119836417 wait complete. 2024-03-30T13:12:37.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:12:37.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152fdd8640 1 Processor -- start 2024-03-30T13:12:37.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152fdd8640 1 -- start start 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152fdd8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1528147e10 0x7f1528071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152fdd8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1528162b90 0x7f152806e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152fdd8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1528072070 con 0x7f152806ec50 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152fdd8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f152806e280 con 0x7f1528147e10 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152fdd8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f152806dc30 con 0x7f1528162b90 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152db74640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1528147e10 0x7f1528071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152db74640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1528147e10 0x7f1528071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34956/0 (socket says 172.21.15.43:34956) 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152db74640 1 -- 172.21.15.43:0/950307402 learned_addr learned my addr 172.21.15.43:0/950307402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152d373640 1 --2- 172.21.15.43:0/950307402 >> v2:172.21.15.113:3300/0 conn(0x7f1528162b90 0x7f152806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152db74640 1 -- 172.21.15.43:0/950307402 >> v2:172.21.15.113:3300/0 conn(0x7f1528162b90 msgr2=0x7f152806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:37.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152db74640 1 --2- 172.21.15.43:0/950307402 >> v2:172.21.15.113:3300/0 conn(0x7f1528162b90 0x7f152806e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152db74640 1 -- 172.21.15.43:0/950307402 >> v1:172.21.15.113:6789/0 conn(0x7f152806ec50 legacy=0x7f152806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.519+0000 7f152db74640 1 -- 172.21.15.43:0/950307402 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152806df10 con 0x7f1528147e10 2024-03-30T13:12:37.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152db74640 1 --2- 172.21.15.43:0/950307402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528071740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f15280d0e60 tx=0x7f152401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3b39b297a6bc5c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:37.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152cb72640 1 -- 172.21.15.43:0/950307402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15240b1070 con 0x7f1528147e10 2024-03-30T13:12:37.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152cb72640 1 -- 172.21.15.43:0/950307402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15240ac600 con 0x7f1528147e10 2024-03-30T13:12:37.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152cb72640 1 -- 172.21.15.43:0/950307402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15240ac9d0 con 0x7f1528147e10 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 -- 172.21.15.43:0/950307402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 msgr2=0x7f1528071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 --2- 172.21.15.43:0/950307402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528071740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f15280d0e60 tx=0x7f152401c490 comp rx=0 tx=0).stop 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 -- 172.21.15.43:0/950307402 shutdown_connections 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 --2- 172.21.15.43:0/950307402 >> v2:172.21.15.113:3300/0 conn(0x7f1528162b90 0x7f152806e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 --2- 172.21.15.43:0/950307402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528071740 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 -- 172.21.15.43:0/950307402 >> 172.21.15.43:0/950307402 conn(0x7f15280cfee0 msgr2=0x7f15280d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 -- 172.21.15.43:0/950307402 shutdown_connections 2024-03-30T13:12:37.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.523+0000 7f152fdd8640 1 -- 172.21.15.43:0/950307402 wait complete. 2024-03-30T13:12:37.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 Processor -- start 2024-03-30T13:12:37.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 -- start start 2024-03-30T13:12:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152806ec50 0x7f1528164190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1528162b90 0x7f1528142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f152806cf00 con 0x7f1528147e10 2024-03-30T13:12:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f152806cd80 con 0x7f152806ec50 2024-03-30T13:12:37.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f152816b0d0 con 0x7f1528162b90 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528143bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34958/0 (socket says 172.21.15.43:34958) 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 -- 172.21.15.43:0/250142641 learned_addr learned my addr 172.21.15.43:0/250142641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152e375640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1528162b90 0x7f1528142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152db74640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152806ec50 0x7f1528164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 -- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1528162b90 msgr2=0x7f1528142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1528162b90 0x7f1528142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 -- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152806ec50 msgr2=0x7f1528164190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152806ec50 0x7f1528164190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 -- 172.21.15.43:0/250142641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1528164870 con 0x7f1528147e10 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152d373640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528143bd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f151800e930 tx=0x7f151800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f151e7fc640 1 -- 172.21.15.43:0/250142641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1518017070 con 0x7f1528147e10 2024-03-30T13:12:37.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f151e7fc640 1 -- 172.21.15.43:0/250142641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f151800ca00 con 0x7f1528147e10 2024-03-30T13:12:37.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1528164b50 con 0x7f1528147e10 2024-03-30T13:12:37.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f151e7fc640 1 -- 172.21.15.43:0/250142641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f151800cdb0 con 0x7f1528147e10 2024-03-30T13:12:37.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1528142150 con 0x7f1528147e10 2024-03-30T13:12:37.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.527+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1528068890 con 0x7f1528147e10 2024-03-30T13:12:37.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.531+0000 7f151e7fc640 1 -- 172.21.15.43:0/250142641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f151801a070 con 0x7f1528147e10 2024-03-30T13:12:37.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.531+0000 7f151e7fc640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f150c067fc0 0x7f150c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:37.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.531+0000 7f151e7fc640 1 -- 172.21.15.43:0/250142641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6159+0+0 (secure 0 0 0) 0x7f1518086f40 con 0x7f1528147e10 2024-03-30T13:12:37.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.531+0000 7f152db74640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f150c067fc0 0x7f150c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:37.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.531+0000 7f152db74640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f150c067fc0 0x7f150c06a480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15240ae000 tx=0x7f1524008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:37.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.535+0000 7f151e7fc640 1 -- 172.21.15.43:0/250142641 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f15180575d0 con 0x7f1528147e10 2024-03-30T13:12:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1528072c70 con 0x7f1528147e10 2024-03-30T13:12:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f151e7fc640 1 -- 172.21.15.43:0/250142641 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5187 (secure 0 0 0) 0x7f1518056ed0 con 0x7f1528147e10 2024-03-30T13:12:37.738 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:12:37.738 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4271726137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4271726137},{"type":"v1","addr":"172.21.15.43:6834","nonce":4271726137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1748034672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1748034672},{"type":"v1","addr":"172.21.15.43:6835","nonce":1748034672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3254700273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3254700273},{"type":"v1","addr":"172.21.15.113:6835","nonce":3254700273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:12:27.356973+0000","modified":"2024-03-30T13:12:30.394858+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4880":{"gid":4880,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1104336007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1104336007},{"type":"v1","addr":"172.21.15.113:6837","nonce":1104336007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4880,"qdb_cluster":[4880]},"id":2}]} 2024-03-30T13:12:37.738 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 18 2024-03-30T13:12:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f150c067fc0 msgr2=0x7f150c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f150c067fc0 0x7f150c06a480 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15240ae000 tx=0x7f1524008040 comp rx=0 tx=0).stop 2024-03-30T13:12:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 msgr2=0x7f1528143bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528143bd0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f151800e930 tx=0x7f151800edf0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 shutdown_connections 2024-03-30T13:12:37.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f150c067fc0 0x7f150c06a480 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1528162b90 0x7f1528142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1528147e10 0x7f1528143bd0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 --2- 172.21.15.43:0/250142641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f152806ec50 0x7f1528164190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:37.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 >> 172.21.15.43:0/250142641 conn(0x7f15280cfee0 msgr2=0x7f15280b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:37.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 shutdown_connections 2024-03-30T13:12:37.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:37.731+0000 7f152fdd8640 1 -- 172.21.15.43:0/250142641 wait complete. 2024-03-30T13:12:38.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-30T13:12:27.356973+0000', 'modified': '2024-03-30T13:12:30.394858+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4880}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4880': {'gid': 4880, 'name': 'd', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.113:6837/1104336007', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.113:6836', 'nonce': 1104336007}, {'type': 'v1', 'addr': '172.21.15.113:6837', 'nonce': 1104336007}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4880, 'qdb_cluster': [4880]} 2024-03-30T13:12:38.160 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-30T13:12:38.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 Processor -- start 2024-03-30T13:12:38.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 -- start start 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b9c144270 0x7f2b9c144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b9c0b4490 0x7f2b9c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2b9c144fa0 con 0x7f2b9c0b4490 2024-03-30T13:12:38.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2b9c15fce0 con 0x7f2b9c144270 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba28d1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9c0b33f0 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b9c0b4490 0x7f2b9c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b9c0b4490 0x7f2b9c06c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34968/0 (socket says 172.21.15.43:34968) 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2646530232 learned_addr learned my addr 172.21.15.43:0/2646530232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/2646530232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/2646530232 >> v2:172.21.15.113:3300/0 conn(0x7f2b9c144270 0x7f2b9c144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2646530232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b2ef0 msgr2=0x7f2b9c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/2646530232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2646530232 >> v2:172.21.15.113:3300/0 conn(0x7f2b9c144270 msgr2=0x7f2b9c144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/2646530232 >> v2:172.21.15.113:3300/0 conn(0x7f2b9c144270 0x7f2b9c144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.291+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2646530232 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c0b36d0 con 0x7f2b9c0b4490 2024-03-30T13:12:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/2646530232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b4490 0x7f2b9c06c670 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c072000 tx=0x7f2b9401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b243da1c404c8db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2b9affd640 1 -- 172.21.15.43:0/2646530232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940a5840 con 0x7f2b9c0b4490 2024-03-30T13:12:38.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2b9affd640 1 -- 172.21.15.43:0/2646530232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b940a59e0 con 0x7f2b9c0b4490 2024-03-30T13:12:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2b9affd640 1 -- 172.21.15.43:0/2646530232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940a5db0 con 0x7f2b9c0b4490 2024-03-30T13:12:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 -- 172.21.15.43:0/2646530232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b4490 msgr2=0x7f2b9c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/2646530232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b4490 0x7f2b9c06c670 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c072000 tx=0x7f2b9401c9c0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 -- 172.21.15.43:0/2646530232 shutdown_connections 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/2646530232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b4490 0x7f2b9c06c670 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/2646530232 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/2646530232 >> v2:172.21.15.113:3300/0 conn(0x7f2b9c144270 0x7f2b9c144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 -- 172.21.15.43:0/2646530232 >> 172.21.15.43:0/2646530232 conn(0x7f2b9c0cfee0 msgr2=0x7f2b9c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 -- 172.21.15.43:0/2646530232 shutdown_connections 2024-03-30T13:12:38.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.295+0000 7f2ba28d1640 1 -- 172.21.15.43:0/2646530232 wait complete. 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 Processor -- start 2024-03-30T13:12:38.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 -- start start 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b4490 0x7f2b9c1431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b9c144270 0x7f2b9c1f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c0cdd10 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c06cfc0 con 0x7f2b9c144270 2024-03-30T13:12:38.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9c0cde90 con 0x7f2b9c0b4490 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34974/0 (socket says 172.21.15.43:34974) 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 -- 172.21.15.43:0/152002179 learned_addr learned my addr 172.21.15.43:0/152002179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b4490 0x7f2b9c1431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b9c144270 0x7f2b9c1f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 -- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b4490 msgr2=0x7f2b9c1431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b4490 0x7f2b9c1431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 -- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b9c144270 msgr2=0x7f2b9c1f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b9c144270 0x7f2b9c1f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 -- 172.21.15.43:0/152002179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c1f85e0 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b9c144270 0x7f2b9c1f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15da50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000d800 tx=0x7f2b9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b98ff9640 1 -- 172.21.15.43:0/152002179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000b6c0 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b98ff9640 1 -- 172.21.15.43:0/152002179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b90019070 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9c15c9b0 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b98ff9640 1 -- 172.21.15.43:0/152002179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000ba60 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9c15cf70 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b98ff9640 1 -- 172.21.15.43:0/152002179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2b9000bc00 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.299+0000 7f2b98ff9640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b88067fc0 0x7f2b8806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b98ff9640 1 -- 172.21.15.43:0/152002179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2b90086c50 con 0x7f2b9c0b2ef0 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b88067fc0 0x7f2b8806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 Processor -- start 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- start start 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b88067fc0 0x7f2b8806a480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0cb9e0 tx=0x7f2b8c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9328b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e9323eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e9323f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559e93091950 con 0x559e9323f0b0 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559e9302e990 con 0x559e9328bdc0 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x559e9308d900 con 0x559e9328b4a0 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e9323f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e9323f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34980/0 (socket says 172.21.15.43:34980) 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2501911557 learned_addr learned my addr 172.21.15.43:0/2501911557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e9323eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9328b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 msgr2=0x559e9328b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9328b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 msgr2=0x559e9323eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e9323eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/2501911557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e930d6100 con 0x559e9323f0b0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e9323eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e9323f490 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2b940af040 tx=0x7f2b94004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576435d48a84ad2a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b81ffb640 1 -- 172.21.15.43:0/2501911557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940a5840 con 0x559e9323f0b0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b81ffb640 1 -- 172.21.15.43:0/2501911557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b940a59e0 con 0x559e9323f0b0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2b81ffb640 1 -- 172.21.15.43:0/2501911557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940a5d90 con 0x559e9323f0b0 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- 172.21.15.43:0/2501911557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 msgr2=0x559e9323f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e9323f490 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2b940af040 tx=0x7f2b94004ac0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- 172.21.15.43:0/2501911557 shutdown_connections 2024-03-30T13:12:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e9323f490 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e9323eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/2501911557 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9328b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- 172.21.15.43:0/2501911557 >> 172.21.15.43:0/2501911557 conn(0x559e93129b80 msgr2=0x559e9326e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- 172.21.15.43:0/2501911557 shutdown_connections 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- 172.21.15.43:0/2501911557 wait complete. 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.303+0000 7f2ba5a53000 1 -- start start 2024-03-30T13:12:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 Processor -- start 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9323f0b0 0x559e93246b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9326f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9328bdc0 0x559e9326fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559e930297b0 con 0x559e9328bdc0 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x559e9304f3e0 con 0x559e9323f0b0 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x559e93095f80 con 0x559e9328b4a0 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 10 client.0 _ll_register_callbacks cb 0x559e930f20f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9328bdc0 0x559e9326fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9328bdc0 0x559e9326fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:34988/0 (socket says 172.21.15.43:34988) 2024-03-30T13:12:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/3429166980 learned_addr learned my addr 172.21.15.43:0/3429166980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9326f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9323f0b0 0x559e93246b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 msgr2=0x559e9326f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9326f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9323f0b0 msgr2=0x559e93246b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9323f0b0 0x559e93246b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 -- 172.21.15.43:0/3429166980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9305b8c0 con 0x559e9328bdc0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9328bdc0 0x559e9326fbd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2b940072a0 tx=0x7f2b9401b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8629931a12044b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b7affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b7affd640 1 -- 172.21.15.43:0/3429166980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b940c9060 con 0x559e9328bdc0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b7affd640 1 -- 172.21.15.43:0/3429166980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2b940039a0 con 0x559e9328bdc0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b940a59e0 con 0x559e9328bdc0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b7affd640 1 -- 172.21.15.43:0/3429166980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2b940cf050 con 0x559e9328bdc0 2024-03-30T13:12:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 10 client.5739 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x559e93019e80 con 0x559e9328bdc0 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b7affd640 1 -- 172.21.15.43:0/3429166980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b940a59e0 con 0x559e9328bdc0 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b7affd640 1 -- 172.21.15.43:0/3429166980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f2b940d2020 con 0x559e9328bdc0 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 10 client.5739 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 10 client.5739 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 20 client.5739 populate_metadata read hostname 'smithi043' 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e93247070 0x559e9329d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 4 client.5739 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e9302cbc0 con 0x559e93247070 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e93247070 0x559e9329d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b7affd640 10 client.5739 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.307+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e93247070 0x559e9329d550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbdfa9c9428d053e server_cookie=7a2c269d68878797 in_seq=0 out_seq=0 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2b7affd640 1 -- 172.21.15.43:0/3429166980 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x559e9302cbc0 con 0x559e93247070 2024-03-30T13:12:38.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2b7affd640 10 client.5739 handle_command_reply: tid=0 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 client.5739 shutdown 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9328bdc0 msgr2=0x559e9326fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9328bdc0 0x559e9326fbd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2b940072a0 tx=0x7f2b9401b200 comp rx=0 tx=0).stop 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 shutdown_connections 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e93247070 0x559e9329d550 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9328bdc0 0x559e9326fbd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e9326f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3429166980 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9323f0b0 0x559e93246b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 >> 172.21.15.43:0/3429166980 conn(0x559e93129b80 msgr2=0x559e9323e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2b7affd640 0 client.5739 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 shutdown_connections 2024-03-30T13:12:38.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3429166980 wait complete. 2024-03-30T13:12:38.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.311+0000 7f2ba5a53000 20 client.5739 trim_cache size 0 max 16384 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.327+0000 7f2ba5a53000 1 Processor -- start 2024-03-30T13:12:38.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.327+0000 7f2ba5a53000 1 -- start start 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328b4a0 0x559e9329d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328bdc0 0x559e9329da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c204500 con 0x559e9323f0b0 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c203580 con 0x559e9328b4a0 2024-03-30T13:12:38.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9c1fb570 con 0x559e9328bdc0 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35000/0 (socket says 172.21.15.43:35000) 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/3160689023 learned_addr learned my addr 172.21.15.43:0/3160689023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/3160689023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328bdc0 msgr2=0x559e9329da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/3160689023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328bdc0 0x559e9329da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/3160689023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328b4a0 msgr2=0x559e9329d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/3160689023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328b4a0 0x559e9329d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/3160689023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e9304c490 con 0x559e9323f0b0 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/3160689023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932a0af0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b4b0 tx=0x7f2b900046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135759db4a3ba171 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b817fa640 1 -- 172.21.15.43:0/3160689023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9005f680 con 0x559e9323f0b0 2024-03-30T13:12:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b817fa640 1 -- 172.21.15.43:0/3160689023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b90054b40 con 0x559e9323f0b0 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b817fa640 1 -- 172.21.15.43:0/3160689023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b90054da0 con 0x559e9323f0b0 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3160689023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 msgr2=0x559e932a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3160689023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932a0af0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b4b0 tx=0x7f2b900046d0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3160689023 shutdown_connections 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3160689023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328bdc0 0x559e9329da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3160689023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328b4a0 0x559e9329d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/3160689023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932a0af0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3160689023 >> 172.21.15.43:0/3160689023 conn(0x559e93129b80 msgr2=0x559e93298200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3160689023 shutdown_connections 2024-03-30T13:12:38.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- 172.21.15.43:0/3160689023 wait complete. 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- start start 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 Processor -- start 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932e0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e932e14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e932e19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x559e93037e80 con 0x559e9323f0b0 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c202f80 con 0x559e9328bdc0 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x559e930297b0 con 0x559e9328b4a0 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 10 client.0 _ll_register_callbacks cb 0x7f2b94009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932e0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932e0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35006/0 (socket says 172.21.15.43:35006) 2024-03-30T13:12:38.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/677324550 learned_addr learned my addr 172.21.15.43:0/677324550 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e932e14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 msgr2=0x559e932e14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e932e14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 msgr2=0x559e932e19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba0e6e640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e932e19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e932e19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 -- 172.21.15.43:0/677324550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e9304ae80 con 0x559e9323f0b0 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e932e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b9bfff640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932e0aa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b4b0 tx=0x7f2b940a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee27b0a826e2cefc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b7affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b7affd640 1 -- 172.21.15.43:0/677324550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9005f680 con 0x559e9323f0b0 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2b7affd640 1 -- 172.21.15.43:0/677324550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2b90075380 con 0x559e9323f0b0 2024-03-30T13:12:38.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.331+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b90054b40 con 0x559e9323f0b0 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b7affd640 1 -- 172.21.15.43:0/677324550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2b90084fa0 con 0x559e9323f0b0 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 10 client.5745 fetch_fsmap learned FSMap version 18 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x559e93038270 con 0x559e9323f0b0 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b7affd640 1 -- 172.21.15.43:0/677324550 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b90054b40 con 0x559e9323f0b0 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b7affd640 1 -- 172.21.15.43:0/677324550 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 18) ==== 3494+0+0 (secure 0 0 0) 0x7f2b90013070 con 0x559e9323f0b0 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 10 client.5745 fetch_fsmap finished waiting for FSMap version 18 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 10 client.5745 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 20 client.5745 populate_metadata read hostname 'smithi043' 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e932e1f30 0x559e932e2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 4 client.5745 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "status"}) -- 0x559e9309ee90 con 0x559e932e1f30 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e932e1f30 0x559e932e2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b9b7fe640 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e932e1f30 0x559e932e2310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8adafcb1b29e39c8 server_cookie=b210aaad002aa0d9 in_seq=0 out_seq=0 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b7affd640 10 client.5745 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:12:38.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b7affd640 1 -- 172.21.15.43:0/677324550 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x559e9309ee90 con 0x559e932e1f30 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b7affd640 10 client.5745 handle_command_reply: tid=0 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 client.5745 shutdown 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 msgr2=0x559e932e0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932e0aa0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000b4b0 tx=0x7f2b940a7000 comp rx=0 tx=0).stop 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 shutdown_connections 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x559e932e1f30 0x559e932e2310 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x559e9328bdc0 0x559e932e19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x559e9328b4a0 0x559e932e14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 --2- 172.21.15.43:0/677324550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x559e9323f0b0 0x559e932e0aa0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2b7affd640 0 client.5745 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 >> 172.21.15.43:0/677324550 conn(0x559e93129b80 msgr2=0x559e93298a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 shutdown_connections 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 1 -- 172.21.15.43:0/677324550 wait complete. 2024-03-30T13:12:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba5a53000 20 client.5745 trim_cache size 0 max 16384 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "ace86ea7-ebbf-4c39-b9cf-d22b6fea8142", 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4880, 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 10.319445234, 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 18, 2024-03-30T13:12:38.349 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 43, 2024-03-30T13:12:38.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 41, 2024-03-30T13:12:38.350 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 17.244056296 2024-03-30T13:12:38.350 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:12:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b88067fc0 msgr2=0x7f2b8806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b88067fc0 0x7f2b8806a480 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0cb9e0 tx=0x7f2b8c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b2ef0 msgr2=0x7f2b9c15da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15da50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000d800 tx=0x7f2b9000dcc0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 shutdown_connections 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b88067fc0 0x7f2b8806a480 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b9c144270 0x7f2b9c1f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b9c0b4490 0x7f2b9c1431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 --2- 172.21.15.43:0/152002179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b9c0b2ef0 0x7f2b9c15da50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 >> 172.21.15.43:0/152002179 conn(0x7f2b9c0cfee0 msgr2=0x7f2b9c0cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 shutdown_connections 2024-03-30T13:12:38.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.335+0000 7f2ba28d1640 1 -- 172.21.15.43:0/152002179 wait complete. 2024-03-30T13:12:38.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:12:38.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.467+0000 7fc080d99640 1 Processor -- start 2024-03-30T13:12:38.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.467+0000 7fc080d99640 1 -- start start 2024-03-30T13:12:38.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc080d99640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc07c15c310 0x7fc07c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc080d99640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc07c15c6f0 con 0x7fc07c0715e0 2024-03-30T13:12:38.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc080d99640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc07c0ce6b0 con 0x7fc07c0d1aa0 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc080d99640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc07c0ce990 con 0x7fc07c15c310 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07ad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc07c0715e0 0x7fc07c0b1e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34536/0 (socket says 172.21.15.43:34536) 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07ad9d640 1 -- 172.21.15.43:0/3223398289 learned_addr learned my addr 172.21.15.43:0/3223398289 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3317395880 0 0) 0x7fc07c15c6f0 con 0x7fc07c0715e0 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc068003530 con 0x7fc07c0715e0 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07a59c640 1 --2- 172.21.15.43:0/3223398289 >> v2:172.21.15.113:3300/0 conn(0x7fc07c15c310 0x7fc07c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 20805539 0 0) 0x7fc068003530 con 0x7fc07c0715e0 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 >> v2:172.21.15.113:3300/0 conn(0x7fc07c15c310 msgr2=0x7fc07c0b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 --2- 172.21.15.43:0/3223398289 >> v2:172.21.15.113:3300/0 conn(0x7fc07c15c310 0x7fc07c0b4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 >> v1:172.21.15.113:6789/0 conn(0x7fc07c0d1aa0 legacy=0x7fc07c070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c0cc890 con 0x7fc07c0715e0 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fc074001a00 con 0x7fc07c0715e0 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 >> v1:172.21.15.43:6789/0 conn(0x7fc07c0715e0 legacy=0x7fc07c0b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc068003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc068004810 0x7fc068004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc068005420 0x7fc06800ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c0ce990 con 0x7fc0680039c0 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc06800c330 con 0x7fc068005420 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc06800c530 con 0x7fc068004810 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07a59c640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc068003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07a59c640 1 -- 172.21.15.43:0/3223398289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc068004810 msgr2=0x7fc068004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07a59c640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc068004810 0x7fc068004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07a59c640 1 -- 172.21.15.43:0/3223398289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc068005420 msgr2=0x7fc06800ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:38.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07a59c640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc068005420 0x7fc06800ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07a59c640 1 -- 172.21.15.43:0/3223398289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc068003530 con 0x7fc0680039c0 2024-03-30T13:12:38.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.471+0000 7fc07ad9d640 1 -- 172.21.15.43:0/3223398289 reap_dead start 2024-03-30T13:12:38.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc07a59c640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc068003de0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00a990 tx=0x7fc06c01c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab75d5d0dc9fe20 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b1070 con 0x7fc0680039c0 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06c0a8050 con 0x7fc0680039c0 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc079d9b640 1 -- 172.21.15.43:0/3223398289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00a590 con 0x7fc0680039c0 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- 172.21.15.43:0/3223398289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 msgr2=0x7fc068003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc068003de0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00a990 tx=0x7fc06c01c5a0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- 172.21.15.43:0/3223398289 shutdown_connections 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 --2- 172.21.15.43:0/3223398289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc068003de0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- 172.21.15.43:0/3223398289 >> 172.21.15.43:0/3223398289 conn(0x7fc07c0d2110 msgr2=0x7fc07c0d2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- 172.21.15.43:0/3223398289 shutdown_connections 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- 172.21.15.43:0/3223398289 wait complete. 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 Processor -- start 2024-03-30T13:12:38.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- start start 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc07c14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07c06f370 0x7fc07c06f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc07c14d460 0x7fc07c14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c070ed0 con 0x7fc0680039c0 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c0ce360 con 0x7fc07c14d460 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc080d99640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc07c071050 con 0x7fc07c06f370 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc07a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc07c14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc07a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc07c14b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35022/0 (socket says 172.21.15.43:35022) 2024-03-30T13:12:38.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc07a59c640 1 -- 172.21.15.43:0/834692539 learned_addr learned my addr 172.21.15.43:0/834692539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc071d9b640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07c06f370 0x7fc07c06f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.475+0000 7fc07ad9d640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc07c14d460 0x7fc07c14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc07a59c640 1 -- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07c06f370 msgr2=0x7fc07c06f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc07a59c640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07c06f370 0x7fc07c06f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc07a59c640 1 -- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc07c14d460 msgr2=0x7fc07c14d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc07a59c640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc07c14d460 0x7fc07c14d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc07a59c640 1 -- 172.21.15.43:0/834692539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c14e190 con 0x7fc0680039c0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc07ad9d640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc07c14d460 0x7fc07c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc07a59c640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc07c14b830 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc06c01c3d0 tx=0x7fc068005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc0737fe640 1 -- 172.21.15.43:0/834692539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0bd070 con 0x7fc0680039c0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc0737fe640 1 -- 172.21.15.43:0/834692539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc06c0acd20 con 0x7fc0680039c0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c14e470 con 0x7fc0680039c0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc0737fe640 1 -- 172.21.15.43:0/834692539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b1040 con 0x7fc0680039c0 2024-03-30T13:12:38.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c14ea30 con 0x7fc0680039c0 2024-03-30T13:12:38.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc07c06fd30 con 0x7fc0680039c0 2024-03-30T13:12:38.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc0737fe640 1 -- 172.21.15.43:0/834692539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc06c0c0070 con 0x7fc0680039c0 2024-03-30T13:12:38.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc0737fe640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc05c067fc0 0x7fc05c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:38.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc0737fe640 1 -- 172.21.15.43:0/834692539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fc06c11fb10 con 0x7fc0680039c0 2024-03-30T13:12:38.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc071d9b640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc05c067fc0 0x7fc05c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:38.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.479+0000 7fc071d9b640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc05c067fc0 0x7fc05c06a480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc064007860 tx=0x7fc064009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:38.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.483+0000 7fc0737fe640 1 -- 172.21.15.43:0/834692539 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc06c0162e0 con 0x7fc0680039c0 2024-03-30T13:12:38.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.683+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc07c068890 con 0x7fc0680039c0 2024-03-30T13:12:38.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.683+0000 7fc0737fe640 1 -- 172.21.15.43:0/834692539 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7fc06c00a630 con 0x7fc0680039c0 2024-03-30T13:12:38.691 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T13:12:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc05c067fc0 msgr2=0x7fc05c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc05c067fc0 0x7fc05c06a480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc064007860 tx=0x7fc064009040 comp rx=0 tx=0).stop 2024-03-30T13:12:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 msgr2=0x7fc07c14b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc07c14b830 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc06c01c3d0 tx=0x7fc068005840 comp rx=0 tx=0).stop 2024-03-30T13:12:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 shutdown_connections 2024-03-30T13:12:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc05c067fc0 0x7fc05c06a480 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc07c14d460 0x7fc07c14d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc07c06f370 0x7fc07c06f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 --2- 172.21.15.43:0/834692539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0680039c0 0x7fc07c14b830 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 >> 172.21.15.43:0/834692539 conn(0x7fc07c0d2110 msgr2=0x7fc07c0c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 shutdown_connections 2024-03-30T13:12:38.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38.687+0000 7fc080d99640 1 -- 172.21.15.43:0/834692539 wait complete. 2024-03-30T13:12:38.709 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:12:38.709 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:12:38.709 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-03-30T13:12:38.709 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-30T13:12:38.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:38.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:12:38.710 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:12:38.710 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:12:38.710 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:12:38.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:38.710 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-03-30T13:12:38.716 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:12:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:12:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:12:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.716 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42396sec preferred_lft 42396sec 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link 2024-03-30T13:12:38.717 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 02:42:bf:2f:c7:16 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.718 INFO:teuthology.orchestra.run.smithi043.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run.smithi043.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:38.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:38.719 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:12:38.769 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-03-30T13:12:38.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:38.773 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-03-30T13:12:38.819 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-03-30T13:12:38.819 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:12:38.819 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-30T13:12:38.819 INFO:teuthology.orchestra.run.smithi043.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T13:12:38.819 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-30T13:12:38.819 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:12:38.819 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:12:38.819 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-03-30T13:12:38.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-30T13:12:38.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-30T13:12:38.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-30T13:12:38.820 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-03-30T13:12:38.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:12:38.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:12:38.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:38.942 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-30T13:12:38.948 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T13:12:38.948 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T13:12:38.948 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:12:38.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:38.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-03-30T13:12:39.000 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-30T13:12:39.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:12:39.000 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:39.045 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-30T13:12:39.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:12:39.046 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:39.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:12:39.151 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:12:39.202 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:12:39.202 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:12:39.202 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:12:39.202 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:12:39.202 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:12:39.203 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:12:39.203 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:12:39.203 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:12:39.203 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:12:39.203 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:12:39.204 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:12:39.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:12:39.206 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:12:39.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:39.207 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:12:39.253 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T13:12:39.253 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-30T13:12:39.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:12:39.314 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:12:39.362 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:12:39.362 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:12:39.362 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:12:39.362 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:12:39.363 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:12:39.364 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:12:39.365 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:12:39.365 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:12:39.365 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:12:39.365 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:12:39.365 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:12:39.365 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:12:39.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:39.365 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:12:39.382 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-03-30T13:12:39.375+0000 7fbde9e36580 -1 init, newargv = 0x558d1cab3b40 newargc=15 2024-03-30T13:12:39.383 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[38949]: starting ceph client 2024-03-30T13:12:39.388 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[38949]: starting fuse 2024-03-30T13:12:40.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-03-30T13:12:40.506 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/run/lock 2024-03-30T13:12:40.556 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/fs/nfsd 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/run/rpc_pipefs 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-03-30T13:12:40.557 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:12:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:40.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:40.559 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-03-30T13:12:40.610 INFO:teuthology.orchestra.run.smithi043.stdout:45 2024-03-30T13:12:40.610 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-30T13:12:40.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:12:40.610 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:12:40.611 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-03-30T13:12:40.612 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-03-30T13:12:40.737 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.38949.asok 2024-03-30T13:12:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:12:40.749 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38949.asok 2024-03-30T13:12:40.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:40.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38949.asok status 2024-03-30T13:12:40.908 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "38949", 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-03-30T13:12:40.909 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5757, 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 5757 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3055345982 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:40.910 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3055345982 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.5757 192.168.0.1:0/3055345982", 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/3055345982", 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 18, 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 43, 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-03-30T13:12:40.911 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:12:40.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:40.923 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-03-30T13:12:40.923 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:40.929 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-03-30T13:12:40.930 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:40.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:40.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-30T13:12:40.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-30T13:12:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa83ccb640 1 Processor -- start 2024-03-30T13:12:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa83ccb640 1 -- start start 2024-03-30T13:12:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa83ccb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffa7c06c670 0x7ffa7c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa83ccb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c0ccef0 con 0x7ffa7c06ca50 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa83ccb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa7c0b3710 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa83ccb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ffa7c0b34c0 con 0x7ffa7c06c670 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa81a67640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffa7c06c670 0x7ffa7c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa82268640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffa7c06ca50 0x7ffa7c0ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34544/0 (socket says 172.21.15.43:34544) 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa82268640 1 -- 172.21.15.43:0/2543360780 learned_addr learned my addr 172.21.15.43:0/2543360780 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa81a67640 1 -- 172.21.15.43:0/2543360780 >> v1:172.21.15.113:6789/0 conn(0x7ffa7c0b41f0 legacy=0x7ffa7c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa81a67640 1 -- 172.21.15.43:0/2543360780 >> v1:172.21.15.43:6789/0 conn(0x7ffa7c06ca50 legacy=0x7ffa7c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.151+0000 7ffa81a67640 1 -- 172.21.15.43:0/2543360780 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c0b39f0 con 0x7ffa7c06c670 2024-03-30T13:12:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa81a67640 1 --2- 172.21.15.43:0/2543360780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c06c670 0x7ffa7c0cc910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0097c0 tx=0x7ffa6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9512bafdad52047 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa80a65640 1 -- 172.21.15.43:0/2543360780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0b1070 con 0x7ffa7c06c670 2024-03-30T13:12:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa80a65640 1 -- 172.21.15.43:0/2543360780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0ac410 con 0x7ffa7c06c670 2024-03-30T13:12:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa80a65640 1 -- 172.21.15.43:0/2543360780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0ac7e0 con 0x7ffa7c06c670 2024-03-30T13:12:41.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2543360780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c06c670 msgr2=0x7ffa7c0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2543360780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c06c670 0x7ffa7c0cc910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0097c0 tx=0x7ffa6c01c4a0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2543360780 shutdown_connections 2024-03-30T13:12:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2543360780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c06c670 0x7ffa7c0cc910 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2543360780 >> 172.21.15.43:0/2543360780 conn(0x7ffa7c0cfee0 msgr2=0x7ffa7c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2543360780 shutdown_connections 2024-03-30T13:12:41.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2543360780 wait complete. 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 Processor -- start 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- start start 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa7c06c670 0x7ffa7c13a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa7c06ca50 0x7ffa7c072270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c0b41f0 0x7ffa7c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c16a080 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c169f00 con 0x7ffa7c06ca50 2024-03-30T13:12:41.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa83ccb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7c16a200 con 0x7ffa7c06c670 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c0b41f0 0x7ffa7c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c0b41f0 0x7ffa7c155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:35038/0 (socket says 172.21.15.43:35038) 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 -- 172.21.15.43:0/2354670921 learned_addr learned my addr 172.21.15.43:0/2354670921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa81266640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa7c06ca50 0x7ffa7c072270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa81a67640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa7c06c670 0x7ffa7c13a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 -- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa7c06c670 msgr2=0x7ffa7c13a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa7c06c670 0x7ffa7c13a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 -- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa7c06ca50 msgr2=0x7ffa7c072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa7c06ca50 0x7ffa7c072270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.155+0000 7ffa82268640 1 -- 172.21.15.43:0/2354670921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c072ba0 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa82268640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c0b41f0 0x7ffa7c155f80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400ca80 tx=0x7ffa7400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa727fc640 1 -- 172.21.15.43:0/2354670921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74017070 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa727fc640 1 -- 172.21.15.43:0/2354670921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa7400ac30 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c072e80 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa727fc640 1 -- 172.21.15.43:0/2354670921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74012600 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c153900 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c0733a0 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa727fc640 1 -- 172.21.15.43:0/2354670921 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ffa7401a070 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa727fc640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffa60067fc0 0x7ffa6006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa727fc640 1 -- 172.21.15.43:0/2354670921 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7ffa74086360 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa81a67640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffa60067fc0 0x7ffa6006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:41.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.159+0000 7ffa81a67640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffa60067fc0 0x7ffa6006a480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0ae000 tx=0x7ffa6c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:41.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.163+0000 7ffa727fc640 1 -- 172.21.15.43:0/2354670921 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ffa7400ae20 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.367+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ffa7c068890 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.367+0000 7ffa727fc640 1 -- 172.21.15.43:0/2354670921 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7ffa74056260 con 0x7ffa7c0b41f0 2024-03-30T13:12:41.375 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-30T13:12:41.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffa60067fc0 msgr2=0x7ffa6006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffa60067fc0 0x7ffa6006a480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0ae000 tx=0x7ffa6c008040 comp rx=0 tx=0).stop 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c0b41f0 msgr2=0x7ffa7c155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c0b41f0 0x7ffa7c155f80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffa7400ca80 tx=0x7ffa7400cf40 comp rx=0 tx=0).stop 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 shutdown_connections 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ffa60067fc0 0x7ffa6006a480 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa7c0b41f0 0x7ffa7c155f80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ffa7c06ca50 0x7ffa7c072270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 --2- 172.21.15.43:0/2354670921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ffa7c06c670 0x7ffa7c13a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 >> 172.21.15.43:0/2354670921 conn(0x7ffa7c0cfee0 msgr2=0x7ffa7c15cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:41.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 shutdown_connections 2024-03-30T13:12:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:41.371+0000 7ffa83ccb640 1 -- 172.21.15.43:0/2354670921 wait complete. 2024-03-30T13:12:41.394 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi113.front.sepia.ceph.com 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-30T13:12:41.395 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:12:41.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:41.395 DEBUG:teuthology.orchestra.run.smithi113:> ip addr 2024-03-30T13:12:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-30T13:12:41.402 INFO:teuthology.orchestra.run.smithi113.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-30T13:12:41.402 INFO:teuthology.orchestra.run.smithi113.stdout: inet 127.0.0.1/8 scope host lo 2024-03-30T13:12:41.402 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.402 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 ::1/128 scope host 2024-03-30T13:12:41.402 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.402 INFO:teuthology.orchestra.run.smithi113.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:44 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f0 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:d8:db:45 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: altname enp7s0f1 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c0 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0c:c4:7a:8f:d7:c1 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.21.15.113/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-30T13:12:41.403 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft 42407sec preferred_lft 42407sec 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::ec4:7aff:fe8f:d7c1/64 scope link 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 02:42:46:0d:39:59 brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:12:41.404 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi113.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi113.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi113.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run.smithi113.stdout: valid_lft forever preferred_lft forever 2024-03-30T13:12:41.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:41.405 DEBUG:teuthology.orchestra.run.smithi113:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-30T13:12:41.458 INFO:teuthology.orchestra.run.smithi113.stdout:1 2024-03-30T13:12:41.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:41.459 DEBUG:teuthology.orchestra.run.smithi113:> route 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:Kernel IP routing table 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-30T13:12:41.509 INFO:teuthology.orchestra.run.smithi113.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-30T13:12:41.510 DEBUG:teuthology.orchestra.run.smithi113:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-30T13:12:41.510 DEBUG:teuthology.orchestra.run.smithi113:> set -e 2024-03-30T13:12:41.510 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-30T13:12:41.510 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-30T13:12:41.510 DEBUG:teuthology.orchestra.run.smithi113:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-30T13:12:41.510 DEBUG:teuthology.orchestra.run.smithi113:> ') 2024-03-30T13:12:41.600 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:12:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-30T13:12:41.626 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:12:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:12:41.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:41.633 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-30T13:12:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-30T13:12:41.681 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T13:12:41.681 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:12:41.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:41.682 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link set brx.0 up 2024-03-30T13:12:41.738 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-30T13:12:41.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:12:41.738 DEBUG:teuthology.orchestra.run.smithi113:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run.smithi113.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-30T13:12:41.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-30T13:12:41.785 DEBUG:teuthology.orchestra.run.smithi113:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:41.833 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:12:41.890 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:12:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:12:41.937 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:12:41.938 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:12:41.939 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:12:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:12:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:12:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:12:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:12:41.940 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:12:41.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:41.941 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:12:41.988 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-30T13:12:41.989 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-30T13:12:42.034 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:12:42.048 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:12:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:12:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:12:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:12:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:12:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:12:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:12:42.098 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:12:42.099 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:12:42.100 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:12:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:12:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:12:42.101 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:12:42.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:42.101 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:12:42.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:2024-03-30T13:12:42.106+0000 7f97bd21f580 -1 init, newargv = 0x55a7569eea70 newargc=15 2024-03-30T13:12:42.114 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[33309]: starting ceph client 2024-03-30T13:12:42.120 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[33309]: starting fuse 2024-03-30T13:12:43.220 DEBUG:teuthology.orchestra.run.smithi113:> sudo modprobe fuse 2024-03-30T13:12:43.234 DEBUG:teuthology.orchestra.run.smithi113:> cat /proc/self/mounts | awk '{print $2}' 2024-03-30T13:12:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:/sys 2024-03-30T13:12:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:/proc 2024-03-30T13:12:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:/dev 2024-03-30T13:12:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/pts 2024-03-30T13:12:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:/run 2024-03-30T13:12:43.281 INFO:teuthology.orchestra.run.smithi113.stdout:/ 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/security 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/shm 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/run/lock 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/cgroup 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/pstore 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/bpf 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/sys/fs/binfmt_misc 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/hugepages 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/dev/mqueue 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/debug 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/tracing 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/proc/fs/nfsd 2024-03-30T13:12:43.282 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/fs/fuse/connections 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/sys/kernel/config 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/credentials/systemd-sysusers.service 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/rpc_pipefs 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/user/1000 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-4 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-5 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-6 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/var/lib/ceph/osd/ceph-7 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:12:43.283 INFO:teuthology.orchestra.run.smithi113.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:12:43.284 INFO:teuthology.orchestra.run.smithi113.stdout:/home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:43.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:43.285 DEBUG:teuthology.orchestra.run.smithi113:> ls /sys/fs/fuse/connections 2024-03-30T13:12:43.332 INFO:teuthology.orchestra.run.smithi113.stdout:45 2024-03-30T13:12:43.333 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> sudo stdin-killer -- python3 -c ' 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> import glob 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> import re 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> import os 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> import subprocess 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> def _find_admin_socket(client_name): 2024-03-30T13:12:43.333 DEBUG:teuthology.orchestra.run.smithi113:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> files = glob.glob(asok_path) 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> # Given a non-glob path, it better be there 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> if "*" not in asok_path: 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> assert(len(files) == 1) 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> return files[0] 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> for f in files: 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> contents = proc_f.read() 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> if mountpoint in contents: 2024-03-30T13:12:43.334 DEBUG:teuthology.orchestra.run.smithi113:> return f 2024-03-30T13:12:43.335 DEBUG:teuthology.orchestra.run.smithi113:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-30T13:12:43.335 DEBUG:teuthology.orchestra.run.smithi113:> 2024-03-30T13:12:43.335 DEBUG:teuthology.orchestra.run.smithi113:> print(_find_admin_socket("client.1")) 2024-03-30T13:12:43.335 DEBUG:teuthology.orchestra.run.smithi113:> ' 2024-03-30T13:12:43.453 INFO:teuthology.orchestra.run.smithi113.stdout:/var/run/ceph/ceph-client.1.33309.asok 2024-03-30T13:12:43.456 INFO:teuthology.orchestra.run.smithi113.stderr:2024-03-30T13:12:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-30T13:12:43.464 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33309.asok 2024-03-30T13:12:43.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:43.464 DEBUG:teuthology.orchestra.run.smithi113:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33309.asok status 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout:{ 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout: "metadata": { 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_sha1": "ac1710396f9f7c389fa4b407711bc5157a8c0bf7", 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout: "ceph_version": "ceph version 19.0.0-2519-gac171039 (ac1710396f9f7c389fa4b407711bc5157a8c0bf7) squid (dev)", 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout: "entity_id": "1", 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout: "hostname": "smithi113", 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-30T13:12:43.621 INFO:teuthology.orchestra.run.smithi113.stdout: "pid": "33309", 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "root": "/" 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_count": 0, 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "dentry_pinned_count": 0, 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "id": 5021, 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "inst": { 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "name": { 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "client", 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "num": 5021 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:12:43.622 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2166690519 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: } 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": { 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "type": "v1", 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "addr": "192.168.0.1:0", 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "nonce": 2166690519 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: }, 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "inst_str": "client.5021 192.168.0.1:0/2166690519", 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "addr_str": "192.168.0.1:0/2166690519", 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "inode_count": 1, 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "mds_epoch": 18, 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch": 43, 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "osd_epoch_barrier": 0, 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "blocklisted": false, 2024-03-30T13:12:43.623 INFO:teuthology.orchestra.run.smithi113.stdout: "fs_name": "cephfs" 2024-03-30T13:12:43.624 INFO:teuthology.orchestra.run.smithi113.stdout:} 2024-03-30T13:12:43.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:43.635 DEBUG:teuthology.orchestra.run.smithi113:> stat --file-system '--printf=%T 2024-03-30T13:12:43.635 DEBUG:teuthology.orchestra.run.smithi113:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:43.641 INFO:teuthology.orchestra.run.smithi113.stdout:fuseblk 2024-03-30T13:12:43.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:43.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:12:43.642 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-30T13:12:43.694 DEBUG:tasks.cephfs.test_uninlining:setting inline_data:1 2024-03-30T13:12:43.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs inline_data 1 --yes-i-really-really-mean-it 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 Processor -- start 2024-03-30T13:12:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 -- start start 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d41449a0 0x7f90d415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90d415f220 0x7f90d40ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90d40ce860 0x7f90d4072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f90d415fc80 con 0x7f90d40ce860 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f90d4144d80 con 0x7f90d415f220 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90db6ed640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f90d415f6c0 con 0x7f90d41449a0 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90d40ce860 0x7f90d4072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90d40ce860 0x7f90d4072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47754/0 (socket says 172.21.15.43:47754) 2024-03-30T13:12:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 -- 172.21.15.43:0/307158411 learned_addr learned my addr 172.21.15.43:0/307158411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9489640 1 --2- 172.21.15.43:0/307158411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d41449a0 0x7f90d415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d8c88640 1 --2- 172.21.15.43:0/307158411 >> v2:172.21.15.113:3300/0 conn(0x7f90d415f220 0x7f90d40ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 -- 172.21.15.43:0/307158411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d41449a0 msgr2=0x7f90d415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 --2- 172.21.15.43:0/307158411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d41449a0 0x7f90d415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 -- 172.21.15.43:0/307158411 >> v2:172.21.15.113:3300/0 conn(0x7f90d415f220 msgr2=0x7f90d40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 --2- 172.21.15.43:0/307158411 >> v2:172.21.15.113:3300/0 conn(0x7f90d415f220 0x7f90d40ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.827+0000 7f90d9c8a640 1 -- 172.21.15.43:0/307158411 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d415f9a0 con 0x7f90d40ce860 2024-03-30T13:12:43.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90d9c8a640 1 --2- 172.21.15.43:0/307158411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d40ce860 0x7f90d4072200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f90d40cd8d0 tx=0x7f90cc01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffc272e46f3952f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90cbfff640 1 -- 172.21.15.43:0/307158411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a5630 con 0x7f90d40ce860 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90cbfff640 1 -- 172.21.15.43:0/307158411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc0a57d0 con 0x7f90d40ce860 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90cbfff640 1 -- 172.21.15.43:0/307158411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0a5ba0 con 0x7f90d40ce860 2024-03-30T13:12:43.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 -- 172.21.15.43:0/307158411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d40ce860 msgr2=0x7f90d4072200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 --2- 172.21.15.43:0/307158411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d40ce860 0x7f90d4072200 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f90d40cd8d0 tx=0x7f90cc01c8a0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 -- 172.21.15.43:0/307158411 shutdown_connections 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 --2- 172.21.15.43:0/307158411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d40ce860 0x7f90d4072200 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 --2- 172.21.15.43:0/307158411 >> v2:172.21.15.113:3300/0 conn(0x7f90d415f220 0x7f90d40ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 --2- 172.21.15.43:0/307158411 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d41449a0 0x7f90d415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 -- 172.21.15.43:0/307158411 >> 172.21.15.43:0/307158411 conn(0x7f90d40b1b80 msgr2=0x7f90d40b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 -- 172.21.15.43:0/307158411 shutdown_connections 2024-03-30T13:12:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.831+0000 7f90db6ed640 1 -- 172.21.15.43:0/307158411 wait complete. 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 Processor -- start 2024-03-30T13:12:43.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 -- start start 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40ce860 0x7f90d4137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d41449a0 0x7f90d415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d415f220 0x7f90d41f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90d40ca5f0 con 0x7f90d41449a0 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90d40ca470 con 0x7f90d40ce860 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f90d40ca770 con 0x7f90d415f220 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d41449a0 0x7f90d415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d41449a0 0x7f90d415ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47764/0 (socket says 172.21.15.43:47764) 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 -- 172.21.15.43:0/3334003201 learned_addr learned my addr 172.21.15.43:0/3334003201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d9489640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40ce860 0x7f90d4137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d9c8a640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d415f220 0x7f90d41f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 -- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d415f220 msgr2=0x7f90d41f8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d415f220 0x7f90d41f8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 -- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40ce860 msgr2=0x7f90d4137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40ce860 0x7f90d4137fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 -- 172.21.15.43:0/3334003201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d41f8850 con 0x7f90d41449a0 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d9489640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40ce860 0x7f90d4137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90d8c88640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d41449a0 0x7f90d415ad40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f90c400d5f0 tx=0x7f90c400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90c9ffb640 1 -- 172.21.15.43:0/3334003201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40125f0 con 0x7f90d41449a0 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90c9ffb640 1 -- 172.21.15.43:0/3334003201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c4019070 con 0x7f90d41449a0 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d41385f0 con 0x7f90d41449a0 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90c9ffb640 1 -- 172.21.15.43:0/3334003201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40129b0 con 0x7f90d41449a0 2024-03-30T13:12:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d4138bb0 con 0x7f90d41449a0 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d40687f0 con 0x7f90d41449a0 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90c9ffb640 1 -- 172.21.15.43:0/3334003201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f90c4012b50 con 0x7f90d41449a0 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.835+0000 7f90c9ffb640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90b8067fc0 0x7f90b806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:12:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.839+0000 7f90c9ffb640 1 -- 172.21.15.43:0/3334003201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f90c4086a60 con 0x7f90d41449a0 2024-03-30T13:12:43.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.839+0000 7f90d9489640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90b8067fc0 0x7f90b806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:12:43.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.839+0000 7f90d9489640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90b8067fc0 0x7f90b806a480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90d0007a70 tx=0x7f90d0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:12:43.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:43.843+0000 7f90c9ffb640 1 -- 172.21.15.43:0/3334003201 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f90d41f8850 con 0x7f90d41449a0 2024-03-30T13:12:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.043+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true} v 0) -- 0x7f90d406f260 con 0x7f90d41449a0 2024-03-30T13:12:44.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.095+0000 7f90c9ffb640 1 -- 172.21.15.43:0/3334003201 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "1", "yes_i_really_really_mean_it": true}]=0 inline data enabled v19) ==== 165+0+0 (secure 0 0 0) 0x7f90c4057050 con 0x7f90d41449a0 2024-03-30T13:12:44.106 INFO:teuthology.orchestra.run.smithi043.stderr:inline data enabled 2024-03-30T13:12:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90b8067fc0 msgr2=0x7f90b806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90b8067fc0 0x7f90b806a480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f90d0007a70 tx=0x7f90d0008040 comp rx=0 tx=0).stop 2024-03-30T13:12:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d41449a0 msgr2=0x7f90d415ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:12:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d41449a0 0x7f90d415ad40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f90c400d5f0 tx=0x7f90c400dab0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 shutdown_connections 2024-03-30T13:12:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90b8067fc0 0x7f90b806a480 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90d415f220 0x7f90d41f8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90d41449a0 0x7f90d415ad40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 --2- 172.21.15.43:0/3334003201 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90d40ce860 0x7f90d4137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:12:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 >> 172.21.15.43:0/3334003201 conn(0x7f90d40b1b80 msgr2=0x7f90d4143580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:12:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 shutdown_connections 2024-03-30T13:12:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:12:44.099+0000 7f90db6ed640 1 -- 172.21.15.43:0/3334003201 wait complete. 2024-03-30T13:12:44.125 DEBUG:tasks.cephfs.test_uninlining:fetching tarball 2024-03-30T13:12:44.125 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && wget http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-30T13:12:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:--2024-03-30 13:12:44-- http://download.ceph.com/qa/linux-5.4.tar.gz 2024-03-30T13:12:44.157 INFO:teuthology.orchestra.run.smithi043.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-30T13:12:44.178 INFO:teuthology.orchestra.run.smithi043.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-30T13:12:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-30T13:12:44.198 INFO:teuthology.orchestra.run.smithi043.stderr:Length: 172616875 (165M) [application/octet-stream] 2024-03-30T13:12:44.205 INFO:teuthology.orchestra.run.smithi043.stderr:Saving to: 'linux-5.4.tar.gz' 2024-03-30T13:12:44.209 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:12:44.240 INFO:teuthology.orchestra.run.smithi043.stderr: 0K .......... .......... .......... .......... .......... 0% 1.42M 1m56s 2024-03-30T13:12:44.261 INFO:teuthology.orchestra.run.smithi043.stderr: 50K .......... .......... .......... .......... .......... 0% 2.48M 91s 2024-03-30T13:12:44.264 INFO:teuthology.orchestra.run.smithi043.stderr: 100K .......... .......... .......... .......... .......... 0% 13.4M 65s 2024-03-30T13:12:44.268 INFO:teuthology.orchestra.run.smithi043.stderr: 150K .......... .......... .......... .......... .......... 0% 12.4M 52s 2024-03-30T13:12:44.285 INFO:teuthology.orchestra.run.smithi043.stderr: 200K .......... .......... .......... .......... .......... 0% 2.90M 53s 2024-03-30T13:12:44.289 INFO:teuthology.orchestra.run.smithi043.stderr: 250K .......... .......... .......... .......... .......... 0% 12.1M 46s 2024-03-30T13:12:44.305 INFO:teuthology.orchestra.run.smithi043.stderr: 300K .......... .......... .......... .......... .......... 0% 3.05M 47s 2024-03-30T13:12:44.310 INFO:teuthology.orchestra.run.smithi043.stderr: 350K .......... .......... .......... .......... .......... 0% 11.9M 43s 2024-03-30T13:12:44.312 INFO:teuthology.orchestra.run.smithi043.stderr: 400K .......... .......... .......... .......... .......... 0% 13.4M 40s 2024-03-30T13:12:44.316 INFO:teuthology.orchestra.run.smithi043.stderr: 450K .......... .......... .......... .......... .......... 0% 14.5M 37s 2024-03-30T13:12:44.319 INFO:teuthology.orchestra.run.smithi043.stderr: 500K .......... .......... .......... .......... .......... 0% 13.6M 35s 2024-03-30T13:12:44.323 INFO:teuthology.orchestra.run.smithi043.stderr: 550K .......... .......... .......... .......... .......... 0% 12.0M 33s 2024-03-30T13:12:44.327 INFO:teuthology.orchestra.run.smithi043.stderr: 600K .......... .......... .......... .......... .......... 0% 14.1M 31s 2024-03-30T13:12:44.330 INFO:teuthology.orchestra.run.smithi043.stderr: 650K .......... .......... .......... .......... .......... 0% 14.0M 30s 2024-03-30T13:12:44.334 INFO:teuthology.orchestra.run.smithi043.stderr: 700K .......... .......... .......... .......... .......... 0% 14.1M 29s 2024-03-30T13:12:44.338 INFO:teuthology.orchestra.run.smithi043.stderr: 750K .......... .......... .......... .......... .......... 0% 11.6M 28s 2024-03-30T13:12:44.342 INFO:teuthology.orchestra.run.smithi043.stderr: 800K .......... .......... .......... .......... .......... 0% 13.4M 27s 2024-03-30T13:12:44.345 INFO:teuthology.orchestra.run.smithi043.stderr: 850K .......... .......... .......... .......... .......... 0% 12.3M 26s 2024-03-30T13:12:44.350 INFO:teuthology.orchestra.run.smithi043.stderr: 900K .......... .......... .......... .......... .......... 0% 13.0M 25s 2024-03-30T13:12:44.354 INFO:teuthology.orchestra.run.smithi043.stderr: 950K .......... .......... .......... .......... .......... 0% 10.2M 25s 2024-03-30T13:12:44.359 INFO:teuthology.orchestra.run.smithi043.stderr: 1000K .......... .......... .......... .......... .......... 0% 10.5M 24s 2024-03-30T13:12:44.364 INFO:teuthology.orchestra.run.smithi043.stderr: 1050K .......... .......... .......... .......... .......... 0% 10.2M 24s 2024-03-30T13:12:44.369 INFO:teuthology.orchestra.run.smithi043.stderr: 1100K .......... .......... .......... .......... .......... 0% 9.72M 24s 2024-03-30T13:12:44.375 INFO:teuthology.orchestra.run.smithi043.stderr: 1150K .......... .......... .......... .......... .......... 0% 7.94M 24s 2024-03-30T13:12:44.380 INFO:teuthology.orchestra.run.smithi043.stderr: 1200K .......... .......... .......... .......... .......... 0% 9.21M 23s 2024-03-30T13:12:44.386 INFO:teuthology.orchestra.run.smithi043.stderr: 1250K .......... .......... .......... .......... .......... 0% 8.90M 23s 2024-03-30T13:12:44.390 INFO:teuthology.orchestra.run.smithi043.stderr: 1300K .......... .......... .......... .......... .......... 0% 10.8M 23s 2024-03-30T13:12:44.396 INFO:teuthology.orchestra.run.smithi043.stderr: 1350K .......... .......... .......... .......... .......... 0% 8.65M 23s 2024-03-30T13:12:44.401 INFO:teuthology.orchestra.run.smithi043.stderr: 1400K .......... .......... .......... .......... .......... 0% 10.4M 22s 2024-03-30T13:12:44.406 INFO:teuthology.orchestra.run.smithi043.stderr: 1450K .......... .......... .......... .......... .......... 0% 9.54M 22s 2024-03-30T13:12:44.411 INFO:teuthology.orchestra.run.smithi043.stderr: 1500K .......... .......... .......... .......... .......... 0% 9.53M 22s 2024-03-30T13:12:44.417 INFO:teuthology.orchestra.run.smithi043.stderr: 1550K .......... .......... .......... .......... .......... 0% 7.42M 22s 2024-03-30T13:12:44.423 INFO:teuthology.orchestra.run.smithi043.stderr: 1600K .......... .......... .......... .......... .......... 0% 8.72M 22s 2024-03-30T13:12:44.429 INFO:teuthology.orchestra.run.smithi043.stderr: 1650K .......... .......... .......... .......... .......... 1% 7.86M 22s 2024-03-30T13:12:44.436 INFO:teuthology.orchestra.run.smithi043.stderr: 1700K .......... .......... .......... .......... .......... 1% 7.47M 22s 2024-03-30T13:12:44.443 INFO:teuthology.orchestra.run.smithi043.stderr: 1750K .......... .......... .......... .......... .......... 1% 6.67M 22s 2024-03-30T13:12:44.450 INFO:teuthology.orchestra.run.smithi043.stderr: 1800K .......... .......... .......... .......... .......... 1% 7.67M 22s 2024-03-30T13:12:44.456 INFO:teuthology.orchestra.run.smithi043.stderr: 1850K .......... .......... .......... .......... .......... 1% 7.29M 22s 2024-03-30T13:12:44.464 INFO:teuthology.orchestra.run.smithi043.stderr: 1900K .......... .......... .......... .......... .......... 1% 7.48M 22s 2024-03-30T13:12:44.471 INFO:teuthology.orchestra.run.smithi043.stderr: 1950K .......... .......... .......... .......... .......... 1% 6.13M 22s 2024-03-30T13:12:44.477 INFO:teuthology.orchestra.run.smithi043.stderr: 2000K .......... .......... .......... .......... .......... 1% 8.29M 22s 2024-03-30T13:12:44.482 INFO:teuthology.orchestra.run.smithi043.stderr: 2050K .......... .......... .......... .......... .......... 1% 8.75M 22s 2024-03-30T13:12:44.486 INFO:teuthology.orchestra.run.smithi043.stderr: 2100K .......... .......... .......... .......... .......... 1% 11.2M 22s 2024-03-30T13:12:44.492 INFO:teuthology.orchestra.run.smithi043.stderr: 2150K .......... .......... .......... .......... .......... 1% 9.17M 22s 2024-03-30T13:12:44.497 INFO:teuthology.orchestra.run.smithi043.stderr: 2200K .......... .......... .......... .......... .......... 1% 10.2M 21s 2024-03-30T13:12:44.502 INFO:teuthology.orchestra.run.smithi043.stderr: 2250K .......... .......... .......... .......... .......... 1% 9.97M 21s 2024-03-30T13:12:44.507 INFO:teuthology.orchestra.run.smithi043.stderr: 2300K .......... .......... .......... .......... .......... 1% 9.78M 21s 2024-03-30T13:12:44.514 INFO:teuthology.orchestra.run.smithi043.stderr: 2350K .......... .......... .......... .......... .......... 1% 7.96M 21s 2024-03-30T13:12:44.518 INFO:teuthology.orchestra.run.smithi043.stderr: 2400K .......... .......... .......... .......... .......... 1% 9.28M 21s 2024-03-30T13:12:44.523 INFO:teuthology.orchestra.run.smithi043.stderr: 2450K .......... .......... .......... .......... .......... 1% 9.72M 21s 2024-03-30T13:12:44.528 INFO:teuthology.orchestra.run.smithi043.stderr: 2500K .......... .......... .......... .......... .......... 1% 9.02M 21s 2024-03-30T13:12:44.535 INFO:teuthology.orchestra.run.smithi043.stderr: 2550K .......... .......... .......... .......... .......... 1% 7.98M 21s 2024-03-30T13:12:44.540 INFO:teuthology.orchestra.run.smithi043.stderr: 2600K .......... .......... .......... .......... .......... 1% 9.29M 21s 2024-03-30T13:12:44.545 INFO:teuthology.orchestra.run.smithi043.stderr: 2650K .......... .......... .......... .......... .......... 1% 9.29M 21s 2024-03-30T13:12:44.551 INFO:teuthology.orchestra.run.smithi043.stderr: 2700K .......... .......... .......... .......... .......... 1% 8.21M 21s 2024-03-30T13:12:44.557 INFO:teuthology.orchestra.run.smithi043.stderr: 2750K .......... .......... .......... .......... .......... 1% 8.13M 21s 2024-03-30T13:12:44.563 INFO:teuthology.orchestra.run.smithi043.stderr: 2800K .......... .......... .......... .......... .......... 1% 8.89M 21s 2024-03-30T13:12:44.569 INFO:teuthology.orchestra.run.smithi043.stderr: 2850K .......... .......... .......... .......... .......... 1% 9.25M 21s 2024-03-30T13:12:44.574 INFO:teuthology.orchestra.run.smithi043.stderr: 2900K .......... .......... .......... .......... .......... 1% 8.37M 21s 2024-03-30T13:12:44.580 INFO:teuthology.orchestra.run.smithi043.stderr: 2950K .......... .......... .......... .......... .......... 1% 8.35M 21s 2024-03-30T13:12:44.584 INFO:teuthology.orchestra.run.smithi043.stderr: 3000K .......... .......... .......... .......... .......... 1% 10.2M 21s 2024-03-30T13:12:44.589 INFO:teuthology.orchestra.run.smithi043.stderr: 3050K .......... .......... .......... .......... .......... 1% 9.96M 20s 2024-03-30T13:12:44.595 INFO:teuthology.orchestra.run.smithi043.stderr: 3100K .......... .......... .......... .......... .......... 1% 9.87M 20s 2024-03-30T13:12:44.600 INFO:teuthology.orchestra.run.smithi043.stderr: 3150K .......... .......... .......... .......... .......... 1% 7.93M 20s 2024-03-30T13:12:44.605 INFO:teuthology.orchestra.run.smithi043.stderr: 3200K .......... .......... .......... .......... .......... 1% 9.57M 20s 2024-03-30T13:12:44.610 INFO:teuthology.orchestra.run.smithi043.stderr: 3250K .......... .......... .......... .......... .......... 1% 9.72M 20s 2024-03-30T13:12:44.616 INFO:teuthology.orchestra.run.smithi043.stderr: 3300K .......... .......... .......... .......... .......... 1% 9.19M 20s 2024-03-30T13:12:44.622 INFO:teuthology.orchestra.run.smithi043.stderr: 3350K .......... .......... .......... .......... .......... 2% 7.73M 20s 2024-03-30T13:12:44.628 INFO:teuthology.orchestra.run.smithi043.stderr: 3400K .......... .......... .......... .......... .......... 2% 8.52M 20s 2024-03-30T13:12:44.634 INFO:teuthology.orchestra.run.smithi043.stderr: 3450K .......... .......... .......... .......... .......... 2% 8.41M 20s 2024-03-30T13:12:44.640 INFO:teuthology.orchestra.run.smithi043.stderr: 3500K .......... .......... .......... .......... .......... 2% 8.22M 20s 2024-03-30T13:12:44.646 INFO:teuthology.orchestra.run.smithi043.stderr: 3550K .......... .......... .......... .......... .......... 2% 7.15M 20s 2024-03-30T13:12:44.652 INFO:teuthology.orchestra.run.smithi043.stderr: 3600K .......... .......... .......... .......... .......... 2% 8.08M 20s 2024-03-30T13:12:44.659 INFO:teuthology.orchestra.run.smithi043.stderr: 3650K .......... .......... .......... .......... .......... 2% 7.80M 20s 2024-03-30T13:12:44.665 INFO:teuthology.orchestra.run.smithi043.stderr: 3700K .......... .......... .......... .......... .......... 2% 7.55M 20s 2024-03-30T13:12:44.673 INFO:teuthology.orchestra.run.smithi043.stderr: 3750K .......... .......... .......... .......... .......... 2% 6.68M 20s 2024-03-30T13:12:44.678 INFO:teuthology.orchestra.run.smithi043.stderr: 3800K .......... .......... .......... .......... .......... 2% 8.25M 20s 2024-03-30T13:12:44.684 INFO:teuthology.orchestra.run.smithi043.stderr: 3850K .......... .......... .......... .......... .......... 2% 8.23M 20s 2024-03-30T13:12:44.690 INFO:teuthology.orchestra.run.smithi043.stderr: 3900K .......... .......... .......... .......... .......... 2% 8.56M 20s 2024-03-30T13:12:44.697 INFO:teuthology.orchestra.run.smithi043.stderr: 3950K .......... .......... .......... .......... .......... 2% 7.10M 20s 2024-03-30T13:12:44.703 INFO:teuthology.orchestra.run.smithi043.stderr: 4000K .......... .......... .......... .......... .......... 2% 8.16M 20s 2024-03-30T13:12:44.709 INFO:teuthology.orchestra.run.smithi043.stderr: 4050K .......... .......... .......... .......... .......... 2% 7.98M 20s 2024-03-30T13:12:44.713 INFO:teuthology.orchestra.run.smithi043.stderr: 4100K .......... .......... .......... .......... .......... 2% 11.7M 20s 2024-03-30T13:12:44.717 INFO:teuthology.orchestra.run.smithi043.stderr: 4150K .......... .......... .......... .......... .......... 2% 10.7M 20s 2024-03-30T13:12:44.721 INFO:teuthology.orchestra.run.smithi043.stderr: 4200K .......... .......... .......... .......... .......... 2% 12.3M 20s 2024-03-30T13:12:44.726 INFO:teuthology.orchestra.run.smithi043.stderr: 4250K .......... .......... .......... .......... .......... 2% 11.8M 20s 2024-03-30T13:12:44.730 INFO:teuthology.orchestra.run.smithi043.stderr: 4300K .......... .......... .......... .......... .......... 2% 11.3M 20s 2024-03-30T13:12:44.735 INFO:teuthology.orchestra.run.smithi043.stderr: 4350K .......... .......... .......... .......... .......... 2% 9.93M 20s 2024-03-30T13:12:44.738 INFO:teuthology.orchestra.run.smithi043.stderr: 4400K .......... .......... .......... .......... .......... 2% 13.3M 20s 2024-03-30T13:12:44.742 INFO:teuthology.orchestra.run.smithi043.stderr: 4450K .......... .......... .......... .......... .......... 2% 12.8M 20s 2024-03-30T13:12:44.746 INFO:teuthology.orchestra.run.smithi043.stderr: 4500K .......... .......... .......... .......... .......... 2% 11.9M 19s 2024-03-30T13:12:44.751 INFO:teuthology.orchestra.run.smithi043.stderr: 4550K .......... .......... .......... .......... .......... 2% 9.66M 19s 2024-03-30T13:12:44.756 INFO:teuthology.orchestra.run.smithi043.stderr: 4600K .......... .......... .......... .......... .......... 2% 11.8M 19s 2024-03-30T13:12:44.760 INFO:teuthology.orchestra.run.smithi043.stderr: 4650K .......... .......... .......... .......... .......... 2% 11.4M 19s 2024-03-30T13:12:44.764 INFO:teuthology.orchestra.run.smithi043.stderr: 4700K .......... .......... .......... .......... .......... 2% 11.5M 19s 2024-03-30T13:12:44.769 INFO:teuthology.orchestra.run.smithi043.stderr: 4750K .......... .......... .......... .......... .......... 2% 9.64M 19s 2024-03-30T13:12:44.773 INFO:teuthology.orchestra.run.smithi043.stderr: 4800K .......... .......... .......... .......... .......... 2% 11.4M 19s 2024-03-30T13:12:44.778 INFO:teuthology.orchestra.run.smithi043.stderr: 4850K .......... .......... .......... .......... .......... 2% 10.9M 19s 2024-03-30T13:12:44.782 INFO:teuthology.orchestra.run.smithi043.stderr: 4900K .......... .......... .......... .......... .......... 2% 11.4M 19s 2024-03-30T13:12:44.787 INFO:teuthology.orchestra.run.smithi043.stderr: 4950K .......... .......... .......... .......... .......... 2% 9.33M 19s 2024-03-30T13:12:44.792 INFO:teuthology.orchestra.run.smithi043.stderr: 5000K .......... .......... .......... .......... .......... 2% 11.4M 19s 2024-03-30T13:12:44.796 INFO:teuthology.orchestra.run.smithi043.stderr: 5050K .......... .......... .......... .......... .......... 3% 10.8M 19s 2024-03-30T13:12:44.801 INFO:teuthology.orchestra.run.smithi043.stderr: 5100K .......... .......... .......... .......... .......... 3% 11.4M 19s 2024-03-30T13:12:44.806 INFO:teuthology.orchestra.run.smithi043.stderr: 5150K .......... .......... .......... .......... .......... 3% 8.52M 19s 2024-03-30T13:12:44.811 INFO:teuthology.orchestra.run.smithi043.stderr: 5200K .......... .......... .......... .......... .......... 3% 11.1M 19s 2024-03-30T13:12:44.816 INFO:teuthology.orchestra.run.smithi043.stderr: 5250K .......... .......... .......... .......... .......... 3% 10.4M 19s 2024-03-30T13:12:44.820 INFO:teuthology.orchestra.run.smithi043.stderr: 5300K .......... .......... .......... .......... .......... 3% 11.3M 19s 2024-03-30T13:12:44.825 INFO:teuthology.orchestra.run.smithi043.stderr: 5350K .......... .......... .......... .......... .......... 3% 9.59M 19s 2024-03-30T13:12:44.829 INFO:teuthology.orchestra.run.smithi043.stderr: 5400K .......... .......... .......... .......... .......... 3% 11.5M 19s 2024-03-30T13:12:44.833 INFO:teuthology.orchestra.run.smithi043.stderr: 5450K .......... .......... .......... .......... .......... 3% 11.0M 19s 2024-03-30T13:12:44.838 INFO:teuthology.orchestra.run.smithi043.stderr: 5500K .......... .......... .......... .......... .......... 3% 11.2M 19s 2024-03-30T13:12:44.843 INFO:teuthology.orchestra.run.smithi043.stderr: 5550K .......... .......... .......... .......... .......... 3% 9.34M 19s 2024-03-30T13:12:44.848 INFO:teuthology.orchestra.run.smithi043.stderr: 5600K .......... .......... .......... .......... .......... 3% 10.8M 18s 2024-03-30T13:12:44.852 INFO:teuthology.orchestra.run.smithi043.stderr: 5650K .......... .......... .......... .......... .......... 3% 11.1M 18s 2024-03-30T13:12:44.856 INFO:teuthology.orchestra.run.smithi043.stderr: 5700K .......... .......... .......... .......... .......... 3% 11.1M 18s 2024-03-30T13:12:44.862 INFO:teuthology.orchestra.run.smithi043.stderr: 5750K .......... .......... .......... .......... .......... 3% 9.36M 18s 2024-03-30T13:12:44.866 INFO:teuthology.orchestra.run.smithi043.stderr: 5800K .......... .......... .......... .......... .......... 3% 10.9M 18s 2024-03-30T13:12:44.871 INFO:teuthology.orchestra.run.smithi043.stderr: 5850K .......... .......... .......... .......... .......... 3% 10.0M 18s 2024-03-30T13:12:44.876 INFO:teuthology.orchestra.run.smithi043.stderr: 5900K .......... .......... .......... .......... .......... 3% 10.8M 18s 2024-03-30T13:12:44.881 INFO:teuthology.orchestra.run.smithi043.stderr: 5950K .......... .......... .......... .......... .......... 3% 8.88M 18s 2024-03-30T13:12:44.886 INFO:teuthology.orchestra.run.smithi043.stderr: 6000K .......... .......... .......... .......... .......... 3% 10.5M 18s 2024-03-30T13:12:44.890 INFO:teuthology.orchestra.run.smithi043.stderr: 6050K .......... .......... .......... .......... .......... 3% 10.5M 18s 2024-03-30T13:12:44.895 INFO:teuthology.orchestra.run.smithi043.stderr: 6100K .......... .......... .......... .......... .......... 3% 10.8M 18s 2024-03-30T13:12:44.901 INFO:teuthology.orchestra.run.smithi043.stderr: 6150K .......... .......... .......... .......... .......... 3% 8.81M 18s 2024-03-30T13:12:44.905 INFO:teuthology.orchestra.run.smithi043.stderr: 6200K .......... .......... .......... .......... .......... 3% 9.77M 18s 2024-03-30T13:12:44.910 INFO:teuthology.orchestra.run.smithi043.stderr: 6250K .......... .......... .......... .......... .......... 3% 10.8M 18s 2024-03-30T13:12:44.915 INFO:teuthology.orchestra.run.smithi043.stderr: 6300K .......... .......... .......... .......... .......... 3% 10.2M 18s 2024-03-30T13:12:44.922 INFO:teuthology.orchestra.run.smithi043.stderr: 6350K .......... .......... .......... .......... .......... 3% 8.57M 18s 2024-03-30T13:12:44.925 INFO:teuthology.orchestra.run.smithi043.stderr: 6400K .......... .......... .......... .......... .......... 3% 10.1M 18s 2024-03-30T13:12:44.930 INFO:teuthology.orchestra.run.smithi043.stderr: 6450K .......... .......... .......... .......... .......... 3% 9.80M 18s 2024-03-30T13:12:44.935 INFO:teuthology.orchestra.run.smithi043.stderr: 6500K .......... .......... .......... .......... .......... 3% 10.5M 18s 2024-03-30T13:12:44.941 INFO:teuthology.orchestra.run.smithi043.stderr: 6550K .......... .......... .......... .......... .......... 3% 8.36M 18s 2024-03-30T13:12:44.946 INFO:teuthology.orchestra.run.smithi043.stderr: 6600K .......... .......... .......... .......... .......... 3% 10.3M 18s 2024-03-30T13:12:44.950 INFO:teuthology.orchestra.run.smithi043.stderr: 6650K .......... .......... .......... .......... .......... 3% 10.1M 18s 2024-03-30T13:12:44.955 INFO:teuthology.orchestra.run.smithi043.stderr: 6700K .......... .......... .......... .......... .......... 4% 10.2M 18s 2024-03-30T13:12:44.961 INFO:teuthology.orchestra.run.smithi043.stderr: 6750K .......... .......... .......... .......... .......... 4% 8.73M 18s 2024-03-30T13:12:44.966 INFO:teuthology.orchestra.run.smithi043.stderr: 6800K .......... .......... .......... .......... .......... 4% 10.1M 18s 2024-03-30T13:12:44.971 INFO:teuthology.orchestra.run.smithi043.stderr: 6850K .......... .......... .......... .......... .......... 4% 9.81M 18s 2024-03-30T13:12:44.976 INFO:teuthology.orchestra.run.smithi043.stderr: 6900K .......... .......... .......... .......... .......... 4% 9.88M 18s 2024-03-30T13:12:44.981 INFO:teuthology.orchestra.run.smithi043.stderr: 6950K .......... .......... .......... .......... .......... 4% 8.41M 18s 2024-03-30T13:12:44.986 INFO:teuthology.orchestra.run.smithi043.stderr: 7000K .......... .......... .......... .......... .......... 4% 9.60M 18s 2024-03-30T13:12:44.992 INFO:teuthology.orchestra.run.smithi043.stderr: 7050K .......... .......... .......... .......... .......... 4% 9.23M 18s 2024-03-30T13:12:44.997 INFO:teuthology.orchestra.run.smithi043.stderr: 7100K .......... .......... .......... .......... .......... 4% 9.73M 18s 2024-03-30T13:12:45.002 INFO:teuthology.orchestra.run.smithi043.stderr: 7150K .......... .......... .......... .......... .......... 4% 8.45M 18s 2024-03-30T13:12:45.008 INFO:teuthology.orchestra.run.smithi043.stderr: 7200K .......... .......... .......... .......... .......... 4% 9.28M 18s 2024-03-30T13:12:45.013 INFO:teuthology.orchestra.run.smithi043.stderr: 7250K .......... .......... .......... .......... .......... 4% 9.58M 18s 2024-03-30T13:12:45.018 INFO:teuthology.orchestra.run.smithi043.stderr: 7300K .......... .......... .......... .......... .......... 4% 9.50M 18s 2024-03-30T13:12:45.024 INFO:teuthology.orchestra.run.smithi043.stderr: 7350K .......... .......... .......... .......... .......... 4% 7.81M 18s 2024-03-30T13:12:45.030 INFO:teuthology.orchestra.run.smithi043.stderr: 7400K .......... .......... .......... .......... .......... 4% 9.42M 18s 2024-03-30T13:12:45.035 INFO:teuthology.orchestra.run.smithi043.stderr: 7450K .......... .......... .......... .......... .......... 4% 9.00M 18s 2024-03-30T13:12:45.041 INFO:teuthology.orchestra.run.smithi043.stderr: 7500K .......... .......... .......... .......... .......... 4% 8.65M 18s 2024-03-30T13:12:45.047 INFO:teuthology.orchestra.run.smithi043.stderr: 7550K .......... .......... .......... .......... .......... 4% 7.54M 18s 2024-03-30T13:12:45.053 INFO:teuthology.orchestra.run.smithi043.stderr: 7600K .......... .......... .......... .......... .......... 4% 8.77M 18s 2024-03-30T13:12:45.058 INFO:teuthology.orchestra.run.smithi043.stderr: 7650K .......... .......... .......... .......... .......... 4% 8.76M 18s 2024-03-30T13:12:45.064 INFO:teuthology.orchestra.run.smithi043.stderr: 7700K .......... .......... .......... .......... .......... 4% 8.23M 18s 2024-03-30T13:12:45.071 INFO:teuthology.orchestra.run.smithi043.stderr: 7750K .......... .......... .......... .......... .......... 4% 7.31M 18s 2024-03-30T13:12:45.077 INFO:teuthology.orchestra.run.smithi043.stderr: 7800K .......... .......... .......... .......... .......... 4% 8.04M 18s 2024-03-30T13:12:45.083 INFO:teuthology.orchestra.run.smithi043.stderr: 7850K .......... .......... .......... .......... .......... 4% 8.47M 18s 2024-03-30T13:12:45.089 INFO:teuthology.orchestra.run.smithi043.stderr: 7900K .......... .......... .......... .......... .......... 4% 7.71M 18s 2024-03-30T13:12:45.096 INFO:teuthology.orchestra.run.smithi043.stderr: 7950K .......... .......... .......... .......... .......... 4% 6.68M 18s 2024-03-30T13:12:45.103 INFO:teuthology.orchestra.run.smithi043.stderr: 8000K .......... .......... .......... .......... .......... 4% 7.76M 18s 2024-03-30T13:12:45.109 INFO:teuthology.orchestra.run.smithi043.stderr: 8050K .......... .......... .......... .......... .......... 4% 7.98M 18s 2024-03-30T13:12:45.115 INFO:teuthology.orchestra.run.smithi043.stderr: 8100K .......... .......... .......... .......... .......... 4% 8.33M 18s 2024-03-30T13:12:45.122 INFO:teuthology.orchestra.run.smithi043.stderr: 8150K .......... .......... .......... .......... .......... 4% 6.93M 18s 2024-03-30T13:12:45.127 INFO:teuthology.orchestra.run.smithi043.stderr: 8200K .......... .......... .......... .......... .......... 4% 8.90M 18s 2024-03-30T13:12:45.132 INFO:teuthology.orchestra.run.smithi043.stderr: 8250K .......... .......... .......... .......... .......... 4% 10.5M 18s 2024-03-30T13:12:45.135 INFO:teuthology.orchestra.run.smithi043.stderr: 8300K .......... .......... .......... .......... .......... 4% 13.5M 18s 2024-03-30T13:12:45.139 INFO:teuthology.orchestra.run.smithi043.stderr: 8350K .......... .......... .......... .......... .......... 4% 11.6M 18s 2024-03-30T13:12:45.143 INFO:teuthology.orchestra.run.smithi043.stderr: 8400K .......... .......... .......... .......... .......... 5% 13.2M 18s 2024-03-30T13:12:45.148 INFO:teuthology.orchestra.run.smithi043.stderr: 8450K .......... .......... .......... .......... .......... 5% 11.4M 18s 2024-03-30T13:12:45.152 INFO:teuthology.orchestra.run.smithi043.stderr: 8500K .......... .......... .......... .......... .......... 5% 9.81M 18s 2024-03-30T13:12:45.158 INFO:teuthology.orchestra.run.smithi043.stderr: 8550K .......... .......... .......... .......... .......... 5% 8.67M 18s 2024-03-30T13:12:45.163 INFO:teuthology.orchestra.run.smithi043.stderr: 8600K .......... .......... .......... .......... .......... 5% 10.2M 18s 2024-03-30T13:12:45.167 INFO:teuthology.orchestra.run.smithi043.stderr: 8650K .......... .......... .......... .......... .......... 5% 10.7M 18s 2024-03-30T13:12:45.172 INFO:teuthology.orchestra.run.smithi043.stderr: 8700K .......... .......... .......... .......... .......... 5% 10.4M 18s 2024-03-30T13:12:45.177 INFO:teuthology.orchestra.run.smithi043.stderr: 8750K .......... .......... .......... .......... .......... 5% 9.60M 18s 2024-03-30T13:12:45.181 INFO:teuthology.orchestra.run.smithi043.stderr: 8800K .......... .......... .......... .......... .......... 5% 12.5M 18s 2024-03-30T13:12:45.185 INFO:teuthology.orchestra.run.smithi043.stderr: 8850K .......... .......... .......... .......... .......... 5% 12.9M 18s 2024-03-30T13:12:45.189 INFO:teuthology.orchestra.run.smithi043.stderr: 8900K .......... .......... .......... .......... .......... 5% 12.4M 18s 2024-03-30T13:12:45.193 INFO:teuthology.orchestra.run.smithi043.stderr: 8950K .......... .......... .......... .......... .......... 5% 11.0M 18s 2024-03-30T13:12:45.199 INFO:teuthology.orchestra.run.smithi043.stderr: 9000K .......... .......... .......... .......... .......... 5% 9.23M 17s 2024-03-30T13:12:45.204 INFO:teuthology.orchestra.run.smithi043.stderr: 9050K .......... .......... .......... .......... .......... 5% 9.75M 17s 2024-03-30T13:12:45.209 INFO:teuthology.orchestra.run.smithi043.stderr: 9100K .......... .......... .......... .......... .......... 5% 9.44M 17s 2024-03-30T13:12:45.214 INFO:teuthology.orchestra.run.smithi043.stderr: 9150K .......... .......... .......... .......... .......... 5% 8.70M 17s 2024-03-30T13:12:45.220 INFO:teuthology.orchestra.run.smithi043.stderr: 9200K .......... .......... .......... .......... .......... 5% 9.42M 17s 2024-03-30T13:12:45.224 INFO:teuthology.orchestra.run.smithi043.stderr: 9250K .......... .......... .......... .......... .......... 5% 10.3M 17s 2024-03-30T13:12:45.229 INFO:teuthology.orchestra.run.smithi043.stderr: 9300K .......... .......... .......... .......... .......... 5% 9.90M 17s 2024-03-30T13:12:45.235 INFO:teuthology.orchestra.run.smithi043.stderr: 9350K .......... .......... .......... .......... .......... 5% 8.54M 17s 2024-03-30T13:12:45.240 INFO:teuthology.orchestra.run.smithi043.stderr: 9400K .......... .......... .......... .......... .......... 5% 10.1M 17s 2024-03-30T13:12:45.244 INFO:teuthology.orchestra.run.smithi043.stderr: 9450K .......... .......... .......... .......... .......... 5% 10.3M 17s 2024-03-30T13:12:45.250 INFO:teuthology.orchestra.run.smithi043.stderr: 9500K .......... .......... .......... .......... .......... 5% 9.18M 17s 2024-03-30T13:12:45.255 INFO:teuthology.orchestra.run.smithi043.stderr: 9550K .......... .......... .......... .......... .......... 5% 8.37M 17s 2024-03-30T13:12:45.260 INFO:teuthology.orchestra.run.smithi043.stderr: 9600K .......... .......... .......... .......... .......... 5% 11.7M 17s 2024-03-30T13:12:45.264 INFO:teuthology.orchestra.run.smithi043.stderr: 9650K .......... .......... .......... .......... .......... 5% 10.9M 17s 2024-03-30T13:12:45.270 INFO:teuthology.orchestra.run.smithi043.stderr: 9700K .......... .......... .......... .......... .......... 5% 8.58M 17s 2024-03-30T13:12:45.276 INFO:teuthology.orchestra.run.smithi043.stderr: 9750K .......... .......... .......... .......... .......... 5% 8.38M 17s 2024-03-30T13:12:45.281 INFO:teuthology.orchestra.run.smithi043.stderr: 9800K .......... .......... .......... .......... .......... 5% 9.70M 17s 2024-03-30T13:12:45.286 INFO:teuthology.orchestra.run.smithi043.stderr: 9850K .......... .......... .......... .......... .......... 5% 9.43M 17s 2024-03-30T13:12:45.292 INFO:teuthology.orchestra.run.smithi043.stderr: 9900K .......... .......... .......... .......... .......... 5% 9.38M 17s 2024-03-30T13:12:45.297 INFO:teuthology.orchestra.run.smithi043.stderr: 9950K .......... .......... .......... .......... .......... 5% 8.17M 17s 2024-03-30T13:12:45.303 INFO:teuthology.orchestra.run.smithi043.stderr: 10000K .......... .......... .......... .......... .......... 5% 9.33M 17s 2024-03-30T13:12:45.308 INFO:teuthology.orchestra.run.smithi043.stderr: 10050K .......... .......... .......... .......... .......... 5% 9.02M 17s 2024-03-30T13:12:45.313 INFO:teuthology.orchestra.run.smithi043.stderr: 10100K .......... .......... .......... .......... .......... 6% 9.29M 17s 2024-03-30T13:12:45.320 INFO:teuthology.orchestra.run.smithi043.stderr: 10150K .......... .......... .......... .......... .......... 6% 7.83M 17s 2024-03-30T13:12:45.325 INFO:teuthology.orchestra.run.smithi043.stderr: 10200K .......... .......... .......... .......... .......... 6% 9.21M 17s 2024-03-30T13:12:45.330 INFO:teuthology.orchestra.run.smithi043.stderr: 10250K .......... .......... .......... .......... .......... 6% 8.92M 17s 2024-03-30T13:12:45.336 INFO:teuthology.orchestra.run.smithi043.stderr: 10300K .......... .......... .......... .......... .......... 6% 8.91M 17s 2024-03-30T13:12:45.342 INFO:teuthology.orchestra.run.smithi043.stderr: 10350K .......... .......... .......... .......... .......... 6% 7.48M 17s 2024-03-30T13:12:45.348 INFO:teuthology.orchestra.run.smithi043.stderr: 10400K .......... .......... .......... .......... .......... 6% 8.69M 17s 2024-03-30T13:12:45.353 INFO:teuthology.orchestra.run.smithi043.stderr: 10450K .......... .......... .......... .......... .......... 6% 9.06M 17s 2024-03-30T13:12:45.359 INFO:teuthology.orchestra.run.smithi043.stderr: 10500K .......... .......... .......... .......... .......... 6% 8.53M 17s 2024-03-30T13:12:45.365 INFO:teuthology.orchestra.run.smithi043.stderr: 10550K .......... .......... .......... .......... .......... 6% 7.63M 17s 2024-03-30T13:12:45.371 INFO:teuthology.orchestra.run.smithi043.stderr: 10600K .......... .......... .......... .......... .......... 6% 9.10M 17s 2024-03-30T13:12:45.376 INFO:teuthology.orchestra.run.smithi043.stderr: 10650K .......... .......... .......... .......... .......... 6% 9.25M 17s 2024-03-30T13:12:45.381 INFO:teuthology.orchestra.run.smithi043.stderr: 10700K .......... .......... .......... .......... .......... 6% 9.07M 17s 2024-03-30T13:12:45.388 INFO:teuthology.orchestra.run.smithi043.stderr: 10750K .......... .......... .......... .......... .......... 6% 7.70M 17s 2024-03-30T13:12:45.393 INFO:teuthology.orchestra.run.smithi043.stderr: 10800K .......... .......... .......... .......... .......... 6% 9.22M 17s 2024-03-30T13:12:45.398 INFO:teuthology.orchestra.run.smithi043.stderr: 10850K .......... .......... .......... .......... .......... 6% 9.08M 17s 2024-03-30T13:12:45.404 INFO:teuthology.orchestra.run.smithi043.stderr: 10900K .......... .......... .......... .......... .......... 6% 9.06M 17s 2024-03-30T13:12:45.410 INFO:teuthology.orchestra.run.smithi043.stderr: 10950K .......... .......... .......... .......... .......... 6% 7.69M 17s 2024-03-30T13:12:45.416 INFO:teuthology.orchestra.run.smithi043.stderr: 11000K .......... .......... .......... .......... .......... 6% 8.93M 17s 2024-03-30T13:12:45.422 INFO:teuthology.orchestra.run.smithi043.stderr: 11050K .......... .......... .......... .......... .......... 6% 8.06M 17s 2024-03-30T13:12:45.428 INFO:teuthology.orchestra.run.smithi043.stderr: 11100K .......... .......... .......... .......... .......... 6% 7.38M 17s 2024-03-30T13:12:45.436 INFO:teuthology.orchestra.run.smithi043.stderr: 11150K .......... .......... .......... .......... .......... 6% 6.36M 17s 2024-03-30T13:12:45.443 INFO:teuthology.orchestra.run.smithi043.stderr: 11200K .......... .......... .......... .......... .......... 6% 7.53M 17s 2024-03-30T13:12:45.449 INFO:teuthology.orchestra.run.smithi043.stderr: 11250K .......... .......... .......... .......... .......... 6% 7.16M 17s 2024-03-30T13:12:45.456 INFO:teuthology.orchestra.run.smithi043.stderr: 11300K .......... .......... .......... .......... .......... 6% 7.55M 17s 2024-03-30T13:12:45.464 INFO:teuthology.orchestra.run.smithi043.stderr: 11350K .......... .......... .......... .......... .......... 6% 6.30M 17s 2024-03-30T13:12:45.470 INFO:teuthology.orchestra.run.smithi043.stderr: 11400K .......... .......... .......... .......... .......... 6% 7.44M 17s 2024-03-30T13:12:45.477 INFO:teuthology.orchestra.run.smithi043.stderr: 11450K .......... .......... .......... .......... .......... 6% 7.30M 17s 2024-03-30T13:12:45.484 INFO:teuthology.orchestra.run.smithi043.stderr: 11500K .......... .......... .......... .......... .......... 6% 7.28M 17s 2024-03-30T13:12:45.491 INFO:teuthology.orchestra.run.smithi043.stderr: 11550K .......... .......... .......... .......... .......... 6% 6.32M 17s 2024-03-30T13:12:45.498 INFO:teuthology.orchestra.run.smithi043.stderr: 11600K .......... .......... .......... .......... .......... 6% 7.03M 17s 2024-03-30T13:12:45.505 INFO:teuthology.orchestra.run.smithi043.stderr: 11650K .......... .......... .......... .......... .......... 6% 7.24M 17s 2024-03-30T13:12:45.512 INFO:teuthology.orchestra.run.smithi043.stderr: 11700K .......... .......... .......... .......... .......... 6% 7.15M 17s 2024-03-30T13:12:45.520 INFO:teuthology.orchestra.run.smithi043.stderr: 11750K .......... .......... .......... .......... .......... 7% 6.18M 17s 2024-03-30T13:12:45.527 INFO:teuthology.orchestra.run.smithi043.stderr: 11800K .......... .......... .......... .......... .......... 7% 7.29M 17s 2024-03-30T13:12:45.533 INFO:teuthology.orchestra.run.smithi043.stderr: 11850K .......... .......... .......... .......... .......... 7% 7.32M 17s 2024-03-30T13:12:45.539 INFO:teuthology.orchestra.run.smithi043.stderr: 11900K .......... .......... .......... .......... .......... 7% 7.67M 17s 2024-03-30T13:12:45.547 INFO:teuthology.orchestra.run.smithi043.stderr: 11950K .......... .......... .......... .......... .......... 7% 6.94M 17s 2024-03-30T13:12:45.553 INFO:teuthology.orchestra.run.smithi043.stderr: 12000K .......... .......... .......... .......... .......... 7% 7.61M 17s 2024-03-30T13:12:45.560 INFO:teuthology.orchestra.run.smithi043.stderr: 12050K .......... .......... .......... .......... .......... 7% 7.10M 18s 2024-03-30T13:12:45.566 INFO:teuthology.orchestra.run.smithi043.stderr: 12100K .......... .......... .......... .......... .......... 7% 7.38M 18s 2024-03-30T13:12:45.574 INFO:teuthology.orchestra.run.smithi043.stderr: 12150K .......... .......... .......... .......... .......... 7% 6.65M 18s 2024-03-30T13:12:45.580 INFO:teuthology.orchestra.run.smithi043.stderr: 12200K .......... .......... .......... .......... .......... 7% 7.45M 18s 2024-03-30T13:12:45.587 INFO:teuthology.orchestra.run.smithi043.stderr: 12250K .......... .......... .......... .......... .......... 7% 7.82M 18s 2024-03-30T13:12:45.593 INFO:teuthology.orchestra.run.smithi043.stderr: 12300K .......... .......... .......... .......... .......... 7% 7.72M 18s 2024-03-30T13:12:45.598 INFO:teuthology.orchestra.run.smithi043.stderr: 12350K .......... .......... .......... .......... .......... 7% 9.04M 18s 2024-03-30T13:12:45.603 INFO:teuthology.orchestra.run.smithi043.stderr: 12400K .......... .......... .......... .......... .......... 7% 10.9M 18s 2024-03-30T13:12:45.607 INFO:teuthology.orchestra.run.smithi043.stderr: 12450K .......... .......... .......... .......... .......... 7% 10.9M 17s 2024-03-30T13:12:45.611 INFO:teuthology.orchestra.run.smithi043.stderr: 12500K .......... .......... .......... .......... .......... 7% 11.0M 17s 2024-03-30T13:12:45.617 INFO:teuthology.orchestra.run.smithi043.stderr: 12550K .......... .......... .......... .......... .......... 7% 9.80M 17s 2024-03-30T13:12:45.621 INFO:teuthology.orchestra.run.smithi043.stderr: 12600K .......... .......... .......... .......... .......... 7% 10.6M 17s 2024-03-30T13:12:45.626 INFO:teuthology.orchestra.run.smithi043.stderr: 12650K .......... .......... .......... .......... .......... 7% 9.92M 17s 2024-03-30T13:12:45.630 INFO:teuthology.orchestra.run.smithi043.stderr: 12700K .......... .......... .......... .......... .......... 7% 10.9M 17s 2024-03-30T13:12:45.636 INFO:teuthology.orchestra.run.smithi043.stderr: 12750K .......... .......... .......... .......... .......... 7% 9.61M 17s 2024-03-30T13:12:45.640 INFO:teuthology.orchestra.run.smithi043.stderr: 12800K .......... .......... .......... .......... .......... 7% 11.2M 17s 2024-03-30T13:12:45.644 INFO:teuthology.orchestra.run.smithi043.stderr: 12850K .......... .......... .......... .......... .......... 7% 11.3M 17s 2024-03-30T13:12:45.649 INFO:teuthology.orchestra.run.smithi043.stderr: 12900K .......... .......... .......... .......... .......... 7% 11.1M 17s 2024-03-30T13:12:45.654 INFO:teuthology.orchestra.run.smithi043.stderr: 12950K .......... .......... .......... .......... .......... 7% 9.36M 17s 2024-03-30T13:12:45.659 INFO:teuthology.orchestra.run.smithi043.stderr: 13000K .......... .......... .......... .......... .......... 7% 10.3M 17s 2024-03-30T13:12:45.663 INFO:teuthology.orchestra.run.smithi043.stderr: 13050K .......... .......... .......... .......... .......... 7% 11.1M 17s 2024-03-30T13:12:45.667 INFO:teuthology.orchestra.run.smithi043.stderr: 13100K .......... .......... .......... .......... .......... 7% 12.0M 17s 2024-03-30T13:12:45.672 INFO:teuthology.orchestra.run.smithi043.stderr: 13150K .......... .......... .......... .......... .......... 7% 10.4M 17s 2024-03-30T13:12:45.676 INFO:teuthology.orchestra.run.smithi043.stderr: 13200K .......... .......... .......... .......... .......... 7% 12.0M 17s 2024-03-30T13:12:45.680 INFO:teuthology.orchestra.run.smithi043.stderr: 13250K .......... .......... .......... .......... .......... 7% 11.5M 17s 2024-03-30T13:12:45.684 INFO:teuthology.orchestra.run.smithi043.stderr: 13300K .......... .......... .......... .......... .......... 7% 11.3M 17s 2024-03-30T13:12:45.689 INFO:teuthology.orchestra.run.smithi043.stderr: 13350K .......... .......... .......... .......... .......... 7% 10.1M 17s 2024-03-30T13:12:45.693 INFO:teuthology.orchestra.run.smithi043.stderr: 13400K .......... .......... .......... .......... .......... 7% 11.5M 17s 2024-03-30T13:12:45.698 INFO:teuthology.orchestra.run.smithi043.stderr: 13450K .......... .......... .......... .......... .......... 8% 11.2M 17s 2024-03-30T13:12:45.702 INFO:teuthology.orchestra.run.smithi043.stderr: 13500K .......... .......... .......... .......... .......... 8% 11.2M 17s 2024-03-30T13:12:45.707 INFO:teuthology.orchestra.run.smithi043.stderr: 13550K .......... .......... .......... .......... .......... 8% 9.51M 17s 2024-03-30T13:12:45.712 INFO:teuthology.orchestra.run.smithi043.stderr: 13600K .......... .......... .......... .......... .......... 8% 10.9M 17s 2024-03-30T13:12:45.716 INFO:teuthology.orchestra.run.smithi043.stderr: 13650K .......... .......... .......... .......... .......... 8% 11.4M 17s 2024-03-30T13:12:45.720 INFO:teuthology.orchestra.run.smithi043.stderr: 13700K .......... .......... .......... .......... .......... 8% 11.4M 17s 2024-03-30T13:12:45.726 INFO:teuthology.orchestra.run.smithi043.stderr: 13750K .......... .......... .......... .......... .......... 8% 9.34M 17s 2024-03-30T13:12:45.730 INFO:teuthology.orchestra.run.smithi043.stderr: 13800K .......... .......... .......... .......... .......... 8% 10.6M 17s 2024-03-30T13:12:45.735 INFO:teuthology.orchestra.run.smithi043.stderr: 13850K .......... .......... .......... .......... .......... 8% 10.9M 17s 2024-03-30T13:12:45.740 INFO:teuthology.orchestra.run.smithi043.stderr: 13900K .......... .......... .......... .......... .......... 8% 9.59M 17s 2024-03-30T13:12:45.746 INFO:teuthology.orchestra.run.smithi043.stderr: 13950K .......... .......... .......... .......... .......... 8% 7.49M 17s 2024-03-30T13:12:45.752 INFO:teuthology.orchestra.run.smithi043.stderr: 14000K .......... .......... .......... .......... .......... 8% 9.03M 17s 2024-03-30T13:12:45.758 INFO:teuthology.orchestra.run.smithi043.stderr: 14050K .......... .......... .......... .......... .......... 8% 8.40M 17s 2024-03-30T13:12:45.763 INFO:teuthology.orchestra.run.smithi043.stderr: 14100K .......... .......... .......... .......... .......... 8% 9.12M 17s 2024-03-30T13:12:45.769 INFO:teuthology.orchestra.run.smithi043.stderr: 14150K .......... .......... .......... .......... .......... 8% 7.99M 17s 2024-03-30T13:12:45.775 INFO:teuthology.orchestra.run.smithi043.stderr: 14200K .......... .......... .......... .......... .......... 8% 9.07M 17s 2024-03-30T13:12:45.780 INFO:teuthology.orchestra.run.smithi043.stderr: 14250K .......... .......... .......... .......... .......... 8% 8.47M 17s 2024-03-30T13:12:45.786 INFO:teuthology.orchestra.run.smithi043.stderr: 14300K .......... .......... .......... .......... .......... 8% 8.46M 17s 2024-03-30T13:12:45.792 INFO:teuthology.orchestra.run.smithi043.stderr: 14350K .......... .......... .......... .......... .......... 8% 8.07M 17s 2024-03-30T13:12:45.798 INFO:teuthology.orchestra.run.smithi043.stderr: 14400K .......... .......... .......... .......... .......... 8% 8.12M 17s 2024-03-30T13:12:45.804 INFO:teuthology.orchestra.run.smithi043.stderr: 14450K .......... .......... .......... .......... .......... 8% 8.62M 17s 2024-03-30T13:12:45.809 INFO:teuthology.orchestra.run.smithi043.stderr: 14500K .......... .......... .......... .......... .......... 8% 9.06M 17s 2024-03-30T13:12:45.816 INFO:teuthology.orchestra.run.smithi043.stderr: 14550K .......... .......... .......... .......... .......... 8% 7.36M 17s 2024-03-30T13:12:45.821 INFO:teuthology.orchestra.run.smithi043.stderr: 14600K .......... .......... .......... .......... .......... 8% 8.40M 17s 2024-03-30T13:12:45.827 INFO:teuthology.orchestra.run.smithi043.stderr: 14650K .......... .......... .......... .......... .......... 8% 9.75M 17s 2024-03-30T13:12:45.832 INFO:teuthology.orchestra.run.smithi043.stderr: 14700K .......... .......... .......... .......... .......... 8% 9.33M 17s 2024-03-30T13:12:45.838 INFO:teuthology.orchestra.run.smithi043.stderr: 14750K .......... .......... .......... .......... .......... 8% 8.10M 17s 2024-03-30T13:12:45.843 INFO:teuthology.orchestra.run.smithi043.stderr: 14800K .......... .......... .......... .......... .......... 8% 9.37M 17s 2024-03-30T13:12:45.848 INFO:teuthology.orchestra.run.smithi043.stderr: 14850K .......... .......... .......... .......... .......... 8% 9.22M 17s 2024-03-30T13:12:45.854 INFO:teuthology.orchestra.run.smithi043.stderr: 14900K .......... .......... .......... .......... .......... 8% 9.26M 17s 2024-03-30T13:12:45.860 INFO:teuthology.orchestra.run.smithi043.stderr: 14950K .......... .......... .......... .......... .......... 8% 7.94M 17s 2024-03-30T13:12:45.865 INFO:teuthology.orchestra.run.smithi043.stderr: 15000K .......... .......... .......... .......... .......... 8% 9.24M 17s 2024-03-30T13:12:45.871 INFO:teuthology.orchestra.run.smithi043.stderr: 15050K .......... .......... .......... .......... .......... 8% 8.91M 17s 2024-03-30T13:12:45.877 INFO:teuthology.orchestra.run.smithi043.stderr: 15100K .......... .......... .......... .......... .......... 8% 8.10M 17s 2024-03-30T13:12:45.885 INFO:teuthology.orchestra.run.smithi043.stderr: 15150K .......... .......... .......... .......... .......... 9% 6.27M 17s 2024-03-30T13:12:45.891 INFO:teuthology.orchestra.run.smithi043.stderr: 15200K .......... .......... .......... .......... .......... 9% 7.88M 17s 2024-03-30T13:12:45.897 INFO:teuthology.orchestra.run.smithi043.stderr: 15250K .......... .......... .......... .......... .......... 9% 7.98M 17s 2024-03-30T13:12:45.904 INFO:teuthology.orchestra.run.smithi043.stderr: 15300K .......... .......... .......... .......... .......... 9% 7.51M 17s 2024-03-30T13:12:45.911 INFO:teuthology.orchestra.run.smithi043.stderr: 15350K .......... .......... .......... .......... .......... 9% 6.12M 17s 2024-03-30T13:12:45.918 INFO:teuthology.orchestra.run.smithi043.stderr: 15400K .......... .......... .......... .......... .......... 9% 7.60M 17s 2024-03-30T13:12:45.924 INFO:teuthology.orchestra.run.smithi043.stderr: 15450K .......... .......... .......... .......... .......... 9% 7.66M 17s 2024-03-30T13:12:45.930 INFO:teuthology.orchestra.run.smithi043.stderr: 15500K .......... .......... .......... .......... .......... 9% 8.27M 17s 2024-03-30T13:12:45.937 INFO:teuthology.orchestra.run.smithi043.stderr: 15550K .......... .......... .......... .......... .......... 9% 6.53M 17s 2024-03-30T13:12:45.944 INFO:teuthology.orchestra.run.smithi043.stderr: 15600K .......... .......... .......... .......... .......... 9% 7.85M 17s 2024-03-30T13:12:45.950 INFO:teuthology.orchestra.run.smithi043.stderr: 15650K .......... .......... .......... .......... .......... 9% 8.14M 17s 2024-03-30T13:12:45.956 INFO:teuthology.orchestra.run.smithi043.stderr: 15700K .......... .......... .......... .......... .......... 9% 7.43M 17s 2024-03-30T13:12:45.964 INFO:teuthology.orchestra.run.smithi043.stderr: 15750K .......... .......... .......... .......... .......... 9% 6.28M 17s 2024-03-30T13:12:45.971 INFO:teuthology.orchestra.run.smithi043.stderr: 15800K .......... .......... .......... .......... .......... 9% 7.59M 17s 2024-03-30T13:12:45.977 INFO:teuthology.orchestra.run.smithi043.stderr: 15850K .......... .......... .......... .......... .......... 9% 7.58M 17s 2024-03-30T13:12:45.984 INFO:teuthology.orchestra.run.smithi043.stderr: 15900K .......... .......... .......... .......... .......... 9% 7.31M 17s 2024-03-30T13:12:45.991 INFO:teuthology.orchestra.run.smithi043.stderr: 15950K .......... .......... .......... .......... .......... 9% 6.70M 17s 2024-03-30T13:12:45.997 INFO:teuthology.orchestra.run.smithi043.stderr: 16000K .......... .......... .......... .......... .......... 9% 7.65M 17s 2024-03-30T13:12:46.004 INFO:teuthology.orchestra.run.smithi043.stderr: 16050K .......... .......... .......... .......... .......... 9% 7.16M 17s 2024-03-30T13:12:46.011 INFO:teuthology.orchestra.run.smithi043.stderr: 16100K .......... .......... .......... .......... .......... 9% 7.13M 17s 2024-03-30T13:12:46.019 INFO:teuthology.orchestra.run.smithi043.stderr: 16150K .......... .......... .......... .......... .......... 9% 5.85M 17s 2024-03-30T13:12:46.027 INFO:teuthology.orchestra.run.smithi043.stderr: 16200K .......... .......... .......... .......... .......... 9% 6.73M 17s 2024-03-30T13:12:46.034 INFO:teuthology.orchestra.run.smithi043.stderr: 16250K .......... .......... .......... .......... .......... 9% 6.51M 17s 2024-03-30T13:12:46.041 INFO:teuthology.orchestra.run.smithi043.stderr: 16300K .......... .......... .......... .......... .......... 9% 7.05M 17s 2024-03-30T13:12:46.049 INFO:teuthology.orchestra.run.smithi043.stderr: 16350K .......... .......... .......... .......... .......... 9% 5.98M 17s 2024-03-30T13:12:46.054 INFO:teuthology.orchestra.run.smithi043.stderr: 16400K .......... .......... .......... .......... .......... 9% 9.11M 17s 2024-03-30T13:12:46.058 INFO:teuthology.orchestra.run.smithi043.stderr: 16450K .......... .......... .......... .......... .......... 9% 12.6M 17s 2024-03-30T13:12:46.062 INFO:teuthology.orchestra.run.smithi043.stderr: 16500K .......... .......... .......... .......... .......... 9% 13.0M 17s 2024-03-30T13:12:46.067 INFO:teuthology.orchestra.run.smithi043.stderr: 16550K .......... .......... .......... .......... .......... 9% 9.48M 17s 2024-03-30T13:12:46.071 INFO:teuthology.orchestra.run.smithi043.stderr: 16600K .......... .......... .......... .......... .......... 9% 11.1M 17s 2024-03-30T13:12:46.077 INFO:teuthology.orchestra.run.smithi043.stderr: 16650K .......... .......... .......... .......... .......... 9% 10.1M 17s 2024-03-30T13:12:46.082 INFO:teuthology.orchestra.run.smithi043.stderr: 16700K .......... .......... .......... .......... .......... 9% 8.76M 17s 2024-03-30T13:12:46.086 INFO:teuthology.orchestra.run.smithi043.stderr: 16750K .......... .......... .......... .......... .......... 9% 10.1M 17s 2024-03-30T13:12:46.091 INFO:teuthology.orchestra.run.smithi043.stderr: 16800K .......... .......... .......... .......... .......... 9% 11.6M 17s 2024-03-30T13:12:46.095 INFO:teuthology.orchestra.run.smithi043.stderr: 16850K .......... .......... .......... .......... .......... 10% 10.8M 17s 2024-03-30T13:12:46.100 INFO:teuthology.orchestra.run.smithi043.stderr: 16900K .......... .......... .......... .......... .......... 10% 10.1M 17s 2024-03-30T13:12:46.106 INFO:teuthology.orchestra.run.smithi043.stderr: 16950K .......... .......... .......... .......... .......... 10% 8.84M 17s 2024-03-30T13:12:46.111 INFO:teuthology.orchestra.run.smithi043.stderr: 17000K .......... .......... .......... .......... .......... 10% 8.70M 17s 2024-03-30T13:12:46.116 INFO:teuthology.orchestra.run.smithi043.stderr: 17050K .......... .......... .......... .......... .......... 10% 9.73M 17s 2024-03-30T13:12:46.121 INFO:teuthology.orchestra.run.smithi043.stderr: 17100K .......... .......... .......... .......... .......... 10% 10.5M 17s 2024-03-30T13:12:46.126 INFO:teuthology.orchestra.run.smithi043.stderr: 17150K .......... .......... .......... .......... .......... 10% 9.01M 17s 2024-03-30T13:12:46.131 INFO:teuthology.orchestra.run.smithi043.stderr: 17200K .......... .......... .......... .......... .......... 10% 11.1M 17s 2024-03-30T13:12:46.134 INFO:teuthology.orchestra.run.smithi043.stderr: 17250K .......... .......... .......... .......... .......... 10% 13.4M 17s 2024-03-30T13:12:46.138 INFO:teuthology.orchestra.run.smithi043.stderr: 17300K .......... .......... .......... .......... .......... 10% 13.6M 17s 2024-03-30T13:12:46.142 INFO:teuthology.orchestra.run.smithi043.stderr: 17350K .......... .......... .......... .......... .......... 10% 11.2M 17s 2024-03-30T13:12:46.147 INFO:teuthology.orchestra.run.smithi043.stderr: 17400K .......... .......... .......... .......... .......... 10% 11.9M 17s 2024-03-30T13:12:46.151 INFO:teuthology.orchestra.run.smithi043.stderr: 17450K .......... .......... .......... .......... .......... 10% 10.8M 17s 2024-03-30T13:12:46.155 INFO:teuthology.orchestra.run.smithi043.stderr: 17500K .......... .......... .......... .......... .......... 10% 11.1M 17s 2024-03-30T13:12:46.160 INFO:teuthology.orchestra.run.smithi043.stderr: 17550K .......... .......... .......... .......... .......... 10% 9.79M 17s 2024-03-30T13:12:46.165 INFO:teuthology.orchestra.run.smithi043.stderr: 17600K .......... .......... .......... .......... .......... 10% 12.0M 17s 2024-03-30T13:12:46.169 INFO:teuthology.orchestra.run.smithi043.stderr: 17650K .......... .......... .......... .......... .......... 10% 11.0M 17s 2024-03-30T13:12:46.173 INFO:teuthology.orchestra.run.smithi043.stderr: 17700K .......... .......... .......... .......... .......... 10% 10.9M 17s 2024-03-30T13:12:46.179 INFO:teuthology.orchestra.run.smithi043.stderr: 17750K .......... .......... .......... .......... .......... 10% 8.86M 17s 2024-03-30T13:12:46.183 INFO:teuthology.orchestra.run.smithi043.stderr: 17800K .......... .......... .......... .......... .......... 10% 11.1M 17s 2024-03-30T13:12:46.188 INFO:teuthology.orchestra.run.smithi043.stderr: 17850K .......... .......... .......... .......... .......... 10% 11.1M 17s 2024-03-30T13:12:46.192 INFO:teuthology.orchestra.run.smithi043.stderr: 17900K .......... .......... .......... .......... .......... 10% 11.1M 17s 2024-03-30T13:12:46.197 INFO:teuthology.orchestra.run.smithi043.stderr: 17950K .......... .......... .......... .......... .......... 10% 9.29M 17s 2024-03-30T13:12:46.202 INFO:teuthology.orchestra.run.smithi043.stderr: 18000K .......... .......... .......... .......... .......... 10% 10.1M 17s 2024-03-30T13:12:46.207 INFO:teuthology.orchestra.run.smithi043.stderr: 18050K .......... .......... .......... .......... .......... 10% 10.9M 17s 2024-03-30T13:12:46.212 INFO:teuthology.orchestra.run.smithi043.stderr: 18100K .......... .......... .......... .......... .......... 10% 8.78M 17s 2024-03-30T13:12:46.217 INFO:teuthology.orchestra.run.smithi043.stderr: 18150K .......... .......... .......... .......... .......... 10% 9.43M 17s 2024-03-30T13:12:46.221 INFO:teuthology.orchestra.run.smithi043.stderr: 18200K .......... .......... .......... .......... .......... 10% 11.3M 17s 2024-03-30T13:12:46.226 INFO:teuthology.orchestra.run.smithi043.stderr: 18250K .......... .......... .......... .......... .......... 10% 11.7M 17s 2024-03-30T13:12:46.230 INFO:teuthology.orchestra.run.smithi043.stderr: 18300K .......... .......... .......... .......... .......... 10% 12.3M 17s 2024-03-30T13:12:46.235 INFO:teuthology.orchestra.run.smithi043.stderr: 18350K .......... .......... .......... .......... .......... 10% 10.2M 17s 2024-03-30T13:12:46.239 INFO:teuthology.orchestra.run.smithi043.stderr: 18400K .......... .......... .......... .......... .......... 10% 11.6M 17s 2024-03-30T13:12:46.243 INFO:teuthology.orchestra.run.smithi043.stderr: 18450K .......... .......... .......... .......... .......... 10% 10.7M 17s 2024-03-30T13:12:46.248 INFO:teuthology.orchestra.run.smithi043.stderr: 18500K .......... .......... .......... .......... .......... 11% 10.1M 17s 2024-03-30T13:12:46.254 INFO:teuthology.orchestra.run.smithi043.stderr: 18550K .......... .......... .......... .......... .......... 11% 8.81M 17s 2024-03-30T13:12:46.259 INFO:teuthology.orchestra.run.smithi043.stderr: 18600K .......... .......... .......... .......... .......... 11% 9.45M 17s 2024-03-30T13:12:46.265 INFO:teuthology.orchestra.run.smithi043.stderr: 18650K .......... .......... .......... .......... .......... 11% 9.05M 16s 2024-03-30T13:12:46.271 INFO:teuthology.orchestra.run.smithi043.stderr: 18700K .......... .......... .......... .......... .......... 11% 8.16M 16s 2024-03-30T13:12:46.277 INFO:teuthology.orchestra.run.smithi043.stderr: 18750K .......... .......... .......... .......... .......... 11% 7.58M 16s 2024-03-30T13:12:46.282 INFO:teuthology.orchestra.run.smithi043.stderr: 18800K .......... .......... .......... .......... .......... 11% 9.21M 16s 2024-03-30T13:12:46.288 INFO:teuthology.orchestra.run.smithi043.stderr: 18850K .......... .......... .......... .......... .......... 11% 9.01M 16s 2024-03-30T13:12:46.293 INFO:teuthology.orchestra.run.smithi043.stderr: 18900K .......... .......... .......... .......... .......... 11% 8.83M 16s 2024-03-30T13:12:46.299 INFO:teuthology.orchestra.run.smithi043.stderr: 18950K .......... .......... .......... .......... .......... 11% 7.81M 16s 2024-03-30T13:12:46.305 INFO:teuthology.orchestra.run.smithi043.stderr: 19000K .......... .......... .......... .......... .......... 11% 8.79M 16s 2024-03-30T13:12:46.311 INFO:teuthology.orchestra.run.smithi043.stderr: 19050K .......... .......... .......... .......... .......... 11% 8.77M 16s 2024-03-30T13:12:46.316 INFO:teuthology.orchestra.run.smithi043.stderr: 19100K .......... .......... .......... .......... .......... 11% 9.47M 16s 2024-03-30T13:12:46.322 INFO:teuthology.orchestra.run.smithi043.stderr: 19150K .......... .......... .......... .......... .......... 11% 7.94M 16s 2024-03-30T13:12:46.327 INFO:teuthology.orchestra.run.smithi043.stderr: 19200K .......... .......... .......... .......... .......... 11% 8.62M 16s 2024-03-30T13:12:46.333 INFO:teuthology.orchestra.run.smithi043.stderr: 19250K .......... .......... .......... .......... .......... 11% 8.49M 16s 2024-03-30T13:12:46.339 INFO:teuthology.orchestra.run.smithi043.stderr: 19300K .......... .......... .......... .......... .......... 11% 8.51M 16s 2024-03-30T13:12:46.346 INFO:teuthology.orchestra.run.smithi043.stderr: 19350K .......... .......... .......... .......... .......... 11% 6.78M 16s 2024-03-30T13:12:46.353 INFO:teuthology.orchestra.run.smithi043.stderr: 19400K .......... .......... .......... .......... .......... 11% 7.91M 16s 2024-03-30T13:12:46.359 INFO:teuthology.orchestra.run.smithi043.stderr: 19450K .......... .......... .......... .......... .......... 11% 7.72M 16s 2024-03-30T13:12:46.365 INFO:teuthology.orchestra.run.smithi043.stderr: 19500K .......... .......... .......... .......... .......... 11% 7.91M 16s 2024-03-30T13:12:46.373 INFO:teuthology.orchestra.run.smithi043.stderr: 19550K .......... .......... .......... .......... .......... 11% 6.40M 16s 2024-03-30T13:12:46.379 INFO:teuthology.orchestra.run.smithi043.stderr: 19600K .......... .......... .......... .......... .......... 11% 7.78M 16s 2024-03-30T13:12:46.385 INFO:teuthology.orchestra.run.smithi043.stderr: 19650K .......... .......... .......... .......... .......... 11% 7.46M 16s 2024-03-30T13:12:46.392 INFO:teuthology.orchestra.run.smithi043.stderr: 19700K .......... .......... .......... .......... .......... 11% 7.40M 16s 2024-03-30T13:12:46.400 INFO:teuthology.orchestra.run.smithi043.stderr: 19750K .......... .......... .......... .......... .......... 11% 6.09M 16s 2024-03-30T13:12:46.407 INFO:teuthology.orchestra.run.smithi043.stderr: 19800K .......... .......... .......... .......... .......... 11% 7.26M 16s 2024-03-30T13:12:46.414 INFO:teuthology.orchestra.run.smithi043.stderr: 19850K .......... .......... .......... .......... .......... 11% 6.98M 16s 2024-03-30T13:12:46.421 INFO:teuthology.orchestra.run.smithi043.stderr: 19900K .......... .......... .......... .......... .......... 11% 7.19M 16s 2024-03-30T13:12:46.429 INFO:teuthology.orchestra.run.smithi043.stderr: 19950K .......... .......... .......... .......... .......... 11% 6.05M 17s 2024-03-30T13:12:46.436 INFO:teuthology.orchestra.run.smithi043.stderr: 20000K .......... .......... .......... .......... .......... 11% 7.22M 17s 2024-03-30T13:12:46.442 INFO:teuthology.orchestra.run.smithi043.stderr: 20050K .......... .......... .......... .......... .......... 11% 7.20M 17s 2024-03-30T13:12:46.449 INFO:teuthology.orchestra.run.smithi043.stderr: 20100K .......... .......... .......... .......... .......... 11% 6.79M 17s 2024-03-30T13:12:46.458 INFO:teuthology.orchestra.run.smithi043.stderr: 20150K .......... .......... .......... .......... .......... 11% 6.03M 17s 2024-03-30T13:12:46.465 INFO:teuthology.orchestra.run.smithi043.stderr: 20200K .......... .......... .......... .......... .......... 12% 7.26M 17s 2024-03-30T13:12:46.472 INFO:teuthology.orchestra.run.smithi043.stderr: 20250K .......... .......... .......... .......... .......... 12% 6.86M 17s 2024-03-30T13:12:46.478 INFO:teuthology.orchestra.run.smithi043.stderr: 20300K .......... .......... .......... .......... .......... 12% 7.20M 17s 2024-03-30T13:12:46.487 INFO:teuthology.orchestra.run.smithi043.stderr: 20350K .......... .......... .......... .......... .......... 12% 5.76M 17s 2024-03-30T13:12:46.494 INFO:teuthology.orchestra.run.smithi043.stderr: 20400K .......... .......... .......... .......... .......... 12% 6.73M 17s 2024-03-30T13:12:46.501 INFO:teuthology.orchestra.run.smithi043.stderr: 20450K .......... .......... .......... .......... .......... 12% 6.43M 17s 2024-03-30T13:12:46.509 INFO:teuthology.orchestra.run.smithi043.stderr: 20500K .......... .......... .......... .......... .......... 12% 8.25M 17s 2024-03-30T13:12:46.512 INFO:teuthology.orchestra.run.smithi043.stderr: 20550K .......... .......... .......... .......... .......... 12% 8.85M 17s 2024-03-30T13:12:46.516 INFO:teuthology.orchestra.run.smithi043.stderr: 20600K .......... .......... .......... .......... .......... 12% 13.1M 17s 2024-03-30T13:12:46.520 INFO:teuthology.orchestra.run.smithi043.stderr: 20650K .......... .......... .......... .......... .......... 12% 13.1M 17s 2024-03-30T13:12:46.524 INFO:teuthology.orchestra.run.smithi043.stderr: 20700K .......... .......... .......... .......... .......... 12% 12.6M 17s 2024-03-30T13:12:46.529 INFO:teuthology.orchestra.run.smithi043.stderr: 20750K .......... .......... .......... .......... .......... 12% 10.2M 17s 2024-03-30T13:12:46.533 INFO:teuthology.orchestra.run.smithi043.stderr: 20800K .......... .......... .......... .......... .......... 12% 11.6M 16s 2024-03-30T13:12:46.537 INFO:teuthology.orchestra.run.smithi043.stderr: 20850K .......... .......... .......... .......... .......... 12% 12.2M 16s 2024-03-30T13:12:46.541 INFO:teuthology.orchestra.run.smithi043.stderr: 20900K .......... .......... .......... .......... .......... 12% 11.7M 16s 2024-03-30T13:12:46.546 INFO:teuthology.orchestra.run.smithi043.stderr: 20950K .......... .......... .......... .......... .......... 12% 9.74M 16s 2024-03-30T13:12:46.550 INFO:teuthology.orchestra.run.smithi043.stderr: 21000K .......... .......... .......... .......... .......... 12% 11.5M 16s 2024-03-30T13:12:46.555 INFO:teuthology.orchestra.run.smithi043.stderr: 21050K .......... .......... .......... .......... .......... 12% 11.2M 16s 2024-03-30T13:12:46.559 INFO:teuthology.orchestra.run.smithi043.stderr: 21100K .......... .......... .......... .......... .......... 12% 11.5M 16s 2024-03-30T13:12:46.564 INFO:teuthology.orchestra.run.smithi043.stderr: 21150K .......... .......... .......... .......... .......... 12% 9.75M 16s 2024-03-30T13:12:46.568 INFO:teuthology.orchestra.run.smithi043.stderr: 21200K .......... .......... .......... .......... .......... 12% 11.6M 16s 2024-03-30T13:12:46.573 INFO:teuthology.orchestra.run.smithi043.stderr: 21250K .......... .......... .......... .......... .......... 12% 11.4M 16s 2024-03-30T13:12:46.578 INFO:teuthology.orchestra.run.smithi043.stderr: 21300K .......... .......... .......... .......... .......... 12% 9.00M 16s 2024-03-30T13:12:46.584 INFO:teuthology.orchestra.run.smithi043.stderr: 21350K .......... .......... .......... .......... .......... 12% 8.53M 16s 2024-03-30T13:12:46.589 INFO:teuthology.orchestra.run.smithi043.stderr: 21400K .......... .......... .......... .......... .......... 12% 9.55M 16s 2024-03-30T13:12:46.594 INFO:teuthology.orchestra.run.smithi043.stderr: 21450K .......... .......... .......... .......... .......... 12% 9.83M 16s 2024-03-30T13:12:46.599 INFO:teuthology.orchestra.run.smithi043.stderr: 21500K .......... .......... .......... .......... .......... 12% 8.89M 16s 2024-03-30T13:12:46.606 INFO:teuthology.orchestra.run.smithi043.stderr: 21550K .......... .......... .......... .......... .......... 12% 7.68M 16s 2024-03-30T13:12:46.611 INFO:teuthology.orchestra.run.smithi043.stderr: 21600K .......... .......... .......... .......... .......... 12% 8.77M 16s 2024-03-30T13:12:46.617 INFO:teuthology.orchestra.run.smithi043.stderr: 21650K .......... .......... .......... .......... .......... 12% 8.88M 16s 2024-03-30T13:12:46.622 INFO:teuthology.orchestra.run.smithi043.stderr: 21700K .......... .......... .......... .......... .......... 12% 9.30M 16s 2024-03-30T13:12:46.628 INFO:teuthology.orchestra.run.smithi043.stderr: 21750K .......... .......... .......... .......... .......... 12% 8.76M 16s 2024-03-30T13:12:46.633 INFO:teuthology.orchestra.run.smithi043.stderr: 21800K .......... .......... .......... .......... .......... 12% 9.77M 16s 2024-03-30T13:12:46.638 INFO:teuthology.orchestra.run.smithi043.stderr: 21850K .......... .......... .......... .......... .......... 12% 9.38M 16s 2024-03-30T13:12:46.643 INFO:teuthology.orchestra.run.smithi043.stderr: 21900K .......... .......... .......... .......... .......... 13% 9.18M 16s 2024-03-30T13:12:46.650 INFO:teuthology.orchestra.run.smithi043.stderr: 21950K .......... .......... .......... .......... .......... 13% 7.13M 16s 2024-03-30T13:12:46.656 INFO:teuthology.orchestra.run.smithi043.stderr: 22000K .......... .......... .......... .......... .......... 13% 8.16M 16s 2024-03-30T13:12:46.661 INFO:teuthology.orchestra.run.smithi043.stderr: 22050K .......... .......... .......... .......... .......... 13% 8.88M 16s 2024-03-30T13:12:46.667 INFO:teuthology.orchestra.run.smithi043.stderr: 22100K .......... .......... .......... .......... .......... 13% 7.99M 16s 2024-03-30T13:12:46.674 INFO:teuthology.orchestra.run.smithi043.stderr: 22150K .......... .......... .......... .......... .......... 13% 7.41M 16s 2024-03-30T13:12:46.680 INFO:teuthology.orchestra.run.smithi043.stderr: 22200K .......... .......... .......... .......... .......... 13% 8.65M 16s 2024-03-30T13:12:46.685 INFO:teuthology.orchestra.run.smithi043.stderr: 22250K .......... .......... .......... .......... .......... 13% 8.68M 16s 2024-03-30T13:12:46.691 INFO:teuthology.orchestra.run.smithi043.stderr: 22300K .......... .......... .......... .......... .......... 13% 8.48M 16s 2024-03-30T13:12:46.698 INFO:teuthology.orchestra.run.smithi043.stderr: 22350K .......... .......... .......... .......... .......... 13% 7.03M 16s 2024-03-30T13:12:46.704 INFO:teuthology.orchestra.run.smithi043.stderr: 22400K .......... .......... .......... .......... .......... 13% 8.33M 16s 2024-03-30T13:12:46.710 INFO:teuthology.orchestra.run.smithi043.stderr: 22450K .......... .......... .......... .......... .......... 13% 8.15M 16s 2024-03-30T13:12:46.716 INFO:teuthology.orchestra.run.smithi043.stderr: 22500K .......... .......... .......... .......... .......... 13% 8.07M 16s 2024-03-30T13:12:46.723 INFO:teuthology.orchestra.run.smithi043.stderr: 22550K .......... .......... .......... .......... .......... 13% 7.49M 16s 2024-03-30T13:12:46.729 INFO:teuthology.orchestra.run.smithi043.stderr: 22600K .......... .......... .......... .......... .......... 13% 8.26M 16s 2024-03-30T13:12:46.735 INFO:teuthology.orchestra.run.smithi043.stderr: 22650K .......... .......... .......... .......... .......... 13% 8.08M 16s 2024-03-30T13:12:46.741 INFO:teuthology.orchestra.run.smithi043.stderr: 22700K .......... .......... .......... .......... .......... 13% 8.32M 16s 2024-03-30T13:12:46.748 INFO:teuthology.orchestra.run.smithi043.stderr: 22750K .......... .......... .......... .......... .......... 13% 7.16M 16s 2024-03-30T13:12:46.753 INFO:teuthology.orchestra.run.smithi043.stderr: 22800K .......... .......... .......... .......... .......... 13% 8.19M 16s 2024-03-30T13:12:46.759 INFO:teuthology.orchestra.run.smithi043.stderr: 22850K .......... .......... .......... .......... .......... 13% 8.24M 16s 2024-03-30T13:12:46.765 INFO:teuthology.orchestra.run.smithi043.stderr: 22900K .......... .......... .......... .......... .......... 13% 8.48M 16s 2024-03-30T13:12:46.772 INFO:teuthology.orchestra.run.smithi043.stderr: 22950K .......... .......... .......... .......... .......... 13% 7.08M 16s 2024-03-30T13:12:46.779 INFO:teuthology.orchestra.run.smithi043.stderr: 23000K .......... .......... .......... .......... .......... 13% 8.06M 16s 2024-03-30T13:12:46.784 INFO:teuthology.orchestra.run.smithi043.stderr: 23050K .......... .......... .......... .......... .......... 13% 8.18M 16s 2024-03-30T13:12:46.790 INFO:teuthology.orchestra.run.smithi043.stderr: 23100K .......... .......... .......... .......... .......... 13% 7.97M 16s 2024-03-30T13:12:46.797 INFO:teuthology.orchestra.run.smithi043.stderr: 23150K .......... .......... .......... .......... .......... 13% 6.77M 16s 2024-03-30T13:12:46.803 INFO:teuthology.orchestra.run.smithi043.stderr: 23200K .......... .......... .......... .......... .......... 13% 8.11M 16s 2024-03-30T13:12:46.810 INFO:teuthology.orchestra.run.smithi043.stderr: 23250K .......... .......... .......... .......... .......... 13% 7.49M 16s 2024-03-30T13:12:46.816 INFO:teuthology.orchestra.run.smithi043.stderr: 23300K .......... .......... .......... .......... .......... 13% 7.90M 16s 2024-03-30T13:12:46.823 INFO:teuthology.orchestra.run.smithi043.stderr: 23350K .......... .......... .......... .......... .......... 13% 6.74M 16s 2024-03-30T13:12:46.829 INFO:teuthology.orchestra.run.smithi043.stderr: 23400K .......... .......... .......... .......... .......... 13% 8.23M 16s 2024-03-30T13:12:46.835 INFO:teuthology.orchestra.run.smithi043.stderr: 23450K .......... .......... .......... .......... .......... 13% 7.94M 16s 2024-03-30T13:12:46.841 INFO:teuthology.orchestra.run.smithi043.stderr: 23500K .......... .......... .......... .......... .......... 13% 8.27M 16s 2024-03-30T13:12:46.848 INFO:teuthology.orchestra.run.smithi043.stderr: 23550K .......... .......... .......... .......... .......... 14% 7.40M 16s 2024-03-30T13:12:46.854 INFO:teuthology.orchestra.run.smithi043.stderr: 23600K .......... .......... .......... .......... .......... 14% 7.69M 16s 2024-03-30T13:12:46.861 INFO:teuthology.orchestra.run.smithi043.stderr: 23650K .......... .......... .......... .......... .......... 14% 7.63M 16s 2024-03-30T13:12:46.867 INFO:teuthology.orchestra.run.smithi043.stderr: 23700K .......... .......... .......... .......... .......... 14% 7.34M 16s 2024-03-30T13:12:46.874 INFO:teuthology.orchestra.run.smithi043.stderr: 23750K .......... .......... .......... .......... .......... 14% 7.02M 16s 2024-03-30T13:12:46.880 INFO:teuthology.orchestra.run.smithi043.stderr: 23800K .......... .......... .......... .......... .......... 14% 8.30M 16s 2024-03-30T13:12:46.886 INFO:teuthology.orchestra.run.smithi043.stderr: 23850K .......... .......... .......... .......... .......... 14% 8.08M 16s 2024-03-30T13:12:46.893 INFO:teuthology.orchestra.run.smithi043.stderr: 23900K .......... .......... .......... .......... .......... 14% 7.92M 16s 2024-03-30T13:12:46.899 INFO:teuthology.orchestra.run.smithi043.stderr: 23950K .......... .......... .......... .......... .......... 14% 7.15M 16s 2024-03-30T13:12:46.905 INFO:teuthology.orchestra.run.smithi043.stderr: 24000K .......... .......... .......... .......... .......... 14% 8.42M 16s 2024-03-30T13:12:46.911 INFO:teuthology.orchestra.run.smithi043.stderr: 24050K .......... .......... .......... .......... .......... 14% 8.15M 16s 2024-03-30T13:12:46.917 INFO:teuthology.orchestra.run.smithi043.stderr: 24100K .......... .......... .......... .......... .......... 14% 8.46M 16s 2024-03-30T13:12:46.924 INFO:teuthology.orchestra.run.smithi043.stderr: 24150K .......... .......... .......... .......... .......... 14% 6.73M 16s 2024-03-30T13:12:46.930 INFO:teuthology.orchestra.run.smithi043.stderr: 24200K .......... .......... .......... .......... .......... 14% 8.01M 16s 2024-03-30T13:12:46.935 INFO:teuthology.orchestra.run.smithi043.stderr: 24250K .......... .......... .......... .......... .......... 14% 8.99M 16s 2024-03-30T13:12:46.941 INFO:teuthology.orchestra.run.smithi043.stderr: 24300K .......... .......... .......... .......... .......... 14% 9.33M 16s 2024-03-30T13:12:46.947 INFO:teuthology.orchestra.run.smithi043.stderr: 24350K .......... .......... .......... .......... .......... 14% 7.70M 16s 2024-03-30T13:12:46.952 INFO:teuthology.orchestra.run.smithi043.stderr: 24400K .......... .......... .......... .......... .......... 14% 9.07M 16s 2024-03-30T13:12:46.958 INFO:teuthology.orchestra.run.smithi043.stderr: 24450K .......... .......... .......... .......... .......... 14% 9.09M 16s 2024-03-30T13:12:46.963 INFO:teuthology.orchestra.run.smithi043.stderr: 24500K .......... .......... .......... .......... .......... 14% 8.87M 16s 2024-03-30T13:12:46.969 INFO:teuthology.orchestra.run.smithi043.stderr: 24550K .......... .......... .......... .......... .......... 14% 8.46M 16s 2024-03-30T13:12:46.973 INFO:teuthology.orchestra.run.smithi043.stderr: 24600K .......... .......... .......... .......... .......... 14% 11.5M 16s 2024-03-30T13:12:46.977 INFO:teuthology.orchestra.run.smithi043.stderr: 24650K .......... .......... .......... .......... .......... 14% 10.8M 16s 2024-03-30T13:12:46.981 INFO:teuthology.orchestra.run.smithi043.stderr: 24700K .......... .......... .......... .......... .......... 14% 14.8M 16s 2024-03-30T13:12:46.986 INFO:teuthology.orchestra.run.smithi043.stderr: 24750K .......... .......... .......... .......... .......... 14% 10.4M 16s 2024-03-30T13:12:46.990 INFO:teuthology.orchestra.run.smithi043.stderr: 24800K .......... .......... .......... .......... .......... 14% 10.8M 16s 2024-03-30T13:12:46.994 INFO:teuthology.orchestra.run.smithi043.stderr: 24850K .......... .......... .......... .......... .......... 14% 11.8M 16s 2024-03-30T13:12:46.999 INFO:teuthology.orchestra.run.smithi043.stderr: 24900K .......... .......... .......... .......... .......... 14% 10.4M 16s 2024-03-30T13:12:47.003 INFO:teuthology.orchestra.run.smithi043.stderr: 24950K .......... .......... .......... .......... .......... 14% 11.4M 16s 2024-03-30T13:12:47.007 INFO:teuthology.orchestra.run.smithi043.stderr: 25000K .......... .......... .......... .......... .......... 14% 11.8M 16s 2024-03-30T13:12:47.012 INFO:teuthology.orchestra.run.smithi043.stderr: 25050K .......... .......... .......... .......... .......... 14% 9.87M 16s 2024-03-30T13:12:47.017 INFO:teuthology.orchestra.run.smithi043.stderr: 25100K .......... .......... .......... .......... .......... 14% 10.0M 16s 2024-03-30T13:12:47.023 INFO:teuthology.orchestra.run.smithi043.stderr: 25150K .......... .......... .......... .......... .......... 14% 8.77M 16s 2024-03-30T13:12:47.027 INFO:teuthology.orchestra.run.smithi043.stderr: 25200K .......... .......... .......... .......... .......... 14% 11.2M 16s 2024-03-30T13:12:47.031 INFO:teuthology.orchestra.run.smithi043.stderr: 25250K .......... .......... .......... .......... .......... 15% 11.5M 16s 2024-03-30T13:12:47.036 INFO:teuthology.orchestra.run.smithi043.stderr: 25300K .......... .......... .......... .......... .......... 15% 10.4M 16s 2024-03-30T13:12:47.041 INFO:teuthology.orchestra.run.smithi043.stderr: 25350K .......... .......... .......... .......... .......... 15% 9.27M 16s 2024-03-30T13:12:47.046 INFO:teuthology.orchestra.run.smithi043.stderr: 25400K .......... .......... .......... .......... .......... 15% 11.0M 16s 2024-03-30T13:12:47.050 INFO:teuthology.orchestra.run.smithi043.stderr: 25450K .......... .......... .......... .......... .......... 15% 11.0M 16s 2024-03-30T13:12:47.055 INFO:teuthology.orchestra.run.smithi043.stderr: 25500K .......... .......... .......... .......... .......... 15% 10.7M 16s 2024-03-30T13:12:47.060 INFO:teuthology.orchestra.run.smithi043.stderr: 25550K .......... .......... .......... .......... .......... 15% 8.67M 16s 2024-03-30T13:12:47.065 INFO:teuthology.orchestra.run.smithi043.stderr: 25600K .......... .......... .......... .......... .......... 15% 10.4M 16s 2024-03-30T13:12:47.070 INFO:teuthology.orchestra.run.smithi043.stderr: 25650K .......... .......... .......... .......... .......... 15% 10.9M 16s 2024-03-30T13:12:47.074 INFO:teuthology.orchestra.run.smithi043.stderr: 25700K .......... .......... .......... .......... .......... 15% 10.4M 16s 2024-03-30T13:12:47.080 INFO:teuthology.orchestra.run.smithi043.stderr: 25750K .......... .......... .......... .......... .......... 15% 8.74M 16s 2024-03-30T13:12:47.086 INFO:teuthology.orchestra.run.smithi043.stderr: 25800K .......... .......... .......... .......... .......... 15% 8.72M 16s 2024-03-30T13:12:47.092 INFO:teuthology.orchestra.run.smithi043.stderr: 25850K .......... .......... .......... .......... .......... 15% 8.78M 16s 2024-03-30T13:12:47.097 INFO:teuthology.orchestra.run.smithi043.stderr: 25900K .......... .......... .......... .......... .......... 15% 8.12M 16s 2024-03-30T13:12:47.104 INFO:teuthology.orchestra.run.smithi043.stderr: 25950K .......... .......... .......... .......... .......... 15% 7.54M 16s 2024-03-30T13:12:47.109 INFO:teuthology.orchestra.run.smithi043.stderr: 26000K .......... .......... .......... .......... .......... 15% 8.86M 16s 2024-03-30T13:12:47.115 INFO:teuthology.orchestra.run.smithi043.stderr: 26050K .......... .......... .......... .......... .......... 15% 8.62M 16s 2024-03-30T13:12:47.120 INFO:teuthology.orchestra.run.smithi043.stderr: 26100K .......... .......... .......... .......... .......... 15% 9.29M 16s 2024-03-30T13:12:47.126 INFO:teuthology.orchestra.run.smithi043.stderr: 26150K .......... .......... .......... .......... .......... 15% 7.80M 16s 2024-03-30T13:12:47.132 INFO:teuthology.orchestra.run.smithi043.stderr: 26200K .......... .......... .......... .......... .......... 15% 8.61M 16s 2024-03-30T13:12:47.137 INFO:teuthology.orchestra.run.smithi043.stderr: 26250K .......... .......... .......... .......... .......... 15% 9.08M 16s 2024-03-30T13:12:47.143 INFO:teuthology.orchestra.run.smithi043.stderr: 26300K .......... .......... .......... .......... .......... 15% 9.24M 16s 2024-03-30T13:12:47.149 INFO:teuthology.orchestra.run.smithi043.stderr: 26350K .......... .......... .......... .......... .......... 15% 7.59M 16s 2024-03-30T13:12:47.154 INFO:teuthology.orchestra.run.smithi043.stderr: 26400K .......... .......... .......... .......... .......... 15% 9.38M 16s 2024-03-30T13:12:47.160 INFO:teuthology.orchestra.run.smithi043.stderr: 26450K .......... .......... .......... .......... .......... 15% 9.19M 16s 2024-03-30T13:12:47.165 INFO:teuthology.orchestra.run.smithi043.stderr: 26500K .......... .......... .......... .......... .......... 15% 9.38M 16s 2024-03-30T13:12:47.171 INFO:teuthology.orchestra.run.smithi043.stderr: 26550K .......... .......... .......... .......... .......... 15% 7.58M 16s 2024-03-30T13:12:47.177 INFO:teuthology.orchestra.run.smithi043.stderr: 26600K .......... .......... .......... .......... .......... 15% 8.43M 16s 2024-03-30T13:12:47.183 INFO:teuthology.orchestra.run.smithi043.stderr: 26650K .......... .......... .......... .......... .......... 15% 8.33M 16s 2024-03-30T13:12:47.189 INFO:teuthology.orchestra.run.smithi043.stderr: 26700K .......... .......... .......... .......... .......... 15% 7.86M 16s 2024-03-30T13:12:47.196 INFO:teuthology.orchestra.run.smithi043.stderr: 26750K .......... .......... .......... .......... .......... 15% 6.92M 16s 2024-03-30T13:12:47.203 INFO:teuthology.orchestra.run.smithi043.stderr: 26800K .......... .......... .......... .......... .......... 15% 7.68M 16s 2024-03-30T13:12:47.209 INFO:teuthology.orchestra.run.smithi043.stderr: 26850K .......... .......... .......... .......... .......... 15% 8.21M 16s 2024-03-30T13:12:47.215 INFO:teuthology.orchestra.run.smithi043.stderr: 26900K .......... .......... .......... .......... .......... 15% 8.21M 16s 2024-03-30T13:12:47.221 INFO:teuthology.orchestra.run.smithi043.stderr: 26950K .......... .......... .......... .......... .......... 16% 7.13M 16s 2024-03-30T13:12:47.227 INFO:teuthology.orchestra.run.smithi043.stderr: 27000K .......... .......... .......... .......... .......... 16% 8.56M 16s 2024-03-30T13:12:47.233 INFO:teuthology.orchestra.run.smithi043.stderr: 27050K .......... .......... .......... .......... .......... 16% 8.46M 16s 2024-03-30T13:12:47.239 INFO:teuthology.orchestra.run.smithi043.stderr: 27100K .......... .......... .......... .......... .......... 16% 8.24M 16s 2024-03-30T13:12:47.246 INFO:teuthology.orchestra.run.smithi043.stderr: 27150K .......... .......... .......... .......... .......... 16% 7.01M 16s 2024-03-30T13:12:47.252 INFO:teuthology.orchestra.run.smithi043.stderr: 27200K .......... .......... .......... .......... .......... 16% 7.87M 16s 2024-03-30T13:12:47.258 INFO:teuthology.orchestra.run.smithi043.stderr: 27250K .......... .......... .......... .......... .......... 16% 7.98M 16s 2024-03-30T13:12:47.264 INFO:teuthology.orchestra.run.smithi043.stderr: 27300K .......... .......... .......... .......... .......... 16% 7.71M 16s 2024-03-30T13:12:47.272 INFO:teuthology.orchestra.run.smithi043.stderr: 27350K .......... .......... .......... .......... .......... 16% 6.44M 16s 2024-03-30T13:12:47.279 INFO:teuthology.orchestra.run.smithi043.stderr: 27400K .......... .......... .......... .......... .......... 16% 7.54M 16s 2024-03-30T13:12:47.285 INFO:teuthology.orchestra.run.smithi043.stderr: 27450K .......... .......... .......... .......... .......... 16% 7.56M 16s 2024-03-30T13:12:47.291 INFO:teuthology.orchestra.run.smithi043.stderr: 27500K .......... .......... .......... .......... .......... 16% 7.68M 16s 2024-03-30T13:12:47.299 INFO:teuthology.orchestra.run.smithi043.stderr: 27550K .......... .......... .......... .......... .......... 16% 6.54M 16s 2024-03-30T13:12:47.305 INFO:teuthology.orchestra.run.smithi043.stderr: 27600K .......... .......... .......... .......... .......... 16% 7.49M 16s 2024-03-30T13:12:47.312 INFO:teuthology.orchestra.run.smithi043.stderr: 27650K .......... .......... .......... .......... .......... 16% 7.32M 16s 2024-03-30T13:12:47.319 INFO:teuthology.orchestra.run.smithi043.stderr: 27700K .......... .......... .......... .......... .......... 16% 7.39M 16s 2024-03-30T13:12:47.327 INFO:teuthology.orchestra.run.smithi043.stderr: 27750K .......... .......... .......... .......... .......... 16% 6.13M 16s 2024-03-30T13:12:47.334 INFO:teuthology.orchestra.run.smithi043.stderr: 27800K .......... .......... .......... .......... .......... 16% 6.79M 16s 2024-03-30T13:12:47.341 INFO:teuthology.orchestra.run.smithi043.stderr: 27850K .......... .......... .......... .......... .......... 16% 7.05M 16s 2024-03-30T13:12:47.348 INFO:teuthology.orchestra.run.smithi043.stderr: 27900K .......... .......... .......... .......... .......... 16% 6.97M 16s 2024-03-30T13:12:47.356 INFO:teuthology.orchestra.run.smithi043.stderr: 27950K .......... .......... .......... .......... .......... 16% 6.11M 16s 2024-03-30T13:12:47.363 INFO:teuthology.orchestra.run.smithi043.stderr: 28000K .......... .......... .......... .......... .......... 16% 7.09M 16s 2024-03-30T13:12:47.370 INFO:teuthology.orchestra.run.smithi043.stderr: 28050K .......... .......... .......... .......... .......... 16% 6.67M 16s 2024-03-30T13:12:47.376 INFO:teuthology.orchestra.run.smithi043.stderr: 28100K .......... .......... .......... .......... .......... 16% 8.25M 16s 2024-03-30T13:12:47.382 INFO:teuthology.orchestra.run.smithi043.stderr: 28150K .......... .......... .......... .......... .......... 16% 7.50M 16s 2024-03-30T13:12:47.388 INFO:teuthology.orchestra.run.smithi043.stderr: 28200K .......... .......... .......... .......... .......... 16% 8.24M 16s 2024-03-30T13:12:47.393 INFO:teuthology.orchestra.run.smithi043.stderr: 28250K .......... .......... .......... .......... .......... 16% 9.05M 16s 2024-03-30T13:12:47.399 INFO:teuthology.orchestra.run.smithi043.stderr: 28300K .......... .......... .......... .......... .......... 16% 8.73M 16s 2024-03-30T13:12:47.406 INFO:teuthology.orchestra.run.smithi043.stderr: 28350K .......... .......... .......... .......... .......... 16% 7.26M 16s 2024-03-30T13:12:47.412 INFO:teuthology.orchestra.run.smithi043.stderr: 28400K .......... .......... .......... .......... .......... 16% 8.27M 16s 2024-03-30T13:12:47.418 INFO:teuthology.orchestra.run.smithi043.stderr: 28450K .......... .......... .......... .......... .......... 16% 8.35M 16s 2024-03-30T13:12:47.424 INFO:teuthology.orchestra.run.smithi043.stderr: 28500K .......... .......... .......... .......... .......... 16% 7.47M 16s 2024-03-30T13:12:47.432 INFO:teuthology.orchestra.run.smithi043.stderr: 28550K .......... .......... .......... .......... .......... 16% 6.35M 16s 2024-03-30T13:12:47.439 INFO:teuthology.orchestra.run.smithi043.stderr: 28600K .......... .......... .......... .......... .......... 16% 7.51M 16s 2024-03-30T13:12:47.444 INFO:teuthology.orchestra.run.smithi043.stderr: 28650K .......... .......... .......... .......... .......... 17% 7.85M 16s 2024-03-30T13:12:47.449 INFO:teuthology.orchestra.run.smithi043.stderr: 28700K .......... .......... .......... .......... .......... 17% 11.0M 16s 2024-03-30T13:12:47.454 INFO:teuthology.orchestra.run.smithi043.stderr: 28750K .......... .......... .......... .......... .......... 17% 8.94M 16s 2024-03-30T13:12:47.458 INFO:teuthology.orchestra.run.smithi043.stderr: 28800K .......... .......... .......... .......... .......... 17% 11.8M 16s 2024-03-30T13:12:47.463 INFO:teuthology.orchestra.run.smithi043.stderr: 28850K .......... .......... .......... .......... .......... 17% 11.6M 16s 2024-03-30T13:12:47.467 INFO:teuthology.orchestra.run.smithi043.stderr: 28900K .......... .......... .......... .......... .......... 17% 11.8M 16s 2024-03-30T13:12:47.472 INFO:teuthology.orchestra.run.smithi043.stderr: 28950K .......... .......... .......... .......... .......... 17% 9.41M 16s 2024-03-30T13:12:47.476 INFO:teuthology.orchestra.run.smithi043.stderr: 29000K .......... .......... .......... .......... .......... 17% 11.2M 16s 2024-03-30T13:12:47.481 INFO:teuthology.orchestra.run.smithi043.stderr: 29050K .......... .......... .......... .......... .......... 17% 11.2M 16s 2024-03-30T13:12:47.485 INFO:teuthology.orchestra.run.smithi043.stderr: 29100K .......... .......... .......... .......... .......... 17% 11.0M 16s 2024-03-30T13:12:47.490 INFO:teuthology.orchestra.run.smithi043.stderr: 29150K .......... .......... .......... .......... .......... 17% 9.97M 16s 2024-03-30T13:12:47.495 INFO:teuthology.orchestra.run.smithi043.stderr: 29200K .......... .......... .......... .......... .......... 17% 10.1M 16s 2024-03-30T13:12:47.500 INFO:teuthology.orchestra.run.smithi043.stderr: 29250K .......... .......... .......... .......... .......... 17% 9.62M 16s 2024-03-30T13:12:47.505 INFO:teuthology.orchestra.run.smithi043.stderr: 29300K .......... .......... .......... .......... .......... 17% 9.81M 16s 2024-03-30T13:12:47.511 INFO:teuthology.orchestra.run.smithi043.stderr: 29350K .......... .......... .......... .......... .......... 17% 8.22M 16s 2024-03-30T13:12:47.516 INFO:teuthology.orchestra.run.smithi043.stderr: 29400K .......... .......... .......... .......... .......... 17% 9.63M 16s 2024-03-30T13:12:47.522 INFO:teuthology.orchestra.run.smithi043.stderr: 29450K .......... .......... .......... .......... .......... 17% 8.90M 16s 2024-03-30T13:12:47.527 INFO:teuthology.orchestra.run.smithi043.stderr: 29500K .......... .......... .......... .......... .......... 17% 9.03M 16s 2024-03-30T13:12:47.533 INFO:teuthology.orchestra.run.smithi043.stderr: 29550K .......... .......... .......... .......... .......... 17% 8.16M 16s 2024-03-30T13:12:47.538 INFO:teuthology.orchestra.run.smithi043.stderr: 29600K .......... .......... .......... .......... .......... 17% 10.2M 16s 2024-03-30T13:12:47.543 INFO:teuthology.orchestra.run.smithi043.stderr: 29650K .......... .......... .......... .......... .......... 17% 10.2M 16s 2024-03-30T13:12:47.547 INFO:teuthology.orchestra.run.smithi043.stderr: 29700K .......... .......... .......... .......... .......... 17% 9.92M 16s 2024-03-30T13:12:47.553 INFO:teuthology.orchestra.run.smithi043.stderr: 29750K .......... .......... .......... .......... .......... 17% 9.35M 16s 2024-03-30T13:12:47.557 INFO:teuthology.orchestra.run.smithi043.stderr: 29800K .......... .......... .......... .......... .......... 17% 10.8M 16s 2024-03-30T13:12:47.562 INFO:teuthology.orchestra.run.smithi043.stderr: 29850K .......... .......... .......... .......... .......... 17% 11.0M 16s 2024-03-30T13:12:47.566 INFO:teuthology.orchestra.run.smithi043.stderr: 29900K .......... .......... .......... .......... .......... 17% 10.4M 16s 2024-03-30T13:12:47.572 INFO:teuthology.orchestra.run.smithi043.stderr: 29950K .......... .......... .......... .......... .......... 17% 8.95M 16s 2024-03-30T13:12:47.577 INFO:teuthology.orchestra.run.smithi043.stderr: 30000K .......... .......... .......... .......... .......... 17% 9.53M 16s 2024-03-30T13:12:47.582 INFO:teuthology.orchestra.run.smithi043.stderr: 30050K .......... .......... .......... .......... .......... 17% 9.76M 16s 2024-03-30T13:12:47.587 INFO:teuthology.orchestra.run.smithi043.stderr: 30100K .......... .......... .......... .......... .......... 17% 9.41M 16s 2024-03-30T13:12:47.593 INFO:teuthology.orchestra.run.smithi043.stderr: 30150K .......... .......... .......... .......... .......... 17% 8.26M 16s 2024-03-30T13:12:47.598 INFO:teuthology.orchestra.run.smithi043.stderr: 30200K .......... .......... .......... .......... .......... 17% 9.59M 16s 2024-03-30T13:12:47.603 INFO:teuthology.orchestra.run.smithi043.stderr: 30250K .......... .......... .......... .......... .......... 17% 9.72M 16s 2024-03-30T13:12:47.608 INFO:teuthology.orchestra.run.smithi043.stderr: 30300K .......... .......... .......... .......... .......... 18% 9.92M 15s 2024-03-30T13:12:47.614 INFO:teuthology.orchestra.run.smithi043.stderr: 30350K .......... .......... .......... .......... .......... 18% 8.23M 15s 2024-03-30T13:12:47.619 INFO:teuthology.orchestra.run.smithi043.stderr: 30400K .......... .......... .......... .......... .......... 18% 10.4M 15s 2024-03-30T13:12:47.624 INFO:teuthology.orchestra.run.smithi043.stderr: 30450K .......... .......... .......... .......... .......... 18% 9.44M 15s 2024-03-30T13:12:47.629 INFO:teuthology.orchestra.run.smithi043.stderr: 30500K .......... .......... .......... .......... .......... 18% 9.51M 15s 2024-03-30T13:12:47.635 INFO:teuthology.orchestra.run.smithi043.stderr: 30550K .......... .......... .......... .......... .......... 18% 8.47M 15s 2024-03-30T13:12:47.640 INFO:teuthology.orchestra.run.smithi043.stderr: 30600K .......... .......... .......... .......... .......... 18% 9.70M 15s 2024-03-30T13:12:47.644 INFO:teuthology.orchestra.run.smithi043.stderr: 30650K .......... .......... .......... .......... .......... 18% 10.4M 15s 2024-03-30T13:12:47.649 INFO:teuthology.orchestra.run.smithi043.stderr: 30700K .......... .......... .......... .......... .......... 18% 10.2M 15s 2024-03-30T13:12:47.655 INFO:teuthology.orchestra.run.smithi043.stderr: 30750K .......... .......... .......... .......... .......... 18% 8.70M 15s 2024-03-30T13:12:47.660 INFO:teuthology.orchestra.run.smithi043.stderr: 30800K .......... .......... .......... .......... .......... 18% 9.64M 15s 2024-03-30T13:12:47.666 INFO:teuthology.orchestra.run.smithi043.stderr: 30850K .......... .......... .......... .......... .......... 18% 9.85M 15s 2024-03-30T13:12:47.670 INFO:teuthology.orchestra.run.smithi043.stderr: 30900K .......... .......... .......... .......... .......... 18% 9.48M 15s 2024-03-30T13:12:47.677 INFO:teuthology.orchestra.run.smithi043.stderr: 30950K .......... .......... .......... .......... .......... 18% 7.01M 15s 2024-03-30T13:12:47.683 INFO:teuthology.orchestra.run.smithi043.stderr: 31000K .......... .......... .......... .......... .......... 18% 8.65M 15s 2024-03-30T13:12:47.689 INFO:teuthology.orchestra.run.smithi043.stderr: 31050K .......... .......... .......... .......... .......... 18% 8.15M 15s 2024-03-30T13:12:47.695 INFO:teuthology.orchestra.run.smithi043.stderr: 31100K .......... .......... .......... .......... .......... 18% 8.03M 15s 2024-03-30T13:12:47.702 INFO:teuthology.orchestra.run.smithi043.stderr: 31150K .......... .......... .......... .......... .......... 18% 6.85M 15s 2024-03-30T13:12:47.708 INFO:teuthology.orchestra.run.smithi043.stderr: 31200K .......... .......... .......... .......... .......... 18% 8.35M 15s 2024-03-30T13:12:47.714 INFO:teuthology.orchestra.run.smithi043.stderr: 31250K .......... .......... .......... .......... .......... 18% 7.43M 15s 2024-03-30T13:12:47.721 INFO:teuthology.orchestra.run.smithi043.stderr: 31300K .......... .......... .......... .......... .......... 18% 7.89M 15s 2024-03-30T13:12:47.729 INFO:teuthology.orchestra.run.smithi043.stderr: 31350K .......... .......... .......... .......... .......... 18% 6.07M 15s 2024-03-30T13:12:47.735 INFO:teuthology.orchestra.run.smithi043.stderr: 31400K .......... .......... .......... .......... .......... 18% 7.59M 15s 2024-03-30T13:12:47.741 INFO:teuthology.orchestra.run.smithi043.stderr: 31450K .......... .......... .......... .......... .......... 18% 8.54M 15s 2024-03-30T13:12:47.746 INFO:teuthology.orchestra.run.smithi043.stderr: 31500K .......... .......... .......... .......... .......... 18% 8.69M 15s 2024-03-30T13:12:47.753 INFO:teuthology.orchestra.run.smithi043.stderr: 31550K .......... .......... .......... .......... .......... 18% 7.68M 15s 2024-03-30T13:12:47.758 INFO:teuthology.orchestra.run.smithi043.stderr: 31600K .......... .......... .......... .......... .......... 18% 8.90M 15s 2024-03-30T13:12:47.764 INFO:teuthology.orchestra.run.smithi043.stderr: 31650K .......... .......... .......... .......... .......... 18% 8.74M 15s 2024-03-30T13:12:47.769 INFO:teuthology.orchestra.run.smithi043.stderr: 31700K .......... .......... .......... .......... .......... 18% 9.28M 15s 2024-03-30T13:12:47.775 INFO:teuthology.orchestra.run.smithi043.stderr: 31750K .......... .......... .......... .......... .......... 18% 8.21M 15s 2024-03-30T13:12:47.780 INFO:teuthology.orchestra.run.smithi043.stderr: 31800K .......... .......... .......... .......... .......... 18% 9.36M 15s 2024-03-30T13:12:47.785 INFO:teuthology.orchestra.run.smithi043.stderr: 31850K .......... .......... .......... .......... .......... 18% 10.2M 15s 2024-03-30T13:12:47.790 INFO:teuthology.orchestra.run.smithi043.stderr: 31900K .......... .......... .......... .......... .......... 18% 8.75M 15s 2024-03-30T13:12:47.796 INFO:teuthology.orchestra.run.smithi043.stderr: 31950K .......... .......... .......... .......... .......... 18% 8.41M 15s 2024-03-30T13:12:47.801 INFO:teuthology.orchestra.run.smithi043.stderr: 32000K .......... .......... .......... .......... .......... 19% 9.33M 15s 2024-03-30T13:12:47.807 INFO:teuthology.orchestra.run.smithi043.stderr: 32050K .......... .......... .......... .......... .......... 19% 8.45M 15s 2024-03-30T13:12:47.813 INFO:teuthology.orchestra.run.smithi043.stderr: 32100K .......... .......... .......... .......... .......... 19% 8.95M 15s 2024-03-30T13:12:47.819 INFO:teuthology.orchestra.run.smithi043.stderr: 32150K .......... .......... .......... .......... .......... 19% 7.38M 15s 2024-03-30T13:12:47.825 INFO:teuthology.orchestra.run.smithi043.stderr: 32200K .......... .......... .......... .......... .......... 19% 8.81M 15s 2024-03-30T13:12:47.831 INFO:teuthology.orchestra.run.smithi043.stderr: 32250K .......... .......... .......... .......... .......... 19% 8.00M 15s 2024-03-30T13:12:47.837 INFO:teuthology.orchestra.run.smithi043.stderr: 32300K .......... .......... .......... .......... .......... 19% 8.00M 15s 2024-03-30T13:12:47.845 INFO:teuthology.orchestra.run.smithi043.stderr: 32350K .......... .......... .......... .......... .......... 19% 6.44M 15s 2024-03-30T13:12:47.851 INFO:teuthology.orchestra.run.smithi043.stderr: 32400K .......... .......... .......... .......... .......... 19% 7.71M 15s 2024-03-30T13:12:47.857 INFO:teuthology.orchestra.run.smithi043.stderr: 32450K .......... .......... .......... .......... .......... 19% 7.61M 15s 2024-03-30T13:12:47.864 INFO:teuthology.orchestra.run.smithi043.stderr: 32500K .......... .......... .......... .......... .......... 19% 7.99M 15s 2024-03-30T13:12:47.871 INFO:teuthology.orchestra.run.smithi043.stderr: 32550K .......... .......... .......... .......... .......... 19% 6.59M 15s 2024-03-30T13:12:47.878 INFO:teuthology.orchestra.run.smithi043.stderr: 32600K .......... .......... .......... .......... .......... 19% 7.57M 15s 2024-03-30T13:12:47.884 INFO:teuthology.orchestra.run.smithi043.stderr: 32650K .......... .......... .......... .......... .......... 19% 7.57M 15s 2024-03-30T13:12:47.890 INFO:teuthology.orchestra.run.smithi043.stderr: 32700K .......... .......... .......... .......... .......... 19% 7.56M 15s 2024-03-30T13:12:47.897 INFO:teuthology.orchestra.run.smithi043.stderr: 32750K .......... .......... .......... .......... .......... 19% 7.50M 15s 2024-03-30T13:12:47.901 INFO:teuthology.orchestra.run.smithi043.stderr: 32800K .......... .......... .......... .......... .......... 19% 11.0M 15s 2024-03-30T13:12:47.905 INFO:teuthology.orchestra.run.smithi043.stderr: 32850K .......... .......... .......... .......... .......... 19% 11.0M 15s 2024-03-30T13:12:47.909 INFO:teuthology.orchestra.run.smithi043.stderr: 32900K .......... .......... .......... .......... .......... 19% 13.1M 15s 2024-03-30T13:12:47.913 INFO:teuthology.orchestra.run.smithi043.stderr: 32950K .......... .......... .......... .......... .......... 19% 13.2M 15s 2024-03-30T13:12:47.917 INFO:teuthology.orchestra.run.smithi043.stderr: 33000K .......... .......... .......... .......... .......... 19% 11.6M 15s 2024-03-30T13:12:47.920 INFO:teuthology.orchestra.run.smithi043.stderr: 33050K .......... .......... .......... .......... .......... 19% 14.1M 15s 2024-03-30T13:12:47.924 INFO:teuthology.orchestra.run.smithi043.stderr: 33100K .......... .......... .......... .......... .......... 19% 13.5M 15s 2024-03-30T13:12:47.927 INFO:teuthology.orchestra.run.smithi043.stderr: 33150K .......... .......... .......... .......... .......... 19% 13.9M 15s 2024-03-30T13:12:47.931 INFO:teuthology.orchestra.run.smithi043.stderr: 33200K .......... .......... .......... .......... .......... 19% 15.7M 15s 2024-03-30T13:12:47.934 INFO:teuthology.orchestra.run.smithi043.stderr: 33250K .......... .......... .......... .......... .......... 19% 15.4M 15s 2024-03-30T13:12:47.937 INFO:teuthology.orchestra.run.smithi043.stderr: 33300K .......... .......... .......... .......... .......... 19% 15.8M 15s 2024-03-30T13:12:47.940 INFO:teuthology.orchestra.run.smithi043.stderr: 33350K .......... .......... .......... .......... .......... 19% 13.8M 15s 2024-03-30T13:12:47.943 INFO:teuthology.orchestra.run.smithi043.stderr: 33400K .......... .......... .......... .......... .......... 19% 15.6M 15s 2024-03-30T13:12:47.947 INFO:teuthology.orchestra.run.smithi043.stderr: 33450K .......... .......... .......... .......... .......... 19% 15.4M 15s 2024-03-30T13:12:47.950 INFO:teuthology.orchestra.run.smithi043.stderr: 33500K .......... .......... .......... .......... .......... 19% 15.4M 15s 2024-03-30T13:12:47.954 INFO:teuthology.orchestra.run.smithi043.stderr: 33550K .......... .......... .......... .......... .......... 19% 13.1M 15s 2024-03-30T13:12:47.957 INFO:teuthology.orchestra.run.smithi043.stderr: 33600K .......... .......... .......... .......... .......... 19% 14.9M 15s 2024-03-30T13:12:47.960 INFO:teuthology.orchestra.run.smithi043.stderr: 33650K .......... .......... .......... .......... .......... 19% 15.2M 15s 2024-03-30T13:12:47.964 INFO:teuthology.orchestra.run.smithi043.stderr: 33700K .......... .......... .......... .......... .......... 20% 14.5M 15s 2024-03-30T13:12:47.967 INFO:teuthology.orchestra.run.smithi043.stderr: 33750K .......... .......... .......... .......... .......... 20% 12.8M 15s 2024-03-30T13:12:47.971 INFO:teuthology.orchestra.run.smithi043.stderr: 33800K .......... .......... .......... .......... .......... 20% 14.6M 15s 2024-03-30T13:12:47.974 INFO:teuthology.orchestra.run.smithi043.stderr: 33850K .......... .......... .......... .......... .......... 20% 14.8M 15s 2024-03-30T13:12:47.977 INFO:teuthology.orchestra.run.smithi043.stderr: 33900K .......... .......... .......... .......... .......... 20% 15.1M 15s 2024-03-30T13:12:47.981 INFO:teuthology.orchestra.run.smithi043.stderr: 33950K .......... .......... .......... .......... .......... 20% 13.8M 15s 2024-03-30T13:12:47.984 INFO:teuthology.orchestra.run.smithi043.stderr: 34000K .......... .......... .......... .......... .......... 20% 15.8M 15s 2024-03-30T13:12:47.987 INFO:teuthology.orchestra.run.smithi043.stderr: 34050K .......... .......... .......... .......... .......... 20% 15.5M 15s 2024-03-30T13:12:47.990 INFO:teuthology.orchestra.run.smithi043.stderr: 34100K .......... .......... .......... .......... .......... 20% 16.1M 15s 2024-03-30T13:12:47.994 INFO:teuthology.orchestra.run.smithi043.stderr: 34150K .......... .......... .......... .......... .......... 20% 13.0M 15s 2024-03-30T13:12:47.997 INFO:teuthology.orchestra.run.smithi043.stderr: 34200K .......... .......... .......... .......... .......... 20% 14.8M 15s 2024-03-30T13:12:48.000 INFO:teuthology.orchestra.run.smithi043.stderr: 34250K .......... .......... .......... .......... .......... 20% 15.5M 15s 2024-03-30T13:12:48.004 INFO:teuthology.orchestra.run.smithi043.stderr: 34300K .......... .......... .......... .......... .......... 20% 14.3M 15s 2024-03-30T13:12:48.008 INFO:teuthology.orchestra.run.smithi043.stderr: 34350K .......... .......... .......... .......... .......... 20% 12.2M 15s 2024-03-30T13:12:48.011 INFO:teuthology.orchestra.run.smithi043.stderr: 34400K .......... .......... .......... .......... .......... 20% 15.0M 15s 2024-03-30T13:12:48.015 INFO:teuthology.orchestra.run.smithi043.stderr: 34450K .......... .......... .......... .......... .......... 20% 15.0M 15s 2024-03-30T13:12:48.017 INFO:teuthology.orchestra.run.smithi043.stderr: 34500K .......... .......... .......... .......... .......... 20% 15.0M 15s 2024-03-30T13:12:48.021 INFO:teuthology.orchestra.run.smithi043.stderr: 34550K .......... .......... .......... .......... .......... 20% 12.2M 15s 2024-03-30T13:12:48.026 INFO:teuthology.orchestra.run.smithi043.stderr: 34600K .......... .......... .......... .......... .......... 20% 12.4M 15s 2024-03-30T13:12:48.029 INFO:teuthology.orchestra.run.smithi043.stderr: 34650K .......... .......... .......... .......... .......... 20% 14.0M 15s 2024-03-30T13:12:48.032 INFO:teuthology.orchestra.run.smithi043.stderr: 34700K .......... .......... .......... .......... .......... 20% 14.3M 15s 2024-03-30T13:12:48.036 INFO:teuthology.orchestra.run.smithi043.stderr: 34750K .......... .......... .......... .......... .......... 20% 11.8M 15s 2024-03-30T13:12:48.040 INFO:teuthology.orchestra.run.smithi043.stderr: 34800K .......... .......... .......... .......... .......... 20% 14.1M 15s 2024-03-30T13:12:48.043 INFO:teuthology.orchestra.run.smithi043.stderr: 34850K .......... .......... .......... .......... .......... 20% 14.2M 15s 2024-03-30T13:12:48.047 INFO:teuthology.orchestra.run.smithi043.stderr: 34900K .......... .......... .......... .......... .......... 20% 13.4M 15s 2024-03-30T13:12:48.051 INFO:teuthology.orchestra.run.smithi043.stderr: 34950K .......... .......... .......... .......... .......... 20% 11.9M 15s 2024-03-30T13:12:48.055 INFO:teuthology.orchestra.run.smithi043.stderr: 35000K .......... .......... .......... .......... .......... 20% 13.9M 15s 2024-03-30T13:12:48.058 INFO:teuthology.orchestra.run.smithi043.stderr: 35050K .......... .......... .......... .......... .......... 20% 13.6M 15s 2024-03-30T13:12:48.062 INFO:teuthology.orchestra.run.smithi043.stderr: 35100K .......... .......... .......... .......... .......... 20% 14.0M 15s 2024-03-30T13:12:48.067 INFO:teuthology.orchestra.run.smithi043.stderr: 35150K .......... .......... .......... .......... .......... 20% 11.3M 15s 2024-03-30T13:12:48.070 INFO:teuthology.orchestra.run.smithi043.stderr: 35200K .......... .......... .......... .......... .......... 20% 12.0M 15s 2024-03-30T13:12:48.075 INFO:teuthology.orchestra.run.smithi043.stderr: 35250K .......... .......... .......... .......... .......... 20% 10.8M 15s 2024-03-30T13:12:48.079 INFO:teuthology.orchestra.run.smithi043.stderr: 35300K .......... .......... .......... .......... .......... 20% 10.9M 15s 2024-03-30T13:12:48.085 INFO:teuthology.orchestra.run.smithi043.stderr: 35350K .......... .......... .......... .......... .......... 21% 9.73M 15s 2024-03-30T13:12:48.089 INFO:teuthology.orchestra.run.smithi043.stderr: 35400K .......... .......... .......... .......... .......... 21% 10.0M 15s 2024-03-30T13:12:48.093 INFO:teuthology.orchestra.run.smithi043.stderr: 35450K .......... .......... .......... .......... .......... 21% 11.5M 15s 2024-03-30T13:12:48.098 INFO:teuthology.orchestra.run.smithi043.stderr: 35500K .......... .......... .......... .......... .......... 21% 11.0M 15s 2024-03-30T13:12:48.104 INFO:teuthology.orchestra.run.smithi043.stderr: 35550K .......... .......... .......... .......... .......... 21% 8.78M 15s 2024-03-30T13:12:48.108 INFO:teuthology.orchestra.run.smithi043.stderr: 35600K .......... .......... .......... .......... .......... 21% 10.1M 15s 2024-03-30T13:12:48.113 INFO:teuthology.orchestra.run.smithi043.stderr: 35650K .......... .......... .......... .......... .......... 21% 10.5M 15s 2024-03-30T13:12:48.118 INFO:teuthology.orchestra.run.smithi043.stderr: 35700K .......... .......... .......... .......... .......... 21% 9.64M 15s 2024-03-30T13:12:48.124 INFO:teuthology.orchestra.run.smithi043.stderr: 35750K .......... .......... .......... .......... .......... 21% 8.16M 15s 2024-03-30T13:12:48.129 INFO:teuthology.orchestra.run.smithi043.stderr: 35800K .......... .......... .......... .......... .......... 21% 9.37M 15s 2024-03-30T13:12:48.134 INFO:teuthology.orchestra.run.smithi043.stderr: 35850K .......... .......... .......... .......... .......... 21% 9.91M 15s 2024-03-30T13:12:48.139 INFO:teuthology.orchestra.run.smithi043.stderr: 35900K .......... .......... .......... .......... .......... 21% 9.76M 15s 2024-03-30T13:12:48.146 INFO:teuthology.orchestra.run.smithi043.stderr: 35950K .......... .......... .......... .......... .......... 21% 7.29M 15s 2024-03-30T13:12:48.151 INFO:teuthology.orchestra.run.smithi043.stderr: 36000K .......... .......... .......... .......... .......... 21% 9.13M 15s 2024-03-30T13:12:48.158 INFO:teuthology.orchestra.run.smithi043.stderr: 36050K .......... .......... .......... .......... .......... 21% 7.72M 14s 2024-03-30T13:12:48.164 INFO:teuthology.orchestra.run.smithi043.stderr: 36100K .......... .......... .......... .......... .......... 21% 7.60M 14s 2024-03-30T13:12:48.171 INFO:teuthology.orchestra.run.smithi043.stderr: 36150K .......... .......... .......... .......... .......... 21% 6.75M 14s 2024-03-30T13:12:48.177 INFO:teuthology.orchestra.run.smithi043.stderr: 36200K .......... .......... .......... .......... .......... 21% 8.39M 14s 2024-03-30T13:12:48.184 INFO:teuthology.orchestra.run.smithi043.stderr: 36250K .......... .......... .......... .......... .......... 21% 7.17M 14s 2024-03-30T13:12:48.191 INFO:teuthology.orchestra.run.smithi043.stderr: 36300K .......... .......... .......... .......... .......... 21% 7.19M 14s 2024-03-30T13:12:48.199 INFO:teuthology.orchestra.run.smithi043.stderr: 36350K .......... .......... .......... .......... .......... 21% 6.18M 14s 2024-03-30T13:12:48.206 INFO:teuthology.orchestra.run.smithi043.stderr: 36400K .......... .......... .......... .......... .......... 21% 6.58M 14s 2024-03-30T13:12:48.214 INFO:teuthology.orchestra.run.smithi043.stderr: 36450K .......... .......... .......... .......... .......... 21% 6.70M 14s 2024-03-30T13:12:48.220 INFO:teuthology.orchestra.run.smithi043.stderr: 36500K .......... .......... .......... .......... .......... 21% 7.09M 14s 2024-03-30T13:12:48.228 INFO:teuthology.orchestra.run.smithi043.stderr: 36550K .......... .......... .......... .......... .......... 21% 6.36M 15s 2024-03-30T13:12:48.235 INFO:teuthology.orchestra.run.smithi043.stderr: 36600K .......... .......... .......... .......... .......... 21% 7.09M 15s 2024-03-30T13:12:48.242 INFO:teuthology.orchestra.run.smithi043.stderr: 36650K .......... .......... .......... .......... .......... 21% 6.93M 15s 2024-03-30T13:12:48.249 INFO:teuthology.orchestra.run.smithi043.stderr: 36700K .......... .......... .......... .......... .......... 21% 6.89M 15s 2024-03-30T13:12:48.257 INFO:teuthology.orchestra.run.smithi043.stderr: 36750K .......... .......... .......... .......... .......... 21% 6.06M 15s 2024-03-30T13:12:48.264 INFO:teuthology.orchestra.run.smithi043.stderr: 36800K .......... .......... .......... .......... .......... 21% 7.03M 15s 2024-03-30T13:12:48.271 INFO:teuthology.orchestra.run.smithi043.stderr: 36850K .......... .......... .......... .......... .......... 21% 7.10M 15s 2024-03-30T13:12:48.277 INFO:teuthology.orchestra.run.smithi043.stderr: 36900K .......... .......... .......... .......... .......... 21% 7.88M 15s 2024-03-30T13:12:48.282 INFO:teuthology.orchestra.run.smithi043.stderr: 36950K .......... .......... .......... .......... .......... 21% 9.15M 14s 2024-03-30T13:12:48.287 INFO:teuthology.orchestra.run.smithi043.stderr: 37000K .......... .......... .......... .......... .......... 21% 10.8M 14s 2024-03-30T13:12:48.292 INFO:teuthology.orchestra.run.smithi043.stderr: 37050K .......... .......... .......... .......... .......... 22% 9.70M 14s 2024-03-30T13:12:48.297 INFO:teuthology.orchestra.run.smithi043.stderr: 37100K .......... .......... .......... .......... .......... 22% 9.34M 14s 2024-03-30T13:12:48.303 INFO:teuthology.orchestra.run.smithi043.stderr: 37150K .......... .......... .......... .......... .......... 22% 8.39M 14s 2024-03-30T13:12:48.308 INFO:teuthology.orchestra.run.smithi043.stderr: 37200K .......... .......... .......... .......... .......... 22% 9.79M 14s 2024-03-30T13:12:48.313 INFO:teuthology.orchestra.run.smithi043.stderr: 37250K .......... .......... .......... .......... .......... 22% 9.06M 14s 2024-03-30T13:12:48.318 INFO:teuthology.orchestra.run.smithi043.stderr: 37300K .......... .......... .......... .......... .......... 22% 9.63M 14s 2024-03-30T13:12:48.324 INFO:teuthology.orchestra.run.smithi043.stderr: 37350K .......... .......... .......... .......... .......... 22% 9.17M 14s 2024-03-30T13:12:48.329 INFO:teuthology.orchestra.run.smithi043.stderr: 37400K .......... .......... .......... .......... .......... 22% 9.87M 14s 2024-03-30T13:12:48.334 INFO:teuthology.orchestra.run.smithi043.stderr: 37450K .......... .......... .......... .......... .......... 22% 8.99M 14s 2024-03-30T13:12:48.339 INFO:teuthology.orchestra.run.smithi043.stderr: 37500K .......... .......... .......... .......... .......... 22% 9.16M 14s 2024-03-30T13:12:48.345 INFO:teuthology.orchestra.run.smithi043.stderr: 37550K .......... .......... .......... .......... .......... 22% 8.22M 14s 2024-03-30T13:12:48.351 INFO:teuthology.orchestra.run.smithi043.stderr: 37600K .......... .......... .......... .......... .......... 22% 8.79M 14s 2024-03-30T13:12:48.356 INFO:teuthology.orchestra.run.smithi043.stderr: 37650K .......... .......... .......... .......... .......... 22% 9.06M 14s 2024-03-30T13:12:48.361 INFO:teuthology.orchestra.run.smithi043.stderr: 37700K .......... .......... .......... .......... .......... 22% 10.1M 14s 2024-03-30T13:12:48.367 INFO:teuthology.orchestra.run.smithi043.stderr: 37750K .......... .......... .......... .......... .......... 22% 8.18M 14s 2024-03-30T13:12:48.373 INFO:teuthology.orchestra.run.smithi043.stderr: 37800K .......... .......... .......... .......... .......... 22% 8.91M 14s 2024-03-30T13:12:48.378 INFO:teuthology.orchestra.run.smithi043.stderr: 37850K .......... .......... .......... .......... .......... 22% 8.84M 14s 2024-03-30T13:12:48.384 INFO:teuthology.orchestra.run.smithi043.stderr: 37900K .......... .......... .......... .......... .......... 22% 8.56M 14s 2024-03-30T13:12:48.391 INFO:teuthology.orchestra.run.smithi043.stderr: 37950K .......... .......... .......... .......... .......... 22% 7.20M 14s 2024-03-30T13:12:48.396 INFO:teuthology.orchestra.run.smithi043.stderr: 38000K .......... .......... .......... .......... .......... 22% 8.51M 14s 2024-03-30T13:12:48.402 INFO:teuthology.orchestra.run.smithi043.stderr: 38050K .......... .......... .......... .......... .......... 22% 9.20M 14s 2024-03-30T13:12:48.408 INFO:teuthology.orchestra.run.smithi043.stderr: 38100K .......... .......... .......... .......... .......... 22% 8.43M 14s 2024-03-30T13:12:48.414 INFO:teuthology.orchestra.run.smithi043.stderr: 38150K .......... .......... .......... .......... .......... 22% 7.41M 14s 2024-03-30T13:12:48.419 INFO:teuthology.orchestra.run.smithi043.stderr: 38200K .......... .......... .......... .......... .......... 22% 9.55M 14s 2024-03-30T13:12:48.424 INFO:teuthology.orchestra.run.smithi043.stderr: 38250K .......... .......... .......... .......... .......... 22% 9.28M 14s 2024-03-30T13:12:48.430 INFO:teuthology.orchestra.run.smithi043.stderr: 38300K .......... .......... .......... .......... .......... 22% 8.47M 14s 2024-03-30T13:12:48.437 INFO:teuthology.orchestra.run.smithi043.stderr: 38350K .......... .......... .......... .......... .......... 22% 7.18M 14s 2024-03-30T13:12:48.443 INFO:teuthology.orchestra.run.smithi043.stderr: 38400K .......... .......... .......... .......... .......... 22% 8.17M 14s 2024-03-30T13:12:48.449 INFO:teuthology.orchestra.run.smithi043.stderr: 38450K .......... .......... .......... .......... .......... 22% 8.73M 14s 2024-03-30T13:12:48.455 INFO:teuthology.orchestra.run.smithi043.stderr: 38500K .......... .......... .......... .......... .......... 22% 7.98M 14s 2024-03-30T13:12:48.462 INFO:teuthology.orchestra.run.smithi043.stderr: 38550K .......... .......... .......... .......... .......... 22% 6.99M 14s 2024-03-30T13:12:48.468 INFO:teuthology.orchestra.run.smithi043.stderr: 38600K .......... .......... .......... .......... .......... 22% 7.56M 14s 2024-03-30T13:12:48.474 INFO:teuthology.orchestra.run.smithi043.stderr: 38650K .......... .......... .......... .......... .......... 22% 8.09M 14s 2024-03-30T13:12:48.480 INFO:teuthology.orchestra.run.smithi043.stderr: 38700K .......... .......... .......... .......... .......... 22% 7.73M 14s 2024-03-30T13:12:48.487 INFO:teuthology.orchestra.run.smithi043.stderr: 38750K .......... .......... .......... .......... .......... 23% 7.40M 14s 2024-03-30T13:12:48.494 INFO:teuthology.orchestra.run.smithi043.stderr: 38800K .......... .......... .......... .......... .......... 23% 8.06M 14s 2024-03-30T13:12:48.499 INFO:teuthology.orchestra.run.smithi043.stderr: 38850K .......... .......... .......... .......... .......... 23% 8.21M 14s 2024-03-30T13:12:48.505 INFO:teuthology.orchestra.run.smithi043.stderr: 38900K .......... .......... .......... .......... .......... 23% 8.16M 14s 2024-03-30T13:12:48.512 INFO:teuthology.orchestra.run.smithi043.stderr: 38950K .......... .......... .......... .......... .......... 23% 6.76M 14s 2024-03-30T13:12:48.518 INFO:teuthology.orchestra.run.smithi043.stderr: 39000K .......... .......... .......... .......... .......... 23% 7.90M 14s 2024-03-30T13:12:48.525 INFO:teuthology.orchestra.run.smithi043.stderr: 39050K .......... .......... .......... .......... .......... 23% 7.90M 14s 2024-03-30T13:12:48.530 INFO:teuthology.orchestra.run.smithi043.stderr: 39100K .......... .......... .......... .......... .......... 23% 8.65M 14s 2024-03-30T13:12:48.537 INFO:teuthology.orchestra.run.smithi043.stderr: 39150K .......... .......... .......... .......... .......... 23% 7.15M 14s 2024-03-30T13:12:48.543 INFO:teuthology.orchestra.run.smithi043.stderr: 39200K .......... .......... .......... .......... .......... 23% 8.06M 14s 2024-03-30T13:12:48.549 INFO:teuthology.orchestra.run.smithi043.stderr: 39250K .......... .......... .......... .......... .......... 23% 7.89M 14s 2024-03-30T13:12:48.556 INFO:teuthology.orchestra.run.smithi043.stderr: 39300K .......... .......... .......... .......... .......... 23% 8.00M 14s 2024-03-30T13:12:48.563 INFO:teuthology.orchestra.run.smithi043.stderr: 39350K .......... .......... .......... .......... .......... 23% 6.80M 14s 2024-03-30T13:12:48.569 INFO:teuthology.orchestra.run.smithi043.stderr: 39400K .......... .......... .......... .......... .......... 23% 7.97M 14s 2024-03-30T13:12:48.575 INFO:teuthology.orchestra.run.smithi043.stderr: 39450K .......... .......... .......... .......... .......... 23% 8.25M 14s 2024-03-30T13:12:48.580 INFO:teuthology.orchestra.run.smithi043.stderr: 39500K .......... .......... .......... .......... .......... 23% 8.39M 14s 2024-03-30T13:12:48.587 INFO:teuthology.orchestra.run.smithi043.stderr: 39550K .......... .......... .......... .......... .......... 23% 7.33M 14s 2024-03-30T13:12:48.593 INFO:teuthology.orchestra.run.smithi043.stderr: 39600K .......... .......... .......... .......... .......... 23% 8.80M 14s 2024-03-30T13:12:48.598 INFO:teuthology.orchestra.run.smithi043.stderr: 39650K .......... .......... .......... .......... .......... 23% 8.83M 14s 2024-03-30T13:12:48.605 INFO:teuthology.orchestra.run.smithi043.stderr: 39700K .......... .......... .......... .......... .......... 23% 8.33M 14s 2024-03-30T13:12:48.611 INFO:teuthology.orchestra.run.smithi043.stderr: 39750K .......... .......... .......... .......... .......... 23% 6.95M 14s 2024-03-30T13:12:48.618 INFO:teuthology.orchestra.run.smithi043.stderr: 39800K .......... .......... .......... .......... .......... 23% 7.55M 14s 2024-03-30T13:12:48.624 INFO:teuthology.orchestra.run.smithi043.stderr: 39850K .......... .......... .......... .......... .......... 23% 7.64M 14s 2024-03-30T13:12:48.631 INFO:teuthology.orchestra.run.smithi043.stderr: 39900K .......... .......... .......... .......... .......... 23% 7.21M 14s 2024-03-30T13:12:48.639 INFO:teuthology.orchestra.run.smithi043.stderr: 39950K .......... .......... .......... .......... .......... 23% 6.15M 14s 2024-03-30T13:12:48.646 INFO:teuthology.orchestra.run.smithi043.stderr: 40000K .......... .......... .......... .......... .......... 23% 7.19M 14s 2024-03-30T13:12:48.652 INFO:teuthology.orchestra.run.smithi043.stderr: 40050K .......... .......... .......... .......... .......... 23% 7.28M 14s 2024-03-30T13:12:48.659 INFO:teuthology.orchestra.run.smithi043.stderr: 40100K .......... .......... .......... .......... .......... 23% 7.43M 14s 2024-03-30T13:12:48.666 INFO:teuthology.orchestra.run.smithi043.stderr: 40150K .......... .......... .......... .......... .......... 23% 7.07M 14s 2024-03-30T13:12:48.671 INFO:teuthology.orchestra.run.smithi043.stderr: 40200K .......... .......... .......... .......... .......... 23% 8.91M 14s 2024-03-30T13:12:48.676 INFO:teuthology.orchestra.run.smithi043.stderr: 40250K .......... .......... .......... .......... .......... 23% 9.27M 14s 2024-03-30T13:12:48.681 INFO:teuthology.orchestra.run.smithi043.stderr: 40300K .......... .......... .......... .......... .......... 23% 11.8M 14s 2024-03-30T13:12:48.687 INFO:teuthology.orchestra.run.smithi043.stderr: 40350K .......... .......... .......... .......... .......... 23% 7.56M 14s 2024-03-30T13:12:48.693 INFO:teuthology.orchestra.run.smithi043.stderr: 40400K .......... .......... .......... .......... .......... 23% 8.16M 14s 2024-03-30T13:12:48.699 INFO:teuthology.orchestra.run.smithi043.stderr: 40450K .......... .......... .......... .......... .......... 24% 8.04M 14s 2024-03-30T13:12:48.705 INFO:teuthology.orchestra.run.smithi043.stderr: 40500K .......... .......... .......... .......... .......... 24% 8.19M 14s 2024-03-30T13:12:48.712 INFO:teuthology.orchestra.run.smithi043.stderr: 40550K .......... .......... .......... .......... .......... 24% 7.20M 14s 2024-03-30T13:12:48.718 INFO:teuthology.orchestra.run.smithi043.stderr: 40600K .......... .......... .......... .......... .......... 24% 8.23M 14s 2024-03-30T13:12:48.724 INFO:teuthology.orchestra.run.smithi043.stderr: 40650K .......... .......... .......... .......... .......... 24% 8.08M 14s 2024-03-30T13:12:48.730 INFO:teuthology.orchestra.run.smithi043.stderr: 40700K .......... .......... .......... .......... .......... 24% 7.81M 14s 2024-03-30T13:12:48.737 INFO:teuthology.orchestra.run.smithi043.stderr: 40750K .......... .......... .......... .......... .......... 24% 6.67M 14s 2024-03-30T13:12:48.746 INFO:teuthology.orchestra.run.smithi043.stderr: 40800K .......... .......... .......... .......... .......... 24% 8.23M 14s 2024-03-30T13:12:48.750 INFO:teuthology.orchestra.run.smithi043.stderr: 40850K .......... .......... .......... .......... .......... 24% 7.33M 14s 2024-03-30T13:12:48.756 INFO:teuthology.orchestra.run.smithi043.stderr: 40900K .......... .......... .......... .......... .......... 24% 8.58M 14s 2024-03-30T13:12:48.762 INFO:teuthology.orchestra.run.smithi043.stderr: 40950K .......... .......... .......... .......... .......... 24% 7.30M 14s 2024-03-30T13:12:48.765 INFO:teuthology.orchestra.run.smithi043.stderr: 41000K .......... .......... .......... .......... .......... 24% 13.8M 14s 2024-03-30T13:12:48.769 INFO:teuthology.orchestra.run.smithi043.stderr: 41050K .......... .......... .......... .......... .......... 24% 12.9M 14s 2024-03-30T13:12:48.774 INFO:teuthology.orchestra.run.smithi043.stderr: 41100K .......... .......... .......... .......... .......... 24% 13.2M 14s 2024-03-30T13:12:48.779 INFO:teuthology.orchestra.run.smithi043.stderr: 41150K .......... .......... .......... .......... .......... 24% 7.57M 14s 2024-03-30T13:12:48.783 INFO:teuthology.orchestra.run.smithi043.stderr: 41200K .......... .......... .......... .......... .......... 24% 13.0M 14s 2024-03-30T13:12:48.787 INFO:teuthology.orchestra.run.smithi043.stderr: 41250K .......... .......... .......... .......... .......... 24% 14.5M 14s 2024-03-30T13:12:48.790 INFO:teuthology.orchestra.run.smithi043.stderr: 41300K .......... .......... .......... .......... .......... 24% 13.1M 14s 2024-03-30T13:12:48.794 INFO:teuthology.orchestra.run.smithi043.stderr: 41350K .......... .......... .......... .......... .......... 24% 13.0M 14s 2024-03-30T13:12:48.797 INFO:teuthology.orchestra.run.smithi043.stderr: 41400K .......... .......... .......... .......... .......... 24% 15.5M 14s 2024-03-30T13:12:48.800 INFO:teuthology.orchestra.run.smithi043.stderr: 41450K .......... .......... .......... .......... .......... 24% 17.7M 14s 2024-03-30T13:12:48.803 INFO:teuthology.orchestra.run.smithi043.stderr: 41500K .......... .......... .......... .......... .......... 24% 17.1M 14s 2024-03-30T13:12:48.806 INFO:teuthology.orchestra.run.smithi043.stderr: 41550K .......... .......... .......... .......... .......... 24% 14.5M 14s 2024-03-30T13:12:48.809 INFO:teuthology.orchestra.run.smithi043.stderr: 41600K .......... .......... .......... .......... .......... 24% 15.1M 14s 2024-03-30T13:12:48.814 INFO:teuthology.orchestra.run.smithi043.stderr: 41650K .......... .......... .......... .......... .......... 24% 15.1M 14s 2024-03-30T13:12:48.815 INFO:teuthology.orchestra.run.smithi043.stderr: 41700K .......... .......... .......... .......... .......... 24% 16.8M 14s 2024-03-30T13:12:48.819 INFO:teuthology.orchestra.run.smithi043.stderr: 41750K .......... .......... .......... .......... .......... 24% 13.3M 14s 2024-03-30T13:12:48.822 INFO:teuthology.orchestra.run.smithi043.stderr: 41800K .......... .......... .......... .......... .......... 24% 15.4M 14s 2024-03-30T13:12:48.825 INFO:teuthology.orchestra.run.smithi043.stderr: 41850K .......... .......... .......... .......... .......... 24% 18.7M 14s 2024-03-30T13:12:48.829 INFO:teuthology.orchestra.run.smithi043.stderr: 41900K .......... .......... .......... .......... .......... 24% 13.3M 14s 2024-03-30T13:12:48.833 INFO:teuthology.orchestra.run.smithi043.stderr: 41950K .......... .......... .......... .......... .......... 24% 12.1M 14s 2024-03-30T13:12:48.837 INFO:teuthology.orchestra.run.smithi043.stderr: 42000K .......... .......... .......... .......... .......... 24% 12.1M 14s 2024-03-30T13:12:48.841 INFO:teuthology.orchestra.run.smithi043.stderr: 42050K .......... .......... .......... .......... .......... 24% 11.5M 14s 2024-03-30T13:12:48.845 INFO:teuthology.orchestra.run.smithi043.stderr: 42100K .......... .......... .......... .......... .......... 25% 12.0M 14s 2024-03-30T13:12:48.850 INFO:teuthology.orchestra.run.smithi043.stderr: 42150K .......... .......... .......... .......... .......... 25% 10.3M 14s 2024-03-30T13:12:48.854 INFO:teuthology.orchestra.run.smithi043.stderr: 42200K .......... .......... .......... .......... .......... 25% 12.2M 14s 2024-03-30T13:12:48.858 INFO:teuthology.orchestra.run.smithi043.stderr: 42250K .......... .......... .......... .......... .......... 25% 11.9M 14s 2024-03-30T13:12:48.862 INFO:teuthology.orchestra.run.smithi043.stderr: 42300K .......... .......... .......... .......... .......... 25% 12.9M 14s 2024-03-30T13:12:48.867 INFO:teuthology.orchestra.run.smithi043.stderr: 42350K .......... .......... .......... .......... .......... 25% 9.62M 14s 2024-03-30T13:12:48.871 INFO:teuthology.orchestra.run.smithi043.stderr: 42400K .......... .......... .......... .......... .......... 25% 11.9M 14s 2024-03-30T13:12:48.874 INFO:teuthology.orchestra.run.smithi043.stderr: 42450K .......... .......... .......... .......... .......... 25% 15.3M 14s 2024-03-30T13:12:48.877 INFO:teuthology.orchestra.run.smithi043.stderr: 42500K .......... .......... .......... .......... .......... 25% 18.3M 14s 2024-03-30T13:12:48.881 INFO:teuthology.orchestra.run.smithi043.stderr: 42550K .......... .......... .......... .......... .......... 25% 10.3M 14s 2024-03-30T13:12:48.885 INFO:teuthology.orchestra.run.smithi043.stderr: 42600K .......... .......... .......... .......... .......... 25% 13.2M 14s 2024-03-30T13:12:48.889 INFO:teuthology.orchestra.run.smithi043.stderr: 42650K .......... .......... .......... .......... .......... 25% 12.0M 14s 2024-03-30T13:12:48.894 INFO:teuthology.orchestra.run.smithi043.stderr: 42700K .......... .......... .......... .......... .......... 25% 10.6M 14s 2024-03-30T13:12:48.899 INFO:teuthology.orchestra.run.smithi043.stderr: 42750K .......... .......... .......... .......... .......... 25% 9.15M 14s 2024-03-30T13:12:48.903 INFO:teuthology.orchestra.run.smithi043.stderr: 42800K .......... .......... .......... .......... .......... 25% 11.3M 14s 2024-03-30T13:12:48.908 INFO:teuthology.orchestra.run.smithi043.stderr: 42850K .......... .......... .......... .......... .......... 25% 10.6M 14s 2024-03-30T13:12:48.913 INFO:teuthology.orchestra.run.smithi043.stderr: 42900K .......... .......... .......... .......... .......... 25% 10.8M 14s 2024-03-30T13:12:48.918 INFO:teuthology.orchestra.run.smithi043.stderr: 42950K .......... .......... .......... .......... .......... 25% 9.64M 14s 2024-03-30T13:12:48.922 INFO:teuthology.orchestra.run.smithi043.stderr: 43000K .......... .......... .......... .......... .......... 25% 10.5M 14s 2024-03-30T13:12:48.927 INFO:teuthology.orchestra.run.smithi043.stderr: 43050K .......... .......... .......... .......... .......... 25% 11.1M 14s 2024-03-30T13:12:48.932 INFO:teuthology.orchestra.run.smithi043.stderr: 43100K .......... .......... .......... .......... .......... 25% 10.3M 14s 2024-03-30T13:12:48.937 INFO:teuthology.orchestra.run.smithi043.stderr: 43150K .......... .......... .......... .......... .......... 25% 8.67M 14s 2024-03-30T13:12:48.943 INFO:teuthology.orchestra.run.smithi043.stderr: 43200K .......... .......... .......... .......... .......... 25% 8.84M 14s 2024-03-30T13:12:48.948 INFO:teuthology.orchestra.run.smithi043.stderr: 43250K .......... .......... .......... .......... .......... 25% 8.85M 14s 2024-03-30T13:12:48.954 INFO:teuthology.orchestra.run.smithi043.stderr: 43300K .......... .......... .......... .......... .......... 25% 8.61M 14s 2024-03-30T13:12:48.961 INFO:teuthology.orchestra.run.smithi043.stderr: 43350K .......... .......... .......... .......... .......... 25% 7.43M 14s 2024-03-30T13:12:48.967 INFO:teuthology.orchestra.run.smithi043.stderr: 43400K .......... .......... .......... .......... .......... 25% 8.30M 14s 2024-03-30T13:12:48.972 INFO:teuthology.orchestra.run.smithi043.stderr: 43450K .......... .......... .......... .......... .......... 25% 8.51M 14s 2024-03-30T13:12:48.978 INFO:teuthology.orchestra.run.smithi043.stderr: 43500K .......... .......... .......... .......... .......... 25% 8.44M 14s 2024-03-30T13:12:48.985 INFO:teuthology.orchestra.run.smithi043.stderr: 43550K .......... .......... .......... .......... .......... 25% 6.86M 14s 2024-03-30T13:12:48.991 INFO:teuthology.orchestra.run.smithi043.stderr: 43600K .......... .......... .......... .......... .......... 25% 8.26M 14s 2024-03-30T13:12:48.997 INFO:teuthology.orchestra.run.smithi043.stderr: 43650K .......... .......... .......... .......... .......... 25% 8.43M 14s 2024-03-30T13:12:49.003 INFO:teuthology.orchestra.run.smithi043.stderr: 43700K .......... .......... .......... .......... .......... 25% 8.11M 14s 2024-03-30T13:12:49.010 INFO:teuthology.orchestra.run.smithi043.stderr: 43750K .......... .......... .......... .......... .......... 25% 6.44M 14s 2024-03-30T13:12:49.017 INFO:teuthology.orchestra.run.smithi043.stderr: 43800K .......... .......... .......... .......... .......... 26% 7.84M 14s 2024-03-30T13:12:49.024 INFO:teuthology.orchestra.run.smithi043.stderr: 43850K .......... .......... .......... .......... .......... 26% 7.57M 14s 2024-03-30T13:12:49.029 INFO:teuthology.orchestra.run.smithi043.stderr: 43900K .......... .......... .......... .......... .......... 26% 8.31M 14s 2024-03-30T13:12:49.037 INFO:teuthology.orchestra.run.smithi043.stderr: 43950K .......... .......... .......... .......... .......... 26% 5.95M 14s 2024-03-30T13:12:49.045 INFO:teuthology.orchestra.run.smithi043.stderr: 44000K .......... .......... .......... .......... .......... 26% 6.83M 14s 2024-03-30T13:12:49.052 INFO:teuthology.orchestra.run.smithi043.stderr: 44050K .......... .......... .......... .......... .......... 26% 6.92M 14s 2024-03-30T13:12:49.059 INFO:teuthology.orchestra.run.smithi043.stderr: 44100K .......... .......... .......... .......... .......... 26% 6.92M 14s 2024-03-30T13:12:49.067 INFO:teuthology.orchestra.run.smithi043.stderr: 44150K .......... .......... .......... .......... .......... 26% 5.95M 14s 2024-03-30T13:12:49.074 INFO:teuthology.orchestra.run.smithi043.stderr: 44200K .......... .......... .......... .......... .......... 26% 6.89M 14s 2024-03-30T13:12:49.081 INFO:teuthology.orchestra.run.smithi043.stderr: 44250K .......... .......... .......... .......... .......... 26% 7.08M 14s 2024-03-30T13:12:49.088 INFO:teuthology.orchestra.run.smithi043.stderr: 44300K .......... .......... .......... .......... .......... 26% 7.08M 14s 2024-03-30T13:12:49.097 INFO:teuthology.orchestra.run.smithi043.stderr: 44350K .......... .......... .......... .......... .......... 26% 5.41M 14s 2024-03-30T13:12:49.104 INFO:teuthology.orchestra.run.smithi043.stderr: 44400K .......... .......... .......... .......... .......... 26% 6.64M 14s 2024-03-30T13:12:49.110 INFO:teuthology.orchestra.run.smithi043.stderr: 44450K .......... .......... .......... .......... .......... 26% 7.43M 14s 2024-03-30T13:12:49.117 INFO:teuthology.orchestra.run.smithi043.stderr: 44500K .......... .......... .......... .......... .......... 26% 7.57M 14s 2024-03-30T13:12:49.124 INFO:teuthology.orchestra.run.smithi043.stderr: 44550K .......... .......... .......... .......... .......... 26% 6.81M 14s 2024-03-30T13:12:49.131 INFO:teuthology.orchestra.run.smithi043.stderr: 44600K .......... .......... .......... .......... .......... 26% 7.30M 14s 2024-03-30T13:12:49.137 INFO:teuthology.orchestra.run.smithi043.stderr: 44650K .......... .......... .......... .......... .......... 26% 8.13M 14s 2024-03-30T13:12:49.143 INFO:teuthology.orchestra.run.smithi043.stderr: 44700K .......... .......... .......... .......... .......... 26% 7.91M 14s 2024-03-30T13:12:49.150 INFO:teuthology.orchestra.run.smithi043.stderr: 44750K .......... .......... .......... .......... .......... 26% 7.00M 14s 2024-03-30T13:12:49.156 INFO:teuthology.orchestra.run.smithi043.stderr: 44800K .......... .......... .......... .......... .......... 26% 7.46M 14s 2024-03-30T13:12:49.163 INFO:teuthology.orchestra.run.smithi043.stderr: 44850K .......... .......... .......... .......... .......... 26% 7.85M 14s 2024-03-30T13:12:49.169 INFO:teuthology.orchestra.run.smithi043.stderr: 44900K .......... .......... .......... .......... .......... 26% 7.58M 14s 2024-03-30T13:12:49.177 INFO:teuthology.orchestra.run.smithi043.stderr: 44950K .......... .......... .......... .......... .......... 26% 6.16M 14s 2024-03-30T13:12:49.183 INFO:teuthology.orchestra.run.smithi043.stderr: 45000K .......... .......... .......... .......... .......... 26% 7.65M 14s 2024-03-30T13:12:49.189 INFO:teuthology.orchestra.run.smithi043.stderr: 45050K .......... .......... .......... .......... .......... 26% 8.64M 14s 2024-03-30T13:12:49.194 INFO:teuthology.orchestra.run.smithi043.stderr: 45100K .......... .......... .......... .......... .......... 26% 9.13M 14s 2024-03-30T13:12:49.200 INFO:teuthology.orchestra.run.smithi043.stderr: 45150K .......... .......... .......... .......... .......... 26% 9.45M 14s 2024-03-30T13:12:49.205 INFO:teuthology.orchestra.run.smithi043.stderr: 45200K .......... .......... .......... .......... .......... 26% 9.11M 14s 2024-03-30T13:12:49.209 INFO:teuthology.orchestra.run.smithi043.stderr: 45250K .......... .......... .......... .......... .......... 26% 12.3M 14s 2024-03-30T13:12:49.213 INFO:teuthology.orchestra.run.smithi043.stderr: 45300K .......... .......... .......... .......... .......... 26% 11.4M 14s 2024-03-30T13:12:49.219 INFO:teuthology.orchestra.run.smithi043.stderr: 45350K .......... .......... .......... .......... .......... 26% 9.82M 14s 2024-03-30T13:12:49.235 INFO:teuthology.orchestra.run.smithi043.stderr: 45400K .......... .......... .......... .......... .......... 26% 11.4M 14s 2024-03-30T13:12:49.235 INFO:teuthology.orchestra.run.smithi043.stderr: 45450K .......... .......... .......... .......... .......... 26% 11.5M 14s 2024-03-30T13:12:49.235 INFO:teuthology.orchestra.run.smithi043.stderr: 45500K .......... .......... .......... .......... .......... 27% 12.0M 14s 2024-03-30T13:12:49.236 INFO:teuthology.orchestra.run.smithi043.stderr: 45550K .......... .......... .......... .......... .......... 27% 9.22M 14s 2024-03-30T13:12:49.241 INFO:teuthology.orchestra.run.smithi043.stderr: 45600K .......... .......... .......... .......... .......... 27% 9.92M 14s 2024-03-30T13:12:49.246 INFO:teuthology.orchestra.run.smithi043.stderr: 45650K .......... .......... .......... .......... .......... 27% 9.61M 14s 2024-03-30T13:12:49.251 INFO:teuthology.orchestra.run.smithi043.stderr: 45700K .......... .......... .......... .......... .......... 27% 9.19M 14s 2024-03-30T13:12:49.258 INFO:teuthology.orchestra.run.smithi043.stderr: 45750K .......... .......... .......... .......... .......... 27% 7.87M 14s 2024-03-30T13:12:49.263 INFO:teuthology.orchestra.run.smithi043.stderr: 45800K .......... .......... .......... .......... .......... 27% 9.22M 14s 2024-03-30T13:12:49.268 INFO:teuthology.orchestra.run.smithi043.stderr: 45850K .......... .......... .......... .......... .......... 27% 9.44M 14s 2024-03-30T13:12:49.274 INFO:teuthology.orchestra.run.smithi043.stderr: 45900K .......... .......... .......... .......... .......... 27% 9.34M 14s 2024-03-30T13:12:49.280 INFO:teuthology.orchestra.run.smithi043.stderr: 45950K .......... .......... .......... .......... .......... 27% 7.36M 14s 2024-03-30T13:12:49.284 INFO:teuthology.orchestra.run.smithi043.stderr: 46000K .......... .......... .......... .......... .......... 27% 11.3M 14s 2024-03-30T13:12:49.288 INFO:teuthology.orchestra.run.smithi043.stderr: 46050K .......... .......... .......... .......... .......... 27% 11.1M 14s 2024-03-30T13:12:49.293 INFO:teuthology.orchestra.run.smithi043.stderr: 46100K .......... .......... .......... .......... .......... 27% 11.7M 13s 2024-03-30T13:12:49.298 INFO:teuthology.orchestra.run.smithi043.stderr: 46150K .......... .......... .......... .......... .......... 27% 9.30M 13s 2024-03-30T13:12:49.302 INFO:teuthology.orchestra.run.smithi043.stderr: 46200K .......... .......... .......... .......... .......... 27% 10.8M 13s 2024-03-30T13:12:49.307 INFO:teuthology.orchestra.run.smithi043.stderr: 46250K .......... .......... .......... .......... .......... 27% 10.6M 13s 2024-03-30T13:12:49.311 INFO:teuthology.orchestra.run.smithi043.stderr: 46300K .......... .......... .......... .......... .......... 27% 11.2M 13s 2024-03-30T13:12:49.317 INFO:teuthology.orchestra.run.smithi043.stderr: 46350K .......... .......... .......... .......... .......... 27% 9.46M 13s 2024-03-30T13:12:49.321 INFO:teuthology.orchestra.run.smithi043.stderr: 46400K .......... .......... .......... .......... .......... 27% 10.8M 13s 2024-03-30T13:12:49.326 INFO:teuthology.orchestra.run.smithi043.stderr: 46450K .......... .......... .......... .......... .......... 27% 10.5M 13s 2024-03-30T13:12:49.330 INFO:teuthology.orchestra.run.smithi043.stderr: 46500K .......... .......... .......... .......... .......... 27% 10.6M 13s 2024-03-30T13:12:49.336 INFO:teuthology.orchestra.run.smithi043.stderr: 46550K .......... .......... .......... .......... .......... 27% 8.88M 13s 2024-03-30T13:12:49.341 INFO:teuthology.orchestra.run.smithi043.stderr: 46600K .......... .......... .......... .......... .......... 27% 9.91M 13s 2024-03-30T13:12:49.346 INFO:teuthology.orchestra.run.smithi043.stderr: 46650K .......... .......... .......... .......... .......... 27% 9.81M 13s 2024-03-30T13:12:49.351 INFO:teuthology.orchestra.run.smithi043.stderr: 46700K .......... .......... .......... .......... .......... 27% 10.0M 13s 2024-03-30T13:12:49.356 INFO:teuthology.orchestra.run.smithi043.stderr: 46750K .......... .......... .......... .......... .......... 27% 8.78M 13s 2024-03-30T13:12:49.361 INFO:teuthology.orchestra.run.smithi043.stderr: 46800K .......... .......... .......... .......... .......... 27% 9.68M 13s 2024-03-30T13:12:49.366 INFO:teuthology.orchestra.run.smithi043.stderr: 46850K .......... .......... .......... .......... .......... 27% 9.61M 13s 2024-03-30T13:12:49.372 INFO:teuthology.orchestra.run.smithi043.stderr: 46900K .......... .......... .......... .......... .......... 27% 9.36M 13s 2024-03-30T13:12:49.378 INFO:teuthology.orchestra.run.smithi043.stderr: 46950K .......... .......... .......... .......... .......... 27% 7.97M 13s 2024-03-30T13:12:49.382 INFO:teuthology.orchestra.run.smithi043.stderr: 47000K .......... .......... .......... .......... .......... 27% 10.2M 13s 2024-03-30T13:12:49.388 INFO:teuthology.orchestra.run.smithi043.stderr: 47050K .......... .......... .......... .......... .......... 27% 10.2M 13s 2024-03-30T13:12:49.393 INFO:teuthology.orchestra.run.smithi043.stderr: 47100K .......... .......... .......... .......... .......... 27% 9.03M 13s 2024-03-30T13:12:49.399 INFO:teuthology.orchestra.run.smithi043.stderr: 47150K .......... .......... .......... .......... .......... 28% 7.77M 13s 2024-03-30T13:12:49.404 INFO:teuthology.orchestra.run.smithi043.stderr: 47200K .......... .......... .......... .......... .......... 28% 9.03M 13s 2024-03-30T13:12:49.410 INFO:teuthology.orchestra.run.smithi043.stderr: 47250K .......... .......... .......... .......... .......... 28% 8.93M 13s 2024-03-30T13:12:49.415 INFO:teuthology.orchestra.run.smithi043.stderr: 47300K .......... .......... .......... .......... .......... 28% 8.94M 13s 2024-03-30T13:12:49.422 INFO:teuthology.orchestra.run.smithi043.stderr: 47350K .......... .......... .......... .......... .......... 28% 7.74M 13s 2024-03-30T13:12:49.430 INFO:teuthology.orchestra.run.smithi043.stderr: 47400K .......... .......... .......... .......... .......... 28% 9.24M 13s 2024-03-30T13:12:49.433 INFO:teuthology.orchestra.run.smithi043.stderr: 47450K .......... .......... .......... .......... .......... 28% 9.61M 13s 2024-03-30T13:12:49.438 INFO:teuthology.orchestra.run.smithi043.stderr: 47500K .......... .......... .......... .......... .......... 28% 8.48M 13s 2024-03-30T13:12:49.445 INFO:teuthology.orchestra.run.smithi043.stderr: 47550K .......... .......... .......... .......... .......... 28% 6.79M 13s 2024-03-30T13:12:49.451 INFO:teuthology.orchestra.run.smithi043.stderr: 47600K .......... .......... .......... .......... .......... 28% 7.79M 13s 2024-03-30T13:12:49.458 INFO:teuthology.orchestra.run.smithi043.stderr: 47650K .......... .......... .......... .......... .......... 28% 7.86M 13s 2024-03-30T13:12:49.464 INFO:teuthology.orchestra.run.smithi043.stderr: 47700K .......... .......... .......... .......... .......... 28% 7.42M 13s 2024-03-30T13:12:49.472 INFO:teuthology.orchestra.run.smithi043.stderr: 47750K .......... .......... .......... .......... .......... 28% 6.14M 13s 2024-03-30T13:12:49.479 INFO:teuthology.orchestra.run.smithi043.stderr: 47800K .......... .......... .......... .......... .......... 28% 7.14M 13s 2024-03-30T13:12:49.486 INFO:teuthology.orchestra.run.smithi043.stderr: 47850K .......... .......... .......... .......... .......... 28% 7.34M 13s 2024-03-30T13:12:49.493 INFO:teuthology.orchestra.run.smithi043.stderr: 47900K .......... .......... .......... .......... .......... 28% 7.03M 13s 2024-03-30T13:12:49.501 INFO:teuthology.orchestra.run.smithi043.stderr: 47950K .......... .......... .......... .......... .......... 28% 6.18M 13s 2024-03-30T13:12:49.509 INFO:teuthology.orchestra.run.smithi043.stderr: 48000K .......... .......... .......... .......... .......... 28% 7.15M 13s 2024-03-30T13:12:49.514 INFO:teuthology.orchestra.run.smithi043.stderr: 48050K .......... .......... .......... .......... .......... 28% 7.05M 13s 2024-03-30T13:12:49.521 INFO:teuthology.orchestra.run.smithi043.stderr: 48100K .......... .......... .......... .......... .......... 28% 7.18M 13s 2024-03-30T13:12:49.529 INFO:teuthology.orchestra.run.smithi043.stderr: 48150K .......... .......... .......... .......... .......... 28% 6.39M 13s 2024-03-30T13:12:49.536 INFO:teuthology.orchestra.run.smithi043.stderr: 48200K .......... .......... .......... .......... .......... 28% 7.14M 13s 2024-03-30T13:12:49.543 INFO:teuthology.orchestra.run.smithi043.stderr: 48250K .......... .......... .......... .......... .......... 28% 6.60M 13s 2024-03-30T13:12:49.550 INFO:teuthology.orchestra.run.smithi043.stderr: 48300K .......... .......... .......... .......... .......... 28% 6.78M 13s 2024-03-30T13:12:49.558 INFO:teuthology.orchestra.run.smithi043.stderr: 48350K .......... .......... .......... .......... .......... 28% 6.06M 13s 2024-03-30T13:12:49.565 INFO:teuthology.orchestra.run.smithi043.stderr: 48400K .......... .......... .......... .......... .......... 28% 7.38M 13s 2024-03-30T13:12:49.571 INFO:teuthology.orchestra.run.smithi043.stderr: 48450K .......... .......... .......... .......... .......... 28% 7.28M 13s 2024-03-30T13:12:49.578 INFO:teuthology.orchestra.run.smithi043.stderr: 48500K .......... .......... .......... .......... .......... 28% 7.56M 13s 2024-03-30T13:12:49.586 INFO:teuthology.orchestra.run.smithi043.stderr: 48550K .......... .......... .......... .......... .......... 28% 6.38M 13s 2024-03-30T13:12:49.592 INFO:teuthology.orchestra.run.smithi043.stderr: 48600K .......... .......... .......... .......... .......... 28% 7.35M 13s 2024-03-30T13:12:49.599 INFO:teuthology.orchestra.run.smithi043.stderr: 48650K .......... .......... .......... .......... .......... 28% 7.43M 13s 2024-03-30T13:12:49.606 INFO:teuthology.orchestra.run.smithi043.stderr: 48700K .......... .......... .......... .......... .......... 28% 7.22M 13s 2024-03-30T13:12:49.614 INFO:teuthology.orchestra.run.smithi043.stderr: 48750K .......... .......... .......... .......... .......... 28% 6.12M 13s 2024-03-30T13:12:49.621 INFO:teuthology.orchestra.run.smithi043.stderr: 48800K .......... .......... .......... .......... .......... 28% 7.23M 13s 2024-03-30T13:12:49.628 INFO:teuthology.orchestra.run.smithi043.stderr: 48850K .......... .......... .......... .......... .......... 29% 6.99M 13s 2024-03-30T13:12:49.635 INFO:teuthology.orchestra.run.smithi043.stderr: 48900K .......... .......... .......... .......... .......... 29% 6.38M 13s 2024-03-30T13:12:49.644 INFO:teuthology.orchestra.run.smithi043.stderr: 48950K .......... .......... .......... .......... .......... 29% 5.56M 13s 2024-03-30T13:12:49.651 INFO:teuthology.orchestra.run.smithi043.stderr: 49000K .......... .......... .......... .......... .......... 29% 6.58M 13s 2024-03-30T13:12:49.659 INFO:teuthology.orchestra.run.smithi043.stderr: 49050K .......... .......... .......... .......... .......... 29% 6.15M 13s 2024-03-30T13:12:49.667 INFO:teuthology.orchestra.run.smithi043.stderr: 49100K .......... .......... .......... .......... .......... 29% 6.29M 13s 2024-03-30T13:12:49.674 INFO:teuthology.orchestra.run.smithi043.stderr: 49150K .......... .......... .......... .......... .......... 29% 7.05M 13s 2024-03-30T13:12:49.679 INFO:teuthology.orchestra.run.smithi043.stderr: 49200K .......... .......... .......... .......... .......... 29% 8.47M 13s 2024-03-30T13:12:49.685 INFO:teuthology.orchestra.run.smithi043.stderr: 49250K .......... .......... .......... .......... .......... 29% 9.21M 13s 2024-03-30T13:12:49.691 INFO:teuthology.orchestra.run.smithi043.stderr: 49300K .......... .......... .......... .......... .......... 29% 7.60M 13s 2024-03-30T13:12:49.697 INFO:teuthology.orchestra.run.smithi043.stderr: 49350K .......... .......... .......... .......... .......... 29% 8.62M 13s 2024-03-30T13:12:49.702 INFO:teuthology.orchestra.run.smithi043.stderr: 49400K .......... .......... .......... .......... .......... 29% 9.96M 13s 2024-03-30T13:12:49.707 INFO:teuthology.orchestra.run.smithi043.stderr: 49450K .......... .......... .......... .......... .......... 29% 9.69M 13s 2024-03-30T13:12:49.712 INFO:teuthology.orchestra.run.smithi043.stderr: 49500K .......... .......... .......... .......... .......... 29% 9.72M 13s 2024-03-30T13:12:49.718 INFO:teuthology.orchestra.run.smithi043.stderr: 49550K .......... .......... .......... .......... .......... 29% 8.20M 13s 2024-03-30T13:12:49.723 INFO:teuthology.orchestra.run.smithi043.stderr: 49600K .......... .......... .......... .......... .......... 29% 8.77M 13s 2024-03-30T13:12:49.728 INFO:teuthology.orchestra.run.smithi043.stderr: 49650K .......... .......... .......... .......... .......... 29% 10.6M 13s 2024-03-30T13:12:49.733 INFO:teuthology.orchestra.run.smithi043.stderr: 49700K .......... .......... .......... .......... .......... 29% 10.7M 13s 2024-03-30T13:12:49.738 INFO:teuthology.orchestra.run.smithi043.stderr: 49750K .......... .......... .......... .......... .......... 29% 8.50M 13s 2024-03-30T13:12:49.743 INFO:teuthology.orchestra.run.smithi043.stderr: 49800K .......... .......... .......... .......... .......... 29% 10.3M 13s 2024-03-30T13:12:49.748 INFO:teuthology.orchestra.run.smithi043.stderr: 49850K .......... .......... .......... .......... .......... 29% 10.3M 13s 2024-03-30T13:12:49.752 INFO:teuthology.orchestra.run.smithi043.stderr: 49900K .......... .......... .......... .......... .......... 29% 10.6M 13s 2024-03-30T13:12:49.758 INFO:teuthology.orchestra.run.smithi043.stderr: 49950K .......... .......... .......... .......... .......... 29% 8.44M 13s 2024-03-30T13:12:49.763 INFO:teuthology.orchestra.run.smithi043.stderr: 50000K .......... .......... .......... .......... .......... 29% 10.1M 13s 2024-03-30T13:12:49.768 INFO:teuthology.orchestra.run.smithi043.stderr: 50050K .......... .......... .......... .......... .......... 29% 10.1M 13s 2024-03-30T13:12:49.773 INFO:teuthology.orchestra.run.smithi043.stderr: 50100K .......... .......... .......... .......... .......... 29% 9.86M 13s 2024-03-30T13:12:49.779 INFO:teuthology.orchestra.run.smithi043.stderr: 50150K .......... .......... .......... .......... .......... 29% 7.54M 13s 2024-03-30T13:12:49.785 INFO:teuthology.orchestra.run.smithi043.stderr: 50200K .......... .......... .......... .......... .......... 29% 8.62M 13s 2024-03-30T13:12:49.791 INFO:teuthology.orchestra.run.smithi043.stderr: 50250K .......... .......... .......... .......... .......... 29% 8.66M 13s 2024-03-30T13:12:49.796 INFO:teuthology.orchestra.run.smithi043.stderr: 50300K .......... .......... .......... .......... .......... 29% 9.09M 13s 2024-03-30T13:12:49.803 INFO:teuthology.orchestra.run.smithi043.stderr: 50350K .......... .......... .......... .......... .......... 29% 7.19M 13s 2024-03-30T13:12:49.808 INFO:teuthology.orchestra.run.smithi043.stderr: 50400K .......... .......... .......... .......... .......... 29% 8.71M 13s 2024-03-30T13:12:49.814 INFO:teuthology.orchestra.run.smithi043.stderr: 50450K .......... .......... .......... .......... .......... 29% 8.90M 13s 2024-03-30T13:12:49.820 INFO:teuthology.orchestra.run.smithi043.stderr: 50500K .......... .......... .......... .......... .......... 29% 8.31M 13s 2024-03-30T13:12:49.827 INFO:teuthology.orchestra.run.smithi043.stderr: 50550K .......... .......... .......... .......... .......... 30% 7.03M 13s 2024-03-30T13:12:49.833 INFO:teuthology.orchestra.run.smithi043.stderr: 50600K .......... .......... .......... .......... .......... 30% 8.11M 13s 2024-03-30T13:12:49.839 INFO:teuthology.orchestra.run.smithi043.stderr: 50650K .......... .......... .......... .......... .......... 30% 8.13M 13s 2024-03-30T13:12:49.845 INFO:teuthology.orchestra.run.smithi043.stderr: 50700K .......... .......... .......... .......... .......... 30% 8.09M 13s 2024-03-30T13:12:49.852 INFO:teuthology.orchestra.run.smithi043.stderr: 50750K .......... .......... .......... .......... .......... 30% 6.98M 13s 2024-03-30T13:12:49.858 INFO:teuthology.orchestra.run.smithi043.stderr: 50800K .......... .......... .......... .......... .......... 30% 7.84M 13s 2024-03-30T13:12:49.864 INFO:teuthology.orchestra.run.smithi043.stderr: 50850K .......... .......... .......... .......... .......... 30% 8.39M 13s 2024-03-30T13:12:49.870 INFO:teuthology.orchestra.run.smithi043.stderr: 50900K .......... .......... .......... .......... .......... 30% 8.33M 13s 2024-03-30T13:12:49.877 INFO:teuthology.orchestra.run.smithi043.stderr: 50950K .......... .......... .......... .......... .......... 30% 6.94M 13s 2024-03-30T13:12:49.882 INFO:teuthology.orchestra.run.smithi043.stderr: 51000K .......... .......... .......... .......... .......... 30% 8.68M 13s 2024-03-30T13:12:49.888 INFO:teuthology.orchestra.run.smithi043.stderr: 51050K .......... .......... .......... .......... .......... 30% 8.36M 13s 2024-03-30T13:12:49.894 INFO:teuthology.orchestra.run.smithi043.stderr: 51100K .......... .......... .......... .......... .......... 30% 8.77M 13s 2024-03-30T13:12:49.900 INFO:teuthology.orchestra.run.smithi043.stderr: 51150K .......... .......... .......... .......... .......... 30% 7.19M 13s 2024-03-30T13:12:49.906 INFO:teuthology.orchestra.run.smithi043.stderr: 51200K .......... .......... .......... .......... .......... 30% 8.50M 13s 2024-03-30T13:12:49.912 INFO:teuthology.orchestra.run.smithi043.stderr: 51250K .......... .......... .......... .......... .......... 30% 7.81M 13s 2024-03-30T13:12:49.918 INFO:teuthology.orchestra.run.smithi043.stderr: 51300K .......... .......... .......... .......... .......... 30% 8.28M 13s 2024-03-30T13:12:49.926 INFO:teuthology.orchestra.run.smithi043.stderr: 51350K .......... .......... .......... .......... .......... 30% 6.78M 13s 2024-03-30T13:12:49.932 INFO:teuthology.orchestra.run.smithi043.stderr: 51400K .......... .......... .......... .......... .......... 30% 8.01M 13s 2024-03-30T13:12:49.938 INFO:teuthology.orchestra.run.smithi043.stderr: 51450K .......... .......... .......... .......... .......... 30% 7.79M 13s 2024-03-30T13:12:49.944 INFO:teuthology.orchestra.run.smithi043.stderr: 51500K .......... .......... .......... .......... .......... 30% 8.09M 13s 2024-03-30T13:12:49.951 INFO:teuthology.orchestra.run.smithi043.stderr: 51550K .......... .......... .......... .......... .......... 30% 7.05M 13s 2024-03-30T13:12:49.957 INFO:teuthology.orchestra.run.smithi043.stderr: 51600K .......... .......... .......... .......... .......... 30% 7.67M 13s 2024-03-30T13:12:49.964 INFO:teuthology.orchestra.run.smithi043.stderr: 51650K .......... .......... .......... .......... .......... 30% 7.59M 13s 2024-03-30T13:12:49.971 INFO:teuthology.orchestra.run.smithi043.stderr: 51700K .......... .......... .......... .......... .......... 30% 7.46M 13s 2024-03-30T13:12:49.978 INFO:teuthology.orchestra.run.smithi043.stderr: 51750K .......... .......... .......... .......... .......... 30% 6.35M 13s 2024-03-30T13:12:49.985 INFO:teuthology.orchestra.run.smithi043.stderr: 51800K .......... .......... .......... .......... .......... 30% 7.43M 13s 2024-03-30T13:12:49.991 INFO:teuthology.orchestra.run.smithi043.stderr: 51850K .......... .......... .......... .......... .......... 30% 7.31M 13s 2024-03-30T13:12:49.998 INFO:teuthology.orchestra.run.smithi043.stderr: 51900K .......... .......... .......... .......... .......... 30% 7.13M 13s 2024-03-30T13:12:50.005 INFO:teuthology.orchestra.run.smithi043.stderr: 51950K .......... .......... .......... .......... .......... 30% 6.37M 13s 2024-03-30T13:12:50.012 INFO:teuthology.orchestra.run.smithi043.stderr: 52000K .......... .......... .......... .......... .......... 30% 8.15M 13s 2024-03-30T13:12:50.017 INFO:teuthology.orchestra.run.smithi043.stderr: 52050K .......... .......... .......... .......... .......... 30% 8.26M 13s 2024-03-30T13:12:50.024 INFO:teuthology.orchestra.run.smithi043.stderr: 52100K .......... .......... .......... .......... .......... 30% 7.70M 13s 2024-03-30T13:12:50.031 INFO:teuthology.orchestra.run.smithi043.stderr: 52150K .......... .......... .......... .......... .......... 30% 6.63M 13s 2024-03-30T13:12:50.038 INFO:teuthology.orchestra.run.smithi043.stderr: 52200K .......... .......... .......... .......... .......... 30% 7.73M 13s 2024-03-30T13:12:50.045 INFO:teuthology.orchestra.run.smithi043.stderr: 52250K .......... .......... .......... .......... .......... 31% 7.25M 13s 2024-03-30T13:12:50.051 INFO:teuthology.orchestra.run.smithi043.stderr: 52300K .......... .......... .......... .......... .......... 31% 7.35M 13s 2024-03-30T13:12:50.059 INFO:teuthology.orchestra.run.smithi043.stderr: 52350K .......... .......... .......... .......... .......... 31% 6.33M 13s 2024-03-30T13:12:50.066 INFO:teuthology.orchestra.run.smithi043.stderr: 52400K .......... .......... .......... .......... .......... 31% 7.20M 13s 2024-03-30T13:12:50.072 INFO:teuthology.orchestra.run.smithi043.stderr: 52450K .......... .......... .......... .......... .......... 31% 7.49M 13s 2024-03-30T13:12:50.078 INFO:teuthology.orchestra.run.smithi043.stderr: 52500K .......... .......... .......... .......... .......... 31% 7.89M 13s 2024-03-30T13:12:50.086 INFO:teuthology.orchestra.run.smithi043.stderr: 52550K .......... .......... .......... .......... .......... 31% 6.84M 13s 2024-03-30T13:12:50.091 INFO:teuthology.orchestra.run.smithi043.stderr: 52600K .......... .......... .......... .......... .......... 31% 7.98M 13s 2024-03-30T13:12:50.097 INFO:teuthology.orchestra.run.smithi043.stderr: 52650K .......... .......... .......... .......... .......... 31% 8.11M 13s 2024-03-30T13:12:50.104 INFO:teuthology.orchestra.run.smithi043.stderr: 52700K .......... .......... .......... .......... .......... 31% 8.26M 13s 2024-03-30T13:12:50.111 INFO:teuthology.orchestra.run.smithi043.stderr: 52750K .......... .......... .......... .......... .......... 31% 6.42M 13s 2024-03-30T13:12:50.118 INFO:teuthology.orchestra.run.smithi043.stderr: 52800K .......... .......... .......... .......... .......... 31% 7.89M 13s 2024-03-30T13:12:50.123 INFO:teuthology.orchestra.run.smithi043.stderr: 52850K .......... .......... .......... .......... .......... 31% 8.26M 13s 2024-03-30T13:12:50.129 INFO:teuthology.orchestra.run.smithi043.stderr: 52900K .......... .......... .......... .......... .......... 31% 9.08M 13s 2024-03-30T13:12:50.135 INFO:teuthology.orchestra.run.smithi043.stderr: 52950K .......... .......... .......... .......... .......... 31% 7.29M 13s 2024-03-30T13:12:50.140 INFO:teuthology.orchestra.run.smithi043.stderr: 53000K .......... .......... .......... .......... .......... 31% 8.92M 13s 2024-03-30T13:12:50.146 INFO:teuthology.orchestra.run.smithi043.stderr: 53050K .......... .......... .......... .......... .......... 31% 8.87M 13s 2024-03-30T13:12:50.151 INFO:teuthology.orchestra.run.smithi043.stderr: 53100K .......... .......... .......... .......... .......... 31% 9.07M 13s 2024-03-30T13:12:50.158 INFO:teuthology.orchestra.run.smithi043.stderr: 53150K .......... .......... .......... .......... .......... 31% 7.83M 13s 2024-03-30T13:12:50.163 INFO:teuthology.orchestra.run.smithi043.stderr: 53200K .......... .......... .......... .......... .......... 31% 8.99M 13s 2024-03-30T13:12:50.168 INFO:teuthology.orchestra.run.smithi043.stderr: 53250K .......... .......... .......... .......... .......... 31% 9.30M 13s 2024-03-30T13:12:50.171 INFO:teuthology.orchestra.run.smithi043.stderr: 53300K .......... .......... .......... .......... .......... 31% 16.3M 13s 2024-03-30T13:12:50.175 INFO:teuthology.orchestra.run.smithi043.stderr: 53350K .......... .......... .......... .......... .......... 31% 13.3M 13s 2024-03-30T13:12:50.178 INFO:teuthology.orchestra.run.smithi043.stderr: 53400K .......... .......... .......... .......... .......... 31% 15.8M 13s 2024-03-30T13:12:50.181 INFO:teuthology.orchestra.run.smithi043.stderr: 53450K .......... .......... .......... .......... .......... 31% 14.1M 13s 2024-03-30T13:12:50.185 INFO:teuthology.orchestra.run.smithi043.stderr: 53500K .......... .......... .......... .......... .......... 31% 13.3M 13s 2024-03-30T13:12:50.189 INFO:teuthology.orchestra.run.smithi043.stderr: 53550K .......... .......... .......... .......... .......... 31% 13.0M 13s 2024-03-30T13:12:50.192 INFO:teuthology.orchestra.run.smithi043.stderr: 53600K .......... .......... .......... .......... .......... 31% 14.8M 13s 2024-03-30T13:12:50.195 INFO:teuthology.orchestra.run.smithi043.stderr: 53650K .......... .......... .......... .......... .......... 31% 15.5M 13s 2024-03-30T13:12:50.198 INFO:teuthology.orchestra.run.smithi043.stderr: 53700K .......... .......... .......... .......... .......... 31% 15.1M 13s 2024-03-30T13:12:50.202 INFO:teuthology.orchestra.run.smithi043.stderr: 53750K .......... .......... .......... .......... .......... 31% 12.9M 13s 2024-03-30T13:12:50.205 INFO:teuthology.orchestra.run.smithi043.stderr: 53800K .......... .......... .......... .......... .......... 31% 14.5M 13s 2024-03-30T13:12:50.209 INFO:teuthology.orchestra.run.smithi043.stderr: 53850K .......... .......... .......... .......... .......... 31% 14.3M 13s 2024-03-30T13:12:50.212 INFO:teuthology.orchestra.run.smithi043.stderr: 53900K .......... .......... .......... .......... .......... 32% 14.0M 13s 2024-03-30T13:12:50.217 INFO:teuthology.orchestra.run.smithi043.stderr: 53950K .......... .......... .......... .......... .......... 32% 11.5M 13s 2024-03-30T13:12:50.220 INFO:teuthology.orchestra.run.smithi043.stderr: 54000K .......... .......... .......... .......... .......... 32% 13.7M 13s 2024-03-30T13:12:50.224 INFO:teuthology.orchestra.run.smithi043.stderr: 54050K .......... .......... .......... .......... .......... 32% 13.3M 13s 2024-03-30T13:12:50.228 INFO:teuthology.orchestra.run.smithi043.stderr: 54100K .......... .......... .......... .......... .......... 32% 12.5M 13s 2024-03-30T13:12:50.232 INFO:teuthology.orchestra.run.smithi043.stderr: 54150K .......... .......... .......... .......... .......... 32% 10.9M 13s 2024-03-30T13:12:50.236 INFO:teuthology.orchestra.run.smithi043.stderr: 54200K .......... .......... .......... .......... .......... 32% 13.6M 13s 2024-03-30T13:12:50.239 INFO:teuthology.orchestra.run.smithi043.stderr: 54250K .......... .......... .......... .......... .......... 32% 13.9M 13s 2024-03-30T13:12:50.243 INFO:teuthology.orchestra.run.smithi043.stderr: 54300K .......... .......... .......... .......... .......... 32% 13.5M 13s 2024-03-30T13:12:50.247 INFO:teuthology.orchestra.run.smithi043.stderr: 54350K .......... .......... .......... .......... .......... 32% 10.9M 13s 2024-03-30T13:12:50.253 INFO:teuthology.orchestra.run.smithi043.stderr: 54400K .......... .......... .......... .......... .......... 32% 13.1M 13s 2024-03-30T13:12:50.255 INFO:teuthology.orchestra.run.smithi043.stderr: 54450K .......... .......... .......... .......... .......... 32% 12.9M 13s 2024-03-30T13:12:50.259 INFO:teuthology.orchestra.run.smithi043.stderr: 54500K .......... .......... .......... .......... .......... 32% 13.4M 13s 2024-03-30T13:12:50.263 INFO:teuthology.orchestra.run.smithi043.stderr: 54550K .......... .......... .......... .......... .......... 32% 11.3M 13s 2024-03-30T13:12:50.267 INFO:teuthology.orchestra.run.smithi043.stderr: 54600K .......... .......... .......... .......... .......... 32% 12.3M 13s 2024-03-30T13:12:50.271 INFO:teuthology.orchestra.run.smithi043.stderr: 54650K .......... .......... .......... .......... .......... 32% 12.9M 13s 2024-03-30T13:12:50.274 INFO:teuthology.orchestra.run.smithi043.stderr: 54700K .......... .......... .......... .......... .......... 32% 12.8M 13s 2024-03-30T13:12:50.280 INFO:teuthology.orchestra.run.smithi043.stderr: 54750K .......... .......... .......... .......... .......... 32% 9.86M 13s 2024-03-30T13:12:50.284 INFO:teuthology.orchestra.run.smithi043.stderr: 54800K .......... .......... .......... .......... .......... 32% 10.9M 13s 2024-03-30T13:12:50.288 INFO:teuthology.orchestra.run.smithi043.stderr: 54850K .......... .......... .......... .......... .......... 32% 11.2M 13s 2024-03-30T13:12:50.293 INFO:teuthology.orchestra.run.smithi043.stderr: 54900K .......... .......... .......... .......... .......... 32% 10.7M 13s 2024-03-30T13:12:50.298 INFO:teuthology.orchestra.run.smithi043.stderr: 54950K .......... .......... .......... .......... .......... 32% 9.32M 13s 2024-03-30T13:12:50.303 INFO:teuthology.orchestra.run.smithi043.stderr: 55000K .......... .......... .......... .......... .......... 32% 11.2M 13s 2024-03-30T13:12:50.308 INFO:teuthology.orchestra.run.smithi043.stderr: 55050K .......... .......... .......... .......... .......... 32% 10.5M 13s 2024-03-30T13:12:50.312 INFO:teuthology.orchestra.run.smithi043.stderr: 55100K .......... .......... .......... .......... .......... 32% 9.73M 13s 2024-03-30T13:12:50.318 INFO:teuthology.orchestra.run.smithi043.stderr: 55150K .......... .......... .......... .......... .......... 32% 8.45M 13s 2024-03-30T13:12:50.323 INFO:teuthology.orchestra.run.smithi043.stderr: 55200K .......... .......... .......... .......... .......... 32% 10.1M 13s 2024-03-30T13:12:50.327 INFO:teuthology.orchestra.run.smithi043.stderr: 55250K .......... .......... .......... .......... .......... 32% 11.0M 13s 2024-03-30T13:12:50.332 INFO:teuthology.orchestra.run.smithi043.stderr: 55300K .......... .......... .......... .......... .......... 32% 10.6M 13s 2024-03-30T13:12:50.337 INFO:teuthology.orchestra.run.smithi043.stderr: 55350K .......... .......... .......... .......... .......... 32% 9.01M 13s 2024-03-30T13:12:50.342 INFO:teuthology.orchestra.run.smithi043.stderr: 55400K .......... .......... .......... .......... .......... 32% 10.7M 13s 2024-03-30T13:12:50.347 INFO:teuthology.orchestra.run.smithi043.stderr: 55450K .......... .......... .......... .......... .......... 32% 10.6M 13s 2024-03-30T13:12:50.351 INFO:teuthology.orchestra.run.smithi043.stderr: 55500K .......... .......... .......... .......... .......... 32% 10.6M 13s 2024-03-30T13:12:50.357 INFO:teuthology.orchestra.run.smithi043.stderr: 55550K .......... .......... .......... .......... .......... 32% 8.74M 12s 2024-03-30T13:12:50.362 INFO:teuthology.orchestra.run.smithi043.stderr: 55600K .......... .......... .......... .......... .......... 33% 10.2M 12s 2024-03-30T13:12:50.366 INFO:teuthology.orchestra.run.smithi043.stderr: 55650K .......... .......... .......... .......... .......... 33% 10.2M 12s 2024-03-30T13:12:50.371 INFO:teuthology.orchestra.run.smithi043.stderr: 55700K .......... .......... .......... .......... .......... 33% 9.99M 12s 2024-03-30T13:12:50.377 INFO:teuthology.orchestra.run.smithi043.stderr: 55750K .......... .......... .......... .......... .......... 33% 9.07M 12s 2024-03-30T13:12:50.382 INFO:teuthology.orchestra.run.smithi043.stderr: 55800K .......... .......... .......... .......... .......... 33% 9.57M 12s 2024-03-30T13:12:50.387 INFO:teuthology.orchestra.run.smithi043.stderr: 55850K .......... .......... .......... .......... .......... 33% 9.12M 12s 2024-03-30T13:12:50.393 INFO:teuthology.orchestra.run.smithi043.stderr: 55900K .......... .......... .......... .......... .......... 33% 8.57M 12s 2024-03-30T13:12:50.399 INFO:teuthology.orchestra.run.smithi043.stderr: 55950K .......... .......... .......... .......... .......... 33% 7.35M 12s 2024-03-30T13:12:50.405 INFO:teuthology.orchestra.run.smithi043.stderr: 56000K .......... .......... .......... .......... .......... 33% 9.68M 12s 2024-03-30T13:12:50.410 INFO:teuthology.orchestra.run.smithi043.stderr: 56050K .......... .......... .......... .......... .......... 33% 8.60M 12s 2024-03-30T13:12:50.416 INFO:teuthology.orchestra.run.smithi043.stderr: 56100K .......... .......... .......... .......... .......... 33% 8.66M 12s 2024-03-30T13:12:50.422 INFO:teuthology.orchestra.run.smithi043.stderr: 56150K .......... .......... .......... .......... .......... 33% 7.54M 12s 2024-03-30T13:12:50.428 INFO:teuthology.orchestra.run.smithi043.stderr: 56200K .......... .......... .......... .......... .......... 33% 8.72M 12s 2024-03-30T13:12:50.433 INFO:teuthology.orchestra.run.smithi043.stderr: 56250K .......... .......... .......... .......... .......... 33% 8.68M 12s 2024-03-30T13:12:50.439 INFO:teuthology.orchestra.run.smithi043.stderr: 56300K .......... .......... .......... .......... .......... 33% 9.62M 12s 2024-03-30T13:12:50.444 INFO:teuthology.orchestra.run.smithi043.stderr: 56350K .......... .......... .......... .......... .......... 33% 8.34M 12s 2024-03-30T13:12:50.450 INFO:teuthology.orchestra.run.smithi043.stderr: 56400K .......... .......... .......... .......... .......... 33% 9.80M 12s 2024-03-30T13:12:50.455 INFO:teuthology.orchestra.run.smithi043.stderr: 56450K .......... .......... .......... .......... .......... 33% 9.47M 12s 2024-03-30T13:12:50.460 INFO:teuthology.orchestra.run.smithi043.stderr: 56500K .......... .......... .......... .......... .......... 33% 9.47M 12s 2024-03-30T13:12:50.466 INFO:teuthology.orchestra.run.smithi043.stderr: 56550K .......... .......... .......... .......... .......... 33% 8.12M 12s 2024-03-30T13:12:50.471 INFO:teuthology.orchestra.run.smithi043.stderr: 56600K .......... .......... .......... .......... .......... 33% 10.3M 12s 2024-03-30T13:12:50.476 INFO:teuthology.orchestra.run.smithi043.stderr: 56650K .......... .......... .......... .......... .......... 33% 9.39M 12s 2024-03-30T13:12:50.481 INFO:teuthology.orchestra.run.smithi043.stderr: 56700K .......... .......... .......... .......... .......... 33% 9.03M 12s 2024-03-30T13:12:50.488 INFO:teuthology.orchestra.run.smithi043.stderr: 56750K .......... .......... .......... .......... .......... 33% 7.76M 12s 2024-03-30T13:12:50.493 INFO:teuthology.orchestra.run.smithi043.stderr: 56800K .......... .......... .......... .......... .......... 33% 8.96M 12s 2024-03-30T13:12:50.499 INFO:teuthology.orchestra.run.smithi043.stderr: 56850K .......... .......... .......... .......... .......... 33% 8.54M 12s 2024-03-30T13:12:50.504 INFO:teuthology.orchestra.run.smithi043.stderr: 56900K .......... .......... .......... .......... .......... 33% 8.99M 12s 2024-03-30T13:12:50.511 INFO:teuthology.orchestra.run.smithi043.stderr: 56950K .......... .......... .......... .......... .......... 33% 7.67M 12s 2024-03-30T13:12:50.516 INFO:teuthology.orchestra.run.smithi043.stderr: 57000K .......... .......... .......... .......... .......... 33% 8.90M 12s 2024-03-30T13:12:50.523 INFO:teuthology.orchestra.run.smithi043.stderr: 57050K .......... .......... .......... .......... .......... 33% 8.59M 12s 2024-03-30T13:12:50.527 INFO:teuthology.orchestra.run.smithi043.stderr: 57100K .......... .......... .......... .......... .......... 33% 9.22M 12s 2024-03-30T13:12:50.533 INFO:teuthology.orchestra.run.smithi043.stderr: 57150K .......... .......... .......... .......... .......... 33% 8.15M 12s 2024-03-30T13:12:50.539 INFO:teuthology.orchestra.run.smithi043.stderr: 57200K .......... .......... .......... .......... .......... 33% 8.46M 12s 2024-03-30T13:12:50.545 INFO:teuthology.orchestra.run.smithi043.stderr: 57250K .......... .......... .......... .......... .......... 33% 8.34M 12s 2024-03-30T13:12:50.550 INFO:teuthology.orchestra.run.smithi043.stderr: 57300K .......... .......... .......... .......... .......... 34% 8.55M 12s 2024-03-30T13:12:50.555 INFO:teuthology.orchestra.run.smithi043.stderr: 57350K .......... .......... .......... .......... .......... 34% 8.96M 12s 2024-03-30T13:12:50.559 INFO:teuthology.orchestra.run.smithi043.stderr: 57400K .......... .......... .......... .......... .......... 34% 14.2M 12s 2024-03-30T13:12:50.564 INFO:teuthology.orchestra.run.smithi043.stderr: 57450K .......... .......... .......... .......... .......... 34% 9.25M 12s 2024-03-30T13:12:50.567 INFO:teuthology.orchestra.run.smithi043.stderr: 57500K .......... .......... .......... .......... .......... 34% 15.3M 12s 2024-03-30T13:12:50.570 INFO:teuthology.orchestra.run.smithi043.stderr: 57550K .......... .......... .......... .......... .......... 34% 14.7M 12s 2024-03-30T13:12:50.574 INFO:teuthology.orchestra.run.smithi043.stderr: 57600K .......... .......... .......... .......... .......... 34% 16.3M 12s 2024-03-30T13:12:50.577 INFO:teuthology.orchestra.run.smithi043.stderr: 57650K .......... .......... .......... .......... .......... 34% 14.8M 12s 2024-03-30T13:12:50.581 INFO:teuthology.orchestra.run.smithi043.stderr: 57700K .......... .......... .......... .......... .......... 34% 11.2M 12s 2024-03-30T13:12:50.586 INFO:teuthology.orchestra.run.smithi043.stderr: 57750K .......... .......... .......... .......... .......... 34% 9.56M 12s 2024-03-30T13:12:50.591 INFO:teuthology.orchestra.run.smithi043.stderr: 57800K .......... .......... .......... .......... .......... 34% 10.9M 12s 2024-03-30T13:12:50.595 INFO:teuthology.orchestra.run.smithi043.stderr: 57850K .......... .......... .......... .......... .......... 34% 11.1M 12s 2024-03-30T13:12:50.600 INFO:teuthology.orchestra.run.smithi043.stderr: 57900K .......... .......... .......... .......... .......... 34% 13.8M 12s 2024-03-30T13:12:50.604 INFO:teuthology.orchestra.run.smithi043.stderr: 57950K .......... .......... .......... .......... .......... 34% 9.73M 12s 2024-03-30T13:12:50.608 INFO:teuthology.orchestra.run.smithi043.stderr: 58000K .......... .......... .......... .......... .......... 34% 10.7M 12s 2024-03-30T13:12:50.613 INFO:teuthology.orchestra.run.smithi043.stderr: 58050K .......... .......... .......... .......... .......... 34% 10.5M 12s 2024-03-30T13:12:50.618 INFO:teuthology.orchestra.run.smithi043.stderr: 58100K .......... .......... .......... .......... .......... 34% 10.6M 12s 2024-03-30T13:12:50.624 INFO:teuthology.orchestra.run.smithi043.stderr: 58150K .......... .......... .......... .......... .......... 34% 8.92M 12s 2024-03-30T13:12:50.628 INFO:teuthology.orchestra.run.smithi043.stderr: 58200K .......... .......... .......... .......... .......... 34% 10.5M 12s 2024-03-30T13:12:50.633 INFO:teuthology.orchestra.run.smithi043.stderr: 58250K .......... .......... .......... .......... .......... 34% 9.62M 12s 2024-03-30T13:12:50.639 INFO:teuthology.orchestra.run.smithi043.stderr: 58300K .......... .......... .......... .......... .......... 34% 9.22M 12s 2024-03-30T13:12:50.645 INFO:teuthology.orchestra.run.smithi043.stderr: 58350K .......... .......... .......... .......... .......... 34% 8.02M 12s 2024-03-30T13:12:50.650 INFO:teuthology.orchestra.run.smithi043.stderr: 58400K .......... .......... .......... .......... .......... 34% 9.18M 12s 2024-03-30T13:12:50.655 INFO:teuthology.orchestra.run.smithi043.stderr: 58450K .......... .......... .......... .......... .......... 34% 8.79M 12s 2024-03-30T13:12:50.661 INFO:teuthology.orchestra.run.smithi043.stderr: 58500K .......... .......... .......... .......... .......... 34% 9.14M 12s 2024-03-30T13:12:50.667 INFO:teuthology.orchestra.run.smithi043.stderr: 58550K .......... .......... .......... .......... .......... 34% 7.52M 12s 2024-03-30T13:12:50.673 INFO:teuthology.orchestra.run.smithi043.stderr: 58600K .......... .......... .......... .......... .......... 34% 8.70M 12s 2024-03-30T13:12:50.677 INFO:teuthology.orchestra.run.smithi043.stderr: 58650K .......... .......... .......... .......... .......... 34% 10.5M 12s 2024-03-30T13:12:50.682 INFO:teuthology.orchestra.run.smithi043.stderr: 58700K .......... .......... .......... .......... .......... 34% 9.76M 12s 2024-03-30T13:12:50.688 INFO:teuthology.orchestra.run.smithi043.stderr: 58750K .......... .......... .......... .......... .......... 34% 8.73M 12s 2024-03-30T13:12:50.693 INFO:teuthology.orchestra.run.smithi043.stderr: 58800K .......... .......... .......... .......... .......... 34% 10.3M 12s 2024-03-30T13:12:50.698 INFO:teuthology.orchestra.run.smithi043.stderr: 58850K .......... .......... .......... .......... .......... 34% 10.2M 12s 2024-03-30T13:12:50.702 INFO:teuthology.orchestra.run.smithi043.stderr: 58900K .......... .......... .......... .......... .......... 34% 10.1M 12s 2024-03-30T13:12:50.708 INFO:teuthology.orchestra.run.smithi043.stderr: 58950K .......... .......... .......... .......... .......... 35% 9.14M 12s 2024-03-30T13:12:50.713 INFO:teuthology.orchestra.run.smithi043.stderr: 59000K .......... .......... .......... .......... .......... 35% 10.5M 12s 2024-03-30T13:12:50.717 INFO:teuthology.orchestra.run.smithi043.stderr: 59050K .......... .......... .......... .......... .......... 35% 10.1M 12s 2024-03-30T13:12:50.723 INFO:teuthology.orchestra.run.smithi043.stderr: 59100K .......... .......... .......... .......... .......... 35% 9.51M 12s 2024-03-30T13:12:50.729 INFO:teuthology.orchestra.run.smithi043.stderr: 59150K .......... .......... .......... .......... .......... 35% 8.41M 12s 2024-03-30T13:12:50.734 INFO:teuthology.orchestra.run.smithi043.stderr: 59200K .......... .......... .......... .......... .......... 35% 9.24M 12s 2024-03-30T13:12:50.740 INFO:teuthology.orchestra.run.smithi043.stderr: 59250K .......... .......... .......... .......... .......... 35% 8.71M 12s 2024-03-30T13:12:50.744 INFO:teuthology.orchestra.run.smithi043.stderr: 59300K .......... .......... .......... .......... .......... 35% 9.23M 12s 2024-03-30T13:12:50.751 INFO:teuthology.orchestra.run.smithi043.stderr: 59350K .......... .......... .......... .......... .......... 35% 7.86M 12s 2024-03-30T13:12:50.756 INFO:teuthology.orchestra.run.smithi043.stderr: 59400K .......... .......... .......... .......... .......... 35% 9.24M 12s 2024-03-30T13:12:50.761 INFO:teuthology.orchestra.run.smithi043.stderr: 59450K .......... .......... .......... .......... .......... 35% 9.16M 12s 2024-03-30T13:12:50.767 INFO:teuthology.orchestra.run.smithi043.stderr: 59500K .......... .......... .......... .......... .......... 35% 9.40M 12s 2024-03-30T13:12:50.773 INFO:teuthology.orchestra.run.smithi043.stderr: 59550K .......... .......... .......... .......... .......... 35% 7.76M 12s 2024-03-30T13:12:50.779 INFO:teuthology.orchestra.run.smithi043.stderr: 59600K .......... .......... .......... .......... .......... 35% 8.98M 12s 2024-03-30T13:12:50.784 INFO:teuthology.orchestra.run.smithi043.stderr: 59650K .......... .......... .......... .......... .......... 35% 8.83M 12s 2024-03-30T13:12:50.790 INFO:teuthology.orchestra.run.smithi043.stderr: 59700K .......... .......... .......... .......... .......... 35% 8.64M 12s 2024-03-30T13:12:50.796 INFO:teuthology.orchestra.run.smithi043.stderr: 59750K .......... .......... .......... .......... .......... 35% 7.45M 12s 2024-03-30T13:12:50.802 INFO:teuthology.orchestra.run.smithi043.stderr: 59800K .......... .......... .......... .......... .......... 35% 8.60M 12s 2024-03-30T13:12:50.808 INFO:teuthology.orchestra.run.smithi043.stderr: 59850K .......... .......... .......... .......... .......... 35% 8.05M 12s 2024-03-30T13:12:50.814 INFO:teuthology.orchestra.run.smithi043.stderr: 59900K .......... .......... .......... .......... .......... 35% 8.34M 12s 2024-03-30T13:12:50.820 INFO:teuthology.orchestra.run.smithi043.stderr: 59950K .......... .......... .......... .......... .......... 35% 7.26M 12s 2024-03-30T13:12:50.826 INFO:teuthology.orchestra.run.smithi043.stderr: 60000K .......... .......... .......... .......... .......... 35% 8.54M 12s 2024-03-30T13:12:50.832 INFO:teuthology.orchestra.run.smithi043.stderr: 60050K .......... .......... .......... .......... .......... 35% 8.24M 12s 2024-03-30T13:12:50.840 INFO:teuthology.orchestra.run.smithi043.stderr: 60100K .......... .......... .......... .......... .......... 35% 8.14M 12s 2024-03-30T13:12:50.845 INFO:teuthology.orchestra.run.smithi043.stderr: 60150K .......... .......... .......... .......... .......... 35% 6.79M 12s 2024-03-30T13:12:50.851 INFO:teuthology.orchestra.run.smithi043.stderr: 60200K .......... .......... .......... .......... .......... 35% 8.00M 12s 2024-03-30T13:12:50.858 INFO:teuthology.orchestra.run.smithi043.stderr: 60250K .......... .......... .......... .......... .......... 35% 8.02M 12s 2024-03-30T13:12:50.864 INFO:teuthology.orchestra.run.smithi043.stderr: 60300K .......... .......... .......... .......... .......... 35% 7.35M 12s 2024-03-30T13:12:50.873 INFO:teuthology.orchestra.run.smithi043.stderr: 60350K .......... .......... .......... .......... .......... 35% 6.77M 12s 2024-03-30T13:12:50.878 INFO:teuthology.orchestra.run.smithi043.stderr: 60400K .......... .......... .......... .......... .......... 35% 7.99M 12s 2024-03-30T13:12:50.884 INFO:teuthology.orchestra.run.smithi043.stderr: 60450K .......... .......... .......... .......... .......... 35% 7.45M 12s 2024-03-30T13:12:50.890 INFO:teuthology.orchestra.run.smithi043.stderr: 60500K .......... .......... .......... .......... .......... 35% 7.68M 12s 2024-03-30T13:12:50.898 INFO:teuthology.orchestra.run.smithi043.stderr: 60550K .......... .......... .......... .......... .......... 35% 6.79M 12s 2024-03-30T13:12:50.904 INFO:teuthology.orchestra.run.smithi043.stderr: 60600K .......... .......... .......... .......... .......... 35% 7.49M 12s 2024-03-30T13:12:50.911 INFO:teuthology.orchestra.run.smithi043.stderr: 60650K .......... .......... .......... .......... .......... 36% 7.13M 12s 2024-03-30T13:12:50.918 INFO:teuthology.orchestra.run.smithi043.stderr: 60700K .......... .......... .......... .......... .......... 36% 7.16M 12s 2024-03-30T13:12:50.926 INFO:teuthology.orchestra.run.smithi043.stderr: 60750K .......... .......... .......... .......... .......... 36% 5.96M 12s 2024-03-30T13:12:50.932 INFO:teuthology.orchestra.run.smithi043.stderr: 60800K .......... .......... .......... .......... .......... 36% 7.50M 12s 2024-03-30T13:12:50.939 INFO:teuthology.orchestra.run.smithi043.stderr: 60850K .......... .......... .......... .......... .......... 36% 7.44M 12s 2024-03-30T13:12:50.946 INFO:teuthology.orchestra.run.smithi043.stderr: 60900K .......... .......... .......... .......... .......... 36% 7.54M 12s 2024-03-30T13:12:50.953 INFO:teuthology.orchestra.run.smithi043.stderr: 60950K .......... .......... .......... .......... .......... 36% 6.32M 12s 2024-03-30T13:12:50.960 INFO:teuthology.orchestra.run.smithi043.stderr: 61000K .......... .......... .......... .......... .......... 36% 7.13M 12s 2024-03-30T13:12:50.967 INFO:teuthology.orchestra.run.smithi043.stderr: 61050K .......... .......... .......... .......... .......... 36% 7.38M 12s 2024-03-30T13:12:50.973 INFO:teuthology.orchestra.run.smithi043.stderr: 61100K .......... .......... .......... .......... .......... 36% 7.48M 12s 2024-03-30T13:12:50.980 INFO:teuthology.orchestra.run.smithi043.stderr: 61150K .......... .......... .......... .......... .......... 36% 7.03M 12s 2024-03-30T13:12:50.986 INFO:teuthology.orchestra.run.smithi043.stderr: 61200K .......... .......... .......... .......... .......... 36% 8.10M 12s 2024-03-30T13:12:50.993 INFO:teuthology.orchestra.run.smithi043.stderr: 61250K .......... .......... .......... .......... .......... 36% 7.63M 12s 2024-03-30T13:12:50.999 INFO:teuthology.orchestra.run.smithi043.stderr: 61300K .......... .......... .......... .......... .......... 36% 7.10M 12s 2024-03-30T13:12:51.008 INFO:teuthology.orchestra.run.smithi043.stderr: 61350K .......... .......... .......... .......... .......... 36% 6.13M 12s 2024-03-30T13:12:51.014 INFO:teuthology.orchestra.run.smithi043.stderr: 61400K .......... .......... .......... .......... .......... 36% 7.83M 12s 2024-03-30T13:12:51.019 INFO:teuthology.orchestra.run.smithi043.stderr: 61450K .......... .......... .......... .......... .......... 36% 9.19M 12s 2024-03-30T13:12:51.023 INFO:teuthology.orchestra.run.smithi043.stderr: 61500K .......... .......... .......... .......... .......... 36% 11.3M 12s 2024-03-30T13:12:51.029 INFO:teuthology.orchestra.run.smithi043.stderr: 61550K .......... .......... .......... .......... .......... 36% 8.67M 12s 2024-03-30T13:12:51.034 INFO:teuthology.orchestra.run.smithi043.stderr: 61600K .......... .......... .......... .......... .......... 36% 9.73M 12s 2024-03-30T13:12:51.037 INFO:teuthology.orchestra.run.smithi043.stderr: 61650K .......... .......... .......... .......... .......... 36% 12.9M 12s 2024-03-30T13:12:51.042 INFO:teuthology.orchestra.run.smithi043.stderr: 61700K .......... .......... .......... .......... .......... 36% 10.2M 12s 2024-03-30T13:12:51.047 INFO:teuthology.orchestra.run.smithi043.stderr: 61750K .......... .......... .......... .......... .......... 36% 9.80M 12s 2024-03-30T13:12:51.052 INFO:teuthology.orchestra.run.smithi043.stderr: 61800K .......... .......... .......... .......... .......... 36% 11.2M 12s 2024-03-30T13:12:51.056 INFO:teuthology.orchestra.run.smithi043.stderr: 61850K .......... .......... .......... .......... .......... 36% 11.2M 12s 2024-03-30T13:12:51.061 INFO:teuthology.orchestra.run.smithi043.stderr: 61900K .......... .......... .......... .......... .......... 36% 10.1M 12s 2024-03-30T13:12:51.067 INFO:teuthology.orchestra.run.smithi043.stderr: 61950K .......... .......... .......... .......... .......... 36% 8.35M 12s 2024-03-30T13:12:51.071 INFO:teuthology.orchestra.run.smithi043.stderr: 62000K .......... .......... .......... .......... .......... 36% 11.0M 12s 2024-03-30T13:12:51.076 INFO:teuthology.orchestra.run.smithi043.stderr: 62050K .......... .......... .......... .......... .......... 36% 10.5M 12s 2024-03-30T13:12:51.082 INFO:teuthology.orchestra.run.smithi043.stderr: 62100K .......... .......... .......... .......... .......... 36% 10.8M 12s 2024-03-30T13:12:51.086 INFO:teuthology.orchestra.run.smithi043.stderr: 62150K .......... .......... .......... .......... .......... 36% 8.58M 12s 2024-03-30T13:12:51.091 INFO:teuthology.orchestra.run.smithi043.stderr: 62200K .......... .......... .......... .......... .......... 36% 10.5M 12s 2024-03-30T13:12:51.096 INFO:teuthology.orchestra.run.smithi043.stderr: 62250K .......... .......... .......... .......... .......... 36% 10.5M 12s 2024-03-30T13:12:51.100 INFO:teuthology.orchestra.run.smithi043.stderr: 62300K .......... .......... .......... .......... .......... 36% 10.8M 12s 2024-03-30T13:12:51.105 INFO:teuthology.orchestra.run.smithi043.stderr: 62350K .......... .......... .......... .......... .......... 37% 8.98M 12s 2024-03-30T13:12:51.110 INFO:teuthology.orchestra.run.smithi043.stderr: 62400K .......... .......... .......... .......... .......... 37% 11.0M 12s 2024-03-30T13:12:51.114 INFO:teuthology.orchestra.run.smithi043.stderr: 62450K .......... .......... .......... .......... .......... 37% 10.2M 12s 2024-03-30T13:12:51.119 INFO:teuthology.orchestra.run.smithi043.stderr: 62500K .......... .......... .......... .......... .......... 37% 10.3M 12s 2024-03-30T13:12:51.125 INFO:teuthology.orchestra.run.smithi043.stderr: 62550K .......... .......... .......... .......... .......... 37% 9.36M 12s 2024-03-30T13:12:51.129 INFO:teuthology.orchestra.run.smithi043.stderr: 62600K .......... .......... .......... .......... .......... 37% 10.9M 12s 2024-03-30T13:12:51.134 INFO:teuthology.orchestra.run.smithi043.stderr: 62650K .......... .......... .......... .......... .......... 37% 10.5M 12s 2024-03-30T13:12:51.138 INFO:teuthology.orchestra.run.smithi043.stderr: 62700K .......... .......... .......... .......... .......... 37% 10.5M 12s 2024-03-30T13:12:51.145 INFO:teuthology.orchestra.run.smithi043.stderr: 62750K .......... .......... .......... .......... .......... 37% 7.72M 12s 2024-03-30T13:12:51.150 INFO:teuthology.orchestra.run.smithi043.stderr: 62800K .......... .......... .......... .......... .......... 37% 9.51M 12s 2024-03-30T13:12:51.156 INFO:teuthology.orchestra.run.smithi043.stderr: 62850K .......... .......... .......... .......... .......... 37% 8.13M 12s 2024-03-30T13:12:51.161 INFO:teuthology.orchestra.run.smithi043.stderr: 62900K .......... .......... .......... .......... .......... 37% 8.63M 12s 2024-03-30T13:12:51.168 INFO:teuthology.orchestra.run.smithi043.stderr: 62950K .......... .......... .......... .......... .......... 37% 7.62M 12s 2024-03-30T13:12:51.173 INFO:teuthology.orchestra.run.smithi043.stderr: 63000K .......... .......... .......... .......... .......... 37% 9.70M 12s 2024-03-30T13:12:51.179 INFO:teuthology.orchestra.run.smithi043.stderr: 63050K .......... .......... .......... .......... .......... 37% 8.25M 12s 2024-03-30T13:12:51.185 INFO:teuthology.orchestra.run.smithi043.stderr: 63100K .......... .......... .......... .......... .......... 37% 8.03M 12s 2024-03-30T13:12:51.193 INFO:teuthology.orchestra.run.smithi043.stderr: 63150K .......... .......... .......... .......... .......... 37% 6.39M 12s 2024-03-30T13:12:51.200 INFO:teuthology.orchestra.run.smithi043.stderr: 63200K .......... .......... .......... .......... .......... 37% 7.20M 12s 2024-03-30T13:12:51.206 INFO:teuthology.orchestra.run.smithi043.stderr: 63250K .......... .......... .......... .......... .......... 37% 7.26M 12s 2024-03-30T13:12:51.211 INFO:teuthology.orchestra.run.smithi043.stderr: 63300K .......... .......... .......... .......... .......... 37% 9.50M 12s 2024-03-30T13:12:51.217 INFO:teuthology.orchestra.run.smithi043.stderr: 63350K .......... .......... .......... .......... .......... 37% 8.34M 12s 2024-03-30T13:12:51.222 INFO:teuthology.orchestra.run.smithi043.stderr: 63400K .......... .......... .......... .......... .......... 37% 9.59M 12s 2024-03-30T13:12:51.227 INFO:teuthology.orchestra.run.smithi043.stderr: 63450K .......... .......... .......... .......... .......... 37% 9.60M 12s 2024-03-30T13:12:51.233 INFO:teuthology.orchestra.run.smithi043.stderr: 63500K .......... .......... .......... .......... .......... 37% 7.80M 12s 2024-03-30T13:12:51.241 INFO:teuthology.orchestra.run.smithi043.stderr: 63550K .......... .......... .......... .......... .......... 37% 6.84M 12s 2024-03-30T13:12:51.247 INFO:teuthology.orchestra.run.smithi043.stderr: 63600K .......... .......... .......... .......... .......... 37% 8.01M 12s 2024-03-30T13:12:51.253 INFO:teuthology.orchestra.run.smithi043.stderr: 63650K .......... .......... .......... .......... .......... 37% 8.15M 12s 2024-03-30T13:12:51.260 INFO:teuthology.orchestra.run.smithi043.stderr: 63700K .......... .......... .......... .......... .......... 37% 7.74M 12s 2024-03-30T13:12:51.266 INFO:teuthology.orchestra.run.smithi043.stderr: 63750K .......... .......... .......... .......... .......... 37% 6.61M 12s 2024-03-30T13:12:51.273 INFO:teuthology.orchestra.run.smithi043.stderr: 63800K .......... .......... .......... .......... .......... 37% 7.90M 12s 2024-03-30T13:12:51.279 INFO:teuthology.orchestra.run.smithi043.stderr: 63850K .......... .......... .......... .......... .......... 37% 7.74M 12s 2024-03-30T13:12:51.285 INFO:teuthology.orchestra.run.smithi043.stderr: 63900K .......... .......... .......... .......... .......... 37% 8.35M 12s 2024-03-30T13:12:51.292 INFO:teuthology.orchestra.run.smithi043.stderr: 63950K .......... .......... .......... .......... .......... 37% 6.91M 12s 2024-03-30T13:12:51.297 INFO:teuthology.orchestra.run.smithi043.stderr: 64000K .......... .......... .......... .......... .......... 37% 8.69M 12s 2024-03-30T13:12:51.303 INFO:teuthology.orchestra.run.smithi043.stderr: 64050K .......... .......... .......... .......... .......... 38% 8.74M 12s 2024-03-30T13:12:51.308 INFO:teuthology.orchestra.run.smithi043.stderr: 64100K .......... .......... .......... .......... .......... 38% 9.14M 12s 2024-03-30T13:12:51.315 INFO:teuthology.orchestra.run.smithi043.stderr: 64150K .......... .......... .......... .......... .......... 38% 7.75M 12s 2024-03-30T13:12:51.320 INFO:teuthology.orchestra.run.smithi043.stderr: 64200K .......... .......... .......... .......... .......... 38% 9.23M 12s 2024-03-30T13:12:51.325 INFO:teuthology.orchestra.run.smithi043.stderr: 64250K .......... .......... .......... .......... .......... 38% 9.14M 12s 2024-03-30T13:12:51.331 INFO:teuthology.orchestra.run.smithi043.stderr: 64300K .......... .......... .......... .......... .......... 38% 8.89M 12s 2024-03-30T13:12:51.337 INFO:teuthology.orchestra.run.smithi043.stderr: 64350K .......... .......... .......... .......... .......... 38% 7.73M 12s 2024-03-30T13:12:51.343 INFO:teuthology.orchestra.run.smithi043.stderr: 64400K .......... .......... .......... .......... .......... 38% 8.61M 12s 2024-03-30T13:12:51.348 INFO:teuthology.orchestra.run.smithi043.stderr: 64450K .......... .......... .......... .......... .......... 38% 8.89M 12s 2024-03-30T13:12:51.354 INFO:teuthology.orchestra.run.smithi043.stderr: 64500K .......... .......... .......... .......... .......... 38% 8.60M 12s 2024-03-30T13:12:51.361 INFO:teuthology.orchestra.run.smithi043.stderr: 64550K .......... .......... .......... .......... .......... 38% 7.23M 12s 2024-03-30T13:12:51.366 INFO:teuthology.orchestra.run.smithi043.stderr: 64600K .......... .......... .......... .......... .......... 38% 8.52M 12s 2024-03-30T13:12:51.372 INFO:teuthology.orchestra.run.smithi043.stderr: 64650K .......... .......... .......... .......... .......... 38% 8.63M 12s 2024-03-30T13:12:51.378 INFO:teuthology.orchestra.run.smithi043.stderr: 64700K .......... .......... .......... .......... .......... 38% 7.94M 11s 2024-03-30T13:12:51.385 INFO:teuthology.orchestra.run.smithi043.stderr: 64750K .......... .......... .......... .......... .......... 38% 6.88M 11s 2024-03-30T13:12:51.391 INFO:teuthology.orchestra.run.smithi043.stderr: 64800K .......... .......... .......... .......... .......... 38% 8.41M 11s 2024-03-30T13:12:51.398 INFO:teuthology.orchestra.run.smithi043.stderr: 64850K .......... .......... .......... .......... .......... 38% 7.61M 11s 2024-03-30T13:12:51.404 INFO:teuthology.orchestra.run.smithi043.stderr: 64900K .......... .......... .......... .......... .......... 38% 7.68M 11s 2024-03-30T13:12:51.412 INFO:teuthology.orchestra.run.smithi043.stderr: 64950K .......... .......... .......... .......... .......... 38% 6.43M 11s 2024-03-30T13:12:51.419 INFO:teuthology.orchestra.run.smithi043.stderr: 65000K .......... .......... .......... .......... .......... 38% 7.15M 11s 2024-03-30T13:12:51.426 INFO:teuthology.orchestra.run.smithi043.stderr: 65050K .......... .......... .......... .......... .......... 38% 6.84M 11s 2024-03-30T13:12:51.433 INFO:teuthology.orchestra.run.smithi043.stderr: 65100K .......... .......... .......... .......... .......... 38% 6.96M 11s 2024-03-30T13:12:51.441 INFO:teuthology.orchestra.run.smithi043.stderr: 65150K .......... .......... .......... .......... .......... 38% 6.41M 11s 2024-03-30T13:12:51.447 INFO:teuthology.orchestra.run.smithi043.stderr: 65200K .......... .......... .......... .......... .......... 38% 7.38M 11s 2024-03-30T13:12:51.453 INFO:teuthology.orchestra.run.smithi043.stderr: 65250K .......... .......... .......... .......... .......... 38% 7.60M 11s 2024-03-30T13:12:51.460 INFO:teuthology.orchestra.run.smithi043.stderr: 65300K .......... .......... .......... .......... .......... 38% 7.22M 11s 2024-03-30T13:12:51.468 INFO:teuthology.orchestra.run.smithi043.stderr: 65350K .......... .......... .......... .......... .......... 38% 5.92M 11s 2024-03-30T13:12:51.474 INFO:teuthology.orchestra.run.smithi043.stderr: 65400K .......... .......... .......... .......... .......... 38% 7.98M 11s 2024-03-30T13:12:51.480 INFO:teuthology.orchestra.run.smithi043.stderr: 65450K .......... .......... .......... .......... .......... 38% 8.61M 11s 2024-03-30T13:12:51.490 INFO:teuthology.orchestra.run.smithi043.stderr: 65500K .......... .......... .......... .......... .......... 38% 8.13M 11s 2024-03-30T13:12:51.492 INFO:teuthology.orchestra.run.smithi043.stderr: 65550K .......... .......... .......... .......... .......... 38% 7.31M 11s 2024-03-30T13:12:51.496 INFO:teuthology.orchestra.run.smithi043.stderr: 65600K .......... .......... .......... .......... .......... 38% 12.4M 11s 2024-03-30T13:12:51.502 INFO:teuthology.orchestra.run.smithi043.stderr: 65650K .......... .......... .......... .......... .......... 38% 8.31M 11s 2024-03-30T13:12:51.508 INFO:teuthology.orchestra.run.smithi043.stderr: 65700K .......... .......... .......... .......... .......... 39% 9.38M 11s 2024-03-30T13:12:51.514 INFO:teuthology.orchestra.run.smithi043.stderr: 65750K .......... .......... .......... .......... .......... 39% 7.81M 11s 2024-03-30T13:12:51.519 INFO:teuthology.orchestra.run.smithi043.stderr: 65800K .......... .......... .......... .......... .......... 39% 9.28M 11s 2024-03-30T13:12:51.523 INFO:teuthology.orchestra.run.smithi043.stderr: 65850K .......... .......... .......... .......... .......... 39% 11.2M 11s 2024-03-30T13:12:51.528 INFO:teuthology.orchestra.run.smithi043.stderr: 65900K .......... .......... .......... .......... .......... 39% 11.3M 11s 2024-03-30T13:12:51.533 INFO:teuthology.orchestra.run.smithi043.stderr: 65950K .......... .......... .......... .......... .......... 39% 8.98M 11s 2024-03-30T13:12:51.538 INFO:teuthology.orchestra.run.smithi043.stderr: 66000K .......... .......... .......... .......... .......... 39% 10.7M 11s 2024-03-30T13:12:51.542 INFO:teuthology.orchestra.run.smithi043.stderr: 66050K .......... .......... .......... .......... .......... 39% 10.6M 11s 2024-03-30T13:12:51.547 INFO:teuthology.orchestra.run.smithi043.stderr: 66100K .......... .......... .......... .......... .......... 39% 10.2M 11s 2024-03-30T13:12:51.553 INFO:teuthology.orchestra.run.smithi043.stderr: 66150K .......... .......... .......... .......... .......... 39% 9.17M 11s 2024-03-30T13:12:51.557 INFO:teuthology.orchestra.run.smithi043.stderr: 66200K .......... .......... .......... .......... .......... 39% 9.75M 11s 2024-03-30T13:12:51.563 INFO:teuthology.orchestra.run.smithi043.stderr: 66250K .......... .......... .......... .......... .......... 39% 9.85M 11s 2024-03-30T13:12:51.567 INFO:teuthology.orchestra.run.smithi043.stderr: 66300K .......... .......... .......... .......... .......... 39% 10.1M 11s 2024-03-30T13:12:51.575 INFO:teuthology.orchestra.run.smithi043.stderr: 66350K .......... .......... .......... .......... .......... 39% 7.78M 11s 2024-03-30T13:12:51.579 INFO:teuthology.orchestra.run.smithi043.stderr: 66400K .......... .......... .......... .......... .......... 39% 9.08M 11s 2024-03-30T13:12:51.585 INFO:teuthology.orchestra.run.smithi043.stderr: 66450K .......... .......... .......... .......... .......... 39% 9.27M 11s 2024-03-30T13:12:51.589 INFO:teuthology.orchestra.run.smithi043.stderr: 66500K .......... .......... .......... .......... .......... 39% 9.91M 11s 2024-03-30T13:12:51.595 INFO:teuthology.orchestra.run.smithi043.stderr: 66550K .......... .......... .......... .......... .......... 39% 7.83M 11s 2024-03-30T13:12:51.600 INFO:teuthology.orchestra.run.smithi043.stderr: 66600K .......... .......... .......... .......... .......... 39% 9.90M 11s 2024-03-30T13:12:51.604 INFO:teuthology.orchestra.run.smithi043.stderr: 66650K .......... .......... .......... .......... .......... 39% 11.5M 11s 2024-03-30T13:12:51.609 INFO:teuthology.orchestra.run.smithi043.stderr: 66700K .......... .......... .......... .......... .......... 39% 11.4M 11s 2024-03-30T13:12:51.614 INFO:teuthology.orchestra.run.smithi043.stderr: 66750K .......... .......... .......... .......... .......... 39% 9.32M 11s 2024-03-30T13:12:51.619 INFO:teuthology.orchestra.run.smithi043.stderr: 66800K .......... .......... .......... .......... .......... 39% 10.9M 11s 2024-03-30T13:12:51.623 INFO:teuthology.orchestra.run.smithi043.stderr: 66850K .......... .......... .......... .......... .......... 39% 11.4M 11s 2024-03-30T13:12:51.627 INFO:teuthology.orchestra.run.smithi043.stderr: 66900K .......... .......... .......... .......... .......... 39% 10.7M 11s 2024-03-30T13:12:51.633 INFO:teuthology.orchestra.run.smithi043.stderr: 66950K .......... .......... .......... .......... .......... 39% 8.81M 11s 2024-03-30T13:12:51.638 INFO:teuthology.orchestra.run.smithi043.stderr: 67000K .......... .......... .......... .......... .......... 39% 10.0M 11s 2024-03-30T13:12:51.642 INFO:teuthology.orchestra.run.smithi043.stderr: 67050K .......... .......... .......... .......... .......... 39% 11.0M 11s 2024-03-30T13:12:51.647 INFO:teuthology.orchestra.run.smithi043.stderr: 67100K .......... .......... .......... .......... .......... 39% 10.4M 11s 2024-03-30T13:12:51.652 INFO:teuthology.orchestra.run.smithi043.stderr: 67150K .......... .......... .......... .......... .......... 39% 9.53M 11s 2024-03-30T13:12:51.657 INFO:teuthology.orchestra.run.smithi043.stderr: 67200K .......... .......... .......... .......... .......... 39% 10.6M 11s 2024-03-30T13:12:51.661 INFO:teuthology.orchestra.run.smithi043.stderr: 67250K .......... .......... .......... .......... .......... 39% 10.3M 11s 2024-03-30T13:12:51.666 INFO:teuthology.orchestra.run.smithi043.stderr: 67300K .......... .......... .......... .......... .......... 39% 10.8M 11s 2024-03-30T13:12:51.672 INFO:teuthology.orchestra.run.smithi043.stderr: 67350K .......... .......... .......... .......... .......... 39% 8.61M 11s 2024-03-30T13:12:51.676 INFO:teuthology.orchestra.run.smithi043.stderr: 67400K .......... .......... .......... .......... .......... 40% 10.2M 11s 2024-03-30T13:12:51.681 INFO:teuthology.orchestra.run.smithi043.stderr: 67450K .......... .......... .......... .......... .......... 40% 9.92M 11s 2024-03-30T13:12:51.686 INFO:teuthology.orchestra.run.smithi043.stderr: 67500K .......... .......... .......... .......... .......... 40% 10.2M 11s 2024-03-30T13:12:51.695 INFO:teuthology.orchestra.run.smithi043.stderr: 67550K .......... .......... .......... .......... .......... 40% 8.15M 11s 2024-03-30T13:12:51.698 INFO:teuthology.orchestra.run.smithi043.stderr: 67600K .......... .......... .......... .......... .......... 40% 9.35M 11s 2024-03-30T13:12:51.703 INFO:teuthology.orchestra.run.smithi043.stderr: 67650K .......... .......... .......... .......... .......... 40% 9.58M 11s 2024-03-30T13:12:51.708 INFO:teuthology.orchestra.run.smithi043.stderr: 67700K .......... .......... .......... .......... .......... 40% 9.25M 11s 2024-03-30T13:12:51.714 INFO:teuthology.orchestra.run.smithi043.stderr: 67750K .......... .......... .......... .......... .......... 40% 8.46M 11s 2024-03-30T13:12:51.719 INFO:teuthology.orchestra.run.smithi043.stderr: 67800K .......... .......... .......... .......... .......... 40% 9.80M 11s 2024-03-30T13:12:51.723 INFO:teuthology.orchestra.run.smithi043.stderr: 67850K .......... .......... .......... .......... .......... 40% 9.98M 11s 2024-03-30T13:12:51.728 INFO:teuthology.orchestra.run.smithi043.stderr: 67900K .......... .......... .......... .......... .......... 40% 10.1M 11s 2024-03-30T13:12:51.734 INFO:teuthology.orchestra.run.smithi043.stderr: 67950K .......... .......... .......... .......... .......... 40% 8.37M 11s 2024-03-30T13:12:51.739 INFO:teuthology.orchestra.run.smithi043.stderr: 68000K .......... .......... .......... .......... .......... 40% 8.95M 11s 2024-03-30T13:12:51.745 INFO:teuthology.orchestra.run.smithi043.stderr: 68050K .......... .......... .......... .......... .......... 40% 8.95M 11s 2024-03-30T13:12:51.750 INFO:teuthology.orchestra.run.smithi043.stderr: 68100K .......... .......... .......... .......... .......... 40% 9.00M 11s 2024-03-30T13:12:51.757 INFO:teuthology.orchestra.run.smithi043.stderr: 68150K .......... .......... .......... .......... .......... 40% 7.58M 11s 2024-03-30T13:12:51.762 INFO:teuthology.orchestra.run.smithi043.stderr: 68200K .......... .......... .......... .......... .......... 40% 9.24M 11s 2024-03-30T13:12:51.767 INFO:teuthology.orchestra.run.smithi043.stderr: 68250K .......... .......... .......... .......... .......... 40% 9.76M 11s 2024-03-30T13:12:51.772 INFO:teuthology.orchestra.run.smithi043.stderr: 68300K .......... .......... .......... .......... .......... 40% 9.58M 11s 2024-03-30T13:12:51.778 INFO:teuthology.orchestra.run.smithi043.stderr: 68350K .......... .......... .......... .......... .......... 40% 8.23M 11s 2024-03-30T13:12:51.783 INFO:teuthology.orchestra.run.smithi043.stderr: 68400K .......... .......... .......... .......... .......... 40% 9.21M 11s 2024-03-30T13:12:51.789 INFO:teuthology.orchestra.run.smithi043.stderr: 68450K .......... .......... .......... .......... .......... 40% 8.59M 11s 2024-03-30T13:12:51.795 INFO:teuthology.orchestra.run.smithi043.stderr: 68500K .......... .......... .......... .......... .......... 40% 8.49M 11s 2024-03-30T13:12:51.802 INFO:teuthology.orchestra.run.smithi043.stderr: 68550K .......... .......... .......... .......... .......... 40% 7.36M 11s 2024-03-30T13:12:51.807 INFO:teuthology.orchestra.run.smithi043.stderr: 68600K .......... .......... .......... .......... .......... 40% 8.82M 11s 2024-03-30T13:12:51.813 INFO:teuthology.orchestra.run.smithi043.stderr: 68650K .......... .......... .......... .......... .......... 40% 8.75M 11s 2024-03-30T13:12:51.818 INFO:teuthology.orchestra.run.smithi043.stderr: 68700K .......... .......... .......... .......... .......... 40% 8.92M 11s 2024-03-30T13:12:51.825 INFO:teuthology.orchestra.run.smithi043.stderr: 68750K .......... .......... .......... .......... .......... 40% 7.43M 11s 2024-03-30T13:12:51.831 INFO:teuthology.orchestra.run.smithi043.stderr: 68800K .......... .......... .......... .......... .......... 40% 8.52M 11s 2024-03-30T13:12:51.836 INFO:teuthology.orchestra.run.smithi043.stderr: 68850K .......... .......... .......... .......... .......... 40% 8.65M 11s 2024-03-30T13:12:51.842 INFO:teuthology.orchestra.run.smithi043.stderr: 68900K .......... .......... .......... .......... .......... 40% 8.37M 11s 2024-03-30T13:12:51.849 INFO:teuthology.orchestra.run.smithi043.stderr: 68950K .......... .......... .......... .......... .......... 40% 7.54M 11s 2024-03-30T13:12:51.854 INFO:teuthology.orchestra.run.smithi043.stderr: 69000K .......... .......... .......... .......... .......... 40% 8.50M 11s 2024-03-30T13:12:51.860 INFO:teuthology.orchestra.run.smithi043.stderr: 69050K .......... .......... .......... .......... .......... 40% 8.49M 11s 2024-03-30T13:12:51.866 INFO:teuthology.orchestra.run.smithi043.stderr: 69100K .......... .......... .......... .......... .......... 41% 8.49M 11s 2024-03-30T13:12:51.872 INFO:teuthology.orchestra.run.smithi043.stderr: 69150K .......... .......... .......... .......... .......... 41% 7.16M 11s 2024-03-30T13:12:51.879 INFO:teuthology.orchestra.run.smithi043.stderr: 69200K .......... .......... .......... .......... .......... 41% 8.18M 11s 2024-03-30T13:12:51.885 INFO:teuthology.orchestra.run.smithi043.stderr: 69250K .......... .......... .......... .......... .......... 41% 7.79M 11s 2024-03-30T13:12:51.891 INFO:teuthology.orchestra.run.smithi043.stderr: 69300K .......... .......... .......... .......... .......... 41% 8.34M 11s 2024-03-30T13:12:51.900 INFO:teuthology.orchestra.run.smithi043.stderr: 69350K .......... .......... .......... .......... .......... 41% 7.21M 11s 2024-03-30T13:12:51.903 INFO:teuthology.orchestra.run.smithi043.stderr: 69400K .......... .......... .......... .......... .......... 41% 8.40M 11s 2024-03-30T13:12:51.909 INFO:teuthology.orchestra.run.smithi043.stderr: 69450K .......... .......... .......... .......... .......... 41% 8.21M 11s 2024-03-30T13:12:51.915 INFO:teuthology.orchestra.run.smithi043.stderr: 69500K .......... .......... .......... .......... .......... 41% 8.40M 11s 2024-03-30T13:12:51.922 INFO:teuthology.orchestra.run.smithi043.stderr: 69550K .......... .......... .......... .......... .......... 41% 7.07M 11s 2024-03-30T13:12:51.928 INFO:teuthology.orchestra.run.smithi043.stderr: 69600K .......... .......... .......... .......... .......... 41% 7.63M 11s 2024-03-30T13:12:51.934 INFO:teuthology.orchestra.run.smithi043.stderr: 69650K .......... .......... .......... .......... .......... 41% 8.87M 11s 2024-03-30T13:12:51.939 INFO:teuthology.orchestra.run.smithi043.stderr: 69700K .......... .......... .......... .......... .......... 41% 9.68M 11s 2024-03-30T13:12:51.943 INFO:teuthology.orchestra.run.smithi043.stderr: 69750K .......... .......... .......... .......... .......... 41% 11.2M 11s 2024-03-30T13:12:51.947 INFO:teuthology.orchestra.run.smithi043.stderr: 69800K .......... .......... .......... .......... .......... 41% 12.8M 11s 2024-03-30T13:12:51.951 INFO:teuthology.orchestra.run.smithi043.stderr: 69850K .......... .......... .......... .......... .......... 41% 12.7M 11s 2024-03-30T13:12:51.955 INFO:teuthology.orchestra.run.smithi043.stderr: 69900K .......... .......... .......... .......... .......... 41% 10.5M 11s 2024-03-30T13:12:51.961 INFO:teuthology.orchestra.run.smithi043.stderr: 69950K .......... .......... .......... .......... .......... 41% 8.65M 11s 2024-03-30T13:12:51.964 INFO:teuthology.orchestra.run.smithi043.stderr: 70000K .......... .......... .......... .......... .......... 41% 13.4M 11s 2024-03-30T13:12:51.969 INFO:teuthology.orchestra.run.smithi043.stderr: 70050K .......... .......... .......... .......... .......... 41% 15.0M 11s 2024-03-30T13:12:51.971 INFO:teuthology.orchestra.run.smithi043.stderr: 70100K .......... .......... .......... .......... .......... 41% 14.4M 11s 2024-03-30T13:12:51.976 INFO:teuthology.orchestra.run.smithi043.stderr: 70150K .......... .......... .......... .......... .......... 41% 10.8M 11s 2024-03-30T13:12:51.979 INFO:teuthology.orchestra.run.smithi043.stderr: 70200K .......... .......... .......... .......... .......... 41% 13.7M 11s 2024-03-30T13:12:51.983 INFO:teuthology.orchestra.run.smithi043.stderr: 70250K .......... .......... .......... .......... .......... 41% 12.6M 11s 2024-03-30T13:12:51.987 INFO:teuthology.orchestra.run.smithi043.stderr: 70300K .......... .......... .......... .......... .......... 41% 12.9M 11s 2024-03-30T13:12:51.992 INFO:teuthology.orchestra.run.smithi043.stderr: 70350K .......... .......... .......... .......... .......... 41% 9.97M 11s 2024-03-30T13:12:51.996 INFO:teuthology.orchestra.run.smithi043.stderr: 70400K .......... .......... .......... .......... .......... 41% 11.9M 11s 2024-03-30T13:12:52.000 INFO:teuthology.orchestra.run.smithi043.stderr: 70450K .......... .......... .......... .......... .......... 41% 12.6M 11s 2024-03-30T13:12:52.004 INFO:teuthology.orchestra.run.smithi043.stderr: 70500K .......... .......... .......... .......... .......... 41% 11.6M 11s 2024-03-30T13:12:52.009 INFO:teuthology.orchestra.run.smithi043.stderr: 70550K .......... .......... .......... .......... .......... 41% 10.5M 11s 2024-03-30T13:12:52.013 INFO:teuthology.orchestra.run.smithi043.stderr: 70600K .......... .......... .......... .......... .......... 41% 12.8M 11s 2024-03-30T13:12:52.016 INFO:teuthology.orchestra.run.smithi043.stderr: 70650K .......... .......... .......... .......... .......... 41% 13.4M 11s 2024-03-30T13:12:52.020 INFO:teuthology.orchestra.run.smithi043.stderr: 70700K .......... .......... .......... .......... .......... 41% 11.9M 11s 2024-03-30T13:12:52.027 INFO:teuthology.orchestra.run.smithi043.stderr: 70750K .......... .......... .......... .......... .......... 42% 7.89M 11s 2024-03-30T13:12:52.033 INFO:teuthology.orchestra.run.smithi043.stderr: 70800K .......... .......... .......... .......... .......... 42% 8.21M 11s 2024-03-30T13:12:52.039 INFO:teuthology.orchestra.run.smithi043.stderr: 70850K .......... .......... .......... .......... .......... 42% 7.79M 11s 2024-03-30T13:12:52.044 INFO:teuthology.orchestra.run.smithi043.stderr: 70900K .......... .......... .......... .......... .......... 42% 9.53M 11s 2024-03-30T13:12:52.049 INFO:teuthology.orchestra.run.smithi043.stderr: 70950K .......... .......... .......... .......... .......... 42% 8.22M 11s 2024-03-30T13:12:52.054 INFO:teuthology.orchestra.run.smithi043.stderr: 71000K .......... .......... .......... .......... .......... 42% 12.2M 11s 2024-03-30T13:12:52.059 INFO:teuthology.orchestra.run.smithi043.stderr: 71050K .......... .......... .......... .......... .......... 42% 10.3M 11s 2024-03-30T13:12:52.064 INFO:teuthology.orchestra.run.smithi043.stderr: 71100K .......... .......... .......... .......... .......... 42% 9.73M 11s 2024-03-30T13:12:52.069 INFO:teuthology.orchestra.run.smithi043.stderr: 71150K .......... .......... .......... .......... .......... 42% 8.38M 11s 2024-03-30T13:12:52.075 INFO:teuthology.orchestra.run.smithi043.stderr: 71200K .......... .......... .......... .......... .......... 42% 8.63M 11s 2024-03-30T13:12:52.080 INFO:teuthology.orchestra.run.smithi043.stderr: 71250K .......... .......... .......... .......... .......... 42% 10.6M 11s 2024-03-30T13:12:52.084 INFO:teuthology.orchestra.run.smithi043.stderr: 71300K .......... .......... .......... .......... .......... 42% 10.9M 11s 2024-03-30T13:12:52.090 INFO:teuthology.orchestra.run.smithi043.stderr: 71350K .......... .......... .......... .......... .......... 42% 8.53M 11s 2024-03-30T13:12:52.095 INFO:teuthology.orchestra.run.smithi043.stderr: 71400K .......... .......... .......... .......... .......... 42% 9.19M 11s 2024-03-30T13:12:52.101 INFO:teuthology.orchestra.run.smithi043.stderr: 71450K .......... .......... .......... .......... .......... 42% 9.44M 11s 2024-03-30T13:12:52.106 INFO:teuthology.orchestra.run.smithi043.stderr: 71500K .......... .......... .......... .......... .......... 42% 8.93M 11s 2024-03-30T13:12:52.112 INFO:teuthology.orchestra.run.smithi043.stderr: 71550K .......... .......... .......... .......... .......... 42% 7.42M 11s 2024-03-30T13:12:52.118 INFO:teuthology.orchestra.run.smithi043.stderr: 71600K .......... .......... .......... .......... .......... 42% 8.63M 11s 2024-03-30T13:12:52.123 INFO:teuthology.orchestra.run.smithi043.stderr: 71650K .......... .......... .......... .......... .......... 42% 10.3M 11s 2024-03-30T13:12:52.127 INFO:teuthology.orchestra.run.smithi043.stderr: 71700K .......... .......... .......... .......... .......... 42% 10.8M 11s 2024-03-30T13:12:52.133 INFO:teuthology.orchestra.run.smithi043.stderr: 71750K .......... .......... .......... .......... .......... 42% 8.93M 11s 2024-03-30T13:12:52.139 INFO:teuthology.orchestra.run.smithi043.stderr: 71800K .......... .......... .......... .......... .......... 42% 8.03M 11s 2024-03-30T13:12:52.146 INFO:teuthology.orchestra.run.smithi043.stderr: 71850K .......... .......... .......... .......... .......... 42% 7.92M 11s 2024-03-30T13:12:52.151 INFO:teuthology.orchestra.run.smithi043.stderr: 71900K .......... .......... .......... .......... .......... 42% 8.28M 11s 2024-03-30T13:12:52.158 INFO:teuthology.orchestra.run.smithi043.stderr: 71950K .......... .......... .......... .......... .......... 42% 7.21M 11s 2024-03-30T13:12:52.164 INFO:teuthology.orchestra.run.smithi043.stderr: 72000K .......... .......... .......... .......... .......... 42% 8.45M 11s 2024-03-30T13:12:52.170 INFO:teuthology.orchestra.run.smithi043.stderr: 72050K .......... .......... .......... .......... .......... 42% 8.19M 11s 2024-03-30T13:12:52.176 INFO:teuthology.orchestra.run.smithi043.stderr: 72100K .......... .......... .......... .......... .......... 42% 7.86M 11s 2024-03-30T13:12:52.183 INFO:teuthology.orchestra.run.smithi043.stderr: 72150K .......... .......... .......... .......... .......... 42% 6.92M 11s 2024-03-30T13:12:52.189 INFO:teuthology.orchestra.run.smithi043.stderr: 72200K .......... .......... .......... .......... .......... 42% 7.78M 11s 2024-03-30T13:12:52.196 INFO:teuthology.orchestra.run.smithi043.stderr: 72250K .......... .......... .......... .......... .......... 42% 7.84M 11s 2024-03-30T13:12:52.202 INFO:teuthology.orchestra.run.smithi043.stderr: 72300K .......... .......... .......... .......... .......... 42% 7.71M 11s 2024-03-30T13:12:52.209 INFO:teuthology.orchestra.run.smithi043.stderr: 72350K .......... .......... .......... .......... .......... 42% 6.58M 11s 2024-03-30T13:12:52.216 INFO:teuthology.orchestra.run.smithi043.stderr: 72400K .......... .......... .......... .......... .......... 42% 7.71M 11s 2024-03-30T13:12:52.222 INFO:teuthology.orchestra.run.smithi043.stderr: 72450K .......... .......... .......... .......... .......... 43% 7.74M 11s 2024-03-30T13:12:52.228 INFO:teuthology.orchestra.run.smithi043.stderr: 72500K .......... .......... .......... .......... .......... 43% 8.36M 11s 2024-03-30T13:12:52.235 INFO:teuthology.orchestra.run.smithi043.stderr: 72550K .......... .......... .......... .......... .......... 43% 7.15M 11s 2024-03-30T13:12:52.240 INFO:teuthology.orchestra.run.smithi043.stderr: 72600K .......... .......... .......... .......... .......... 43% 8.54M 11s 2024-03-30T13:12:52.246 INFO:teuthology.orchestra.run.smithi043.stderr: 72650K .......... .......... .......... .......... .......... 43% 8.23M 11s 2024-03-30T13:12:52.252 INFO:teuthology.orchestra.run.smithi043.stderr: 72700K .......... .......... .......... .......... .......... 43% 8.31M 11s 2024-03-30T13:12:52.259 INFO:teuthology.orchestra.run.smithi043.stderr: 72750K .......... .......... .......... .......... .......... 43% 6.71M 11s 2024-03-30T13:12:52.266 INFO:teuthology.orchestra.run.smithi043.stderr: 72800K .......... .......... .......... .......... .......... 43% 7.95M 11s 2024-03-30T13:12:52.272 INFO:teuthology.orchestra.run.smithi043.stderr: 72850K .......... .......... .......... .......... .......... 43% 7.78M 11s 2024-03-30T13:12:52.279 INFO:teuthology.orchestra.run.smithi043.stderr: 72900K .......... .......... .......... .......... .......... 43% 7.14M 11s 2024-03-30T13:12:52.287 INFO:teuthology.orchestra.run.smithi043.stderr: 72950K .......... .......... .......... .......... .......... 43% 5.95M 11s 2024-03-30T13:12:52.294 INFO:teuthology.orchestra.run.smithi043.stderr: 73000K .......... .......... .......... .......... .......... 43% 6.71M 11s 2024-03-30T13:12:52.301 INFO:teuthology.orchestra.run.smithi043.stderr: 73050K .......... .......... .......... .......... .......... 43% 6.75M 11s 2024-03-30T13:12:52.311 INFO:teuthology.orchestra.run.smithi043.stderr: 73100K .......... .......... .......... .......... .......... 43% 6.64M 11s 2024-03-30T13:12:52.317 INFO:teuthology.orchestra.run.smithi043.stderr: 73150K .......... .......... .......... .......... .......... 43% 6.03M 11s 2024-03-30T13:12:52.324 INFO:teuthology.orchestra.run.smithi043.stderr: 73200K .......... .......... .......... .......... .......... 43% 6.87M 11s 2024-03-30T13:12:52.331 INFO:teuthology.orchestra.run.smithi043.stderr: 73250K .......... .......... .......... .......... .......... 43% 7.35M 11s 2024-03-30T13:12:52.338 INFO:teuthology.orchestra.run.smithi043.stderr: 73300K .......... .......... .......... .......... .......... 43% 7.50M 11s 2024-03-30T13:12:52.345 INFO:teuthology.orchestra.run.smithi043.stderr: 73350K .......... .......... .......... .......... .......... 43% 6.04M 11s 2024-03-30T13:12:52.352 INFO:teuthology.orchestra.run.smithi043.stderr: 73400K .......... .......... .......... .......... .......... 43% 7.17M 11s 2024-03-30T13:12:52.359 INFO:teuthology.orchestra.run.smithi043.stderr: 73450K .......... .......... .......... .......... .......... 43% 7.10M 11s 2024-03-30T13:12:52.366 INFO:teuthology.orchestra.run.smithi043.stderr: 73500K .......... .......... .......... .......... .......... 43% 7.06M 11s 2024-03-30T13:12:52.374 INFO:teuthology.orchestra.run.smithi043.stderr: 73550K .......... .......... .......... .......... .......... 43% 6.13M 11s 2024-03-30T13:12:52.381 INFO:teuthology.orchestra.run.smithi043.stderr: 73600K .......... .......... .......... .......... .......... 43% 6.82M 11s 2024-03-30T13:12:52.388 INFO:teuthology.orchestra.run.smithi043.stderr: 73650K .......... .......... .......... .......... .......... 43% 7.20M 11s 2024-03-30T13:12:52.394 INFO:teuthology.orchestra.run.smithi043.stderr: 73700K .......... .......... .......... .......... .......... 43% 7.79M 11s 2024-03-30T13:12:52.399 INFO:teuthology.orchestra.run.smithi043.stderr: 73750K .......... .......... .......... .......... .......... 43% 8.26M 11s 2024-03-30T13:12:52.405 INFO:teuthology.orchestra.run.smithi043.stderr: 73800K .......... .......... .......... .......... .......... 43% 9.83M 11s 2024-03-30T13:12:52.410 INFO:teuthology.orchestra.run.smithi043.stderr: 73850K .......... .......... .......... .......... .......... 43% 9.97M 11s 2024-03-30T13:12:52.416 INFO:teuthology.orchestra.run.smithi043.stderr: 73900K .......... .......... .......... .......... .......... 43% 8.57M 11s 2024-03-30T13:12:52.420 INFO:teuthology.orchestra.run.smithi043.stderr: 73950K .......... .......... .......... .......... .......... 43% 9.23M 10s 2024-03-30T13:12:52.425 INFO:teuthology.orchestra.run.smithi043.stderr: 74000K .......... .......... .......... .......... .......... 43% 11.2M 10s 2024-03-30T13:12:52.429 INFO:teuthology.orchestra.run.smithi043.stderr: 74050K .......... .......... .......... .......... .......... 43% 11.6M 10s 2024-03-30T13:12:52.433 INFO:teuthology.orchestra.run.smithi043.stderr: 74100K .......... .......... .......... .......... .......... 43% 11.8M 10s 2024-03-30T13:12:52.437 INFO:teuthology.orchestra.run.smithi043.stderr: 74150K .......... .......... .......... .......... .......... 44% 11.2M 10s 2024-03-30T13:12:52.441 INFO:teuthology.orchestra.run.smithi043.stderr: 74200K .......... .......... .......... .......... .......... 44% 13.0M 10s 2024-03-30T13:12:52.446 INFO:teuthology.orchestra.run.smithi043.stderr: 74250K .......... .......... .......... .......... .......... 44% 12.3M 10s 2024-03-30T13:12:52.450 INFO:teuthology.orchestra.run.smithi043.stderr: 74300K .......... .......... .......... .......... .......... 44% 10.6M 10s 2024-03-30T13:12:52.456 INFO:teuthology.orchestra.run.smithi043.stderr: 74350K .......... .......... .......... .......... .......... 44% 8.88M 10s 2024-03-30T13:12:52.460 INFO:teuthology.orchestra.run.smithi043.stderr: 74400K .......... .......... .......... .......... .......... 44% 9.75M 10s 2024-03-30T13:12:52.466 INFO:teuthology.orchestra.run.smithi043.stderr: 74450K .......... .......... .......... .......... .......... 44% 9.37M 10s 2024-03-30T13:12:52.471 INFO:teuthology.orchestra.run.smithi043.stderr: 74500K .......... .......... .......... .......... .......... 44% 8.71M 10s 2024-03-30T13:12:52.477 INFO:teuthology.orchestra.run.smithi043.stderr: 74550K .......... .......... .......... .......... .......... 44% 8.03M 10s 2024-03-30T13:12:52.482 INFO:teuthology.orchestra.run.smithi043.stderr: 74600K .......... .......... .......... .......... .......... 44% 9.48M 10s 2024-03-30T13:12:52.488 INFO:teuthology.orchestra.run.smithi043.stderr: 74650K .......... .......... .......... .......... .......... 44% 9.32M 10s 2024-03-30T13:12:52.493 INFO:teuthology.orchestra.run.smithi043.stderr: 74700K .......... .......... .......... .......... .......... 44% 9.27M 10s 2024-03-30T13:12:52.499 INFO:teuthology.orchestra.run.smithi043.stderr: 74750K .......... .......... .......... .......... .......... 44% 7.71M 10s 2024-03-30T13:12:52.505 INFO:teuthology.orchestra.run.smithi043.stderr: 74800K .......... .......... .......... .......... .......... 44% 9.23M 10s 2024-03-30T13:12:52.510 INFO:teuthology.orchestra.run.smithi043.stderr: 74850K .......... .......... .......... .......... .......... 44% 8.70M 10s 2024-03-30T13:12:52.516 INFO:teuthology.orchestra.run.smithi043.stderr: 74900K .......... .......... .......... .......... .......... 44% 7.97M 10s 2024-03-30T13:12:52.523 INFO:teuthology.orchestra.run.smithi043.stderr: 74950K .......... .......... .......... .......... .......... 44% 6.96M 10s 2024-03-30T13:12:52.529 INFO:teuthology.orchestra.run.smithi043.stderr: 75000K .......... .......... .......... .......... .......... 44% 8.15M 10s 2024-03-30T13:12:52.536 INFO:teuthology.orchestra.run.smithi043.stderr: 75050K .......... .......... .......... .......... .......... 44% 7.79M 10s 2024-03-30T13:12:52.542 INFO:teuthology.orchestra.run.smithi043.stderr: 75100K .......... .......... .......... .......... .......... 44% 7.70M 10s 2024-03-30T13:12:52.550 INFO:teuthology.orchestra.run.smithi043.stderr: 75150K .......... .......... .......... .......... .......... 44% 6.15M 10s 2024-03-30T13:12:52.557 INFO:teuthology.orchestra.run.smithi043.stderr: 75200K .......... .......... .......... .......... .......... 44% 7.27M 10s 2024-03-30T13:12:52.564 INFO:teuthology.orchestra.run.smithi043.stderr: 75250K .......... .......... .......... .......... .......... 44% 7.32M 10s 2024-03-30T13:12:52.570 INFO:teuthology.orchestra.run.smithi043.stderr: 75300K .......... .......... .......... .......... .......... 44% 7.04M 10s 2024-03-30T13:12:52.578 INFO:teuthology.orchestra.run.smithi043.stderr: 75350K .......... .......... .......... .......... .......... 44% 6.15M 10s 2024-03-30T13:12:52.585 INFO:teuthology.orchestra.run.smithi043.stderr: 75400K .......... .......... .......... .......... .......... 44% 7.63M 10s 2024-03-30T13:12:52.592 INFO:teuthology.orchestra.run.smithi043.stderr: 75450K .......... .......... .......... .......... .......... 44% 7.18M 10s 2024-03-30T13:12:52.599 INFO:teuthology.orchestra.run.smithi043.stderr: 75500K .......... .......... .......... .......... .......... 44% 7.05M 10s 2024-03-30T13:12:52.606 INFO:teuthology.orchestra.run.smithi043.stderr: 75550K .......... .......... .......... .......... .......... 44% 6.39M 10s 2024-03-30T13:12:52.613 INFO:teuthology.orchestra.run.smithi043.stderr: 75600K .......... .......... .......... .......... .......... 44% 6.96M 10s 2024-03-30T13:12:52.620 INFO:teuthology.orchestra.run.smithi043.stderr: 75650K .......... .......... .......... .......... .......... 44% 7.16M 10s 2024-03-30T13:12:52.627 INFO:teuthology.orchestra.run.smithi043.stderr: 75700K .......... .......... .......... .......... .......... 44% 7.07M 10s 2024-03-30T13:12:52.635 INFO:teuthology.orchestra.run.smithi043.stderr: 75750K .......... .......... .......... .......... .......... 44% 6.08M 10s 2024-03-30T13:12:52.642 INFO:teuthology.orchestra.run.smithi043.stderr: 75800K .......... .......... .......... .......... .......... 44% 7.26M 10s 2024-03-30T13:12:52.649 INFO:teuthology.orchestra.run.smithi043.stderr: 75850K .......... .......... .......... .......... .......... 45% 6.98M 10s 2024-03-30T13:12:52.656 INFO:teuthology.orchestra.run.smithi043.stderr: 75900K .......... .......... .......... .......... .......... 45% 6.80M 10s 2024-03-30T13:12:52.664 INFO:teuthology.orchestra.run.smithi043.stderr: 75950K .......... .......... .......... .......... .......... 45% 5.80M 10s 2024-03-30T13:12:52.671 INFO:teuthology.orchestra.run.smithi043.stderr: 76000K .......... .......... .......... .......... .......... 45% 7.14M 10s 2024-03-30T13:12:52.678 INFO:teuthology.orchestra.run.smithi043.stderr: 76050K .......... .......... .......... .......... .......... 45% 6.95M 10s 2024-03-30T13:12:52.685 INFO:teuthology.orchestra.run.smithi043.stderr: 76100K .......... .......... .......... .......... .......... 45% 6.89M 10s 2024-03-30T13:12:52.694 INFO:teuthology.orchestra.run.smithi043.stderr: 76150K .......... .......... .......... .......... .......... 45% 6.03M 10s 2024-03-30T13:12:52.701 INFO:teuthology.orchestra.run.smithi043.stderr: 76200K .......... .......... .......... .......... .......... 45% 6.66M 10s 2024-03-30T13:12:52.708 INFO:teuthology.orchestra.run.smithi043.stderr: 76250K .......... .......... .......... .......... .......... 45% 6.80M 10s 2024-03-30T13:12:52.715 INFO:teuthology.orchestra.run.smithi043.stderr: 76300K .......... .......... .......... .......... .......... 45% 6.96M 10s 2024-03-30T13:12:52.723 INFO:teuthology.orchestra.run.smithi043.stderr: 76350K .......... .......... .......... .......... .......... 45% 5.96M 10s 2024-03-30T13:12:52.730 INFO:teuthology.orchestra.run.smithi043.stderr: 76400K .......... .......... .......... .......... .......... 45% 6.94M 10s 2024-03-30T13:12:52.737 INFO:teuthology.orchestra.run.smithi043.stderr: 76450K .......... .......... .......... .......... .......... 45% 6.87M 10s 2024-03-30T13:12:52.744 INFO:teuthology.orchestra.run.smithi043.stderr: 76500K .......... .......... .......... .......... .......... 45% 7.21M 10s 2024-03-30T13:12:52.752 INFO:teuthology.orchestra.run.smithi043.stderr: 76550K .......... .......... .......... .......... .......... 45% 5.91M 10s 2024-03-30T13:12:52.759 INFO:teuthology.orchestra.run.smithi043.stderr: 76600K .......... .......... .......... .......... .......... 45% 7.10M 10s 2024-03-30T13:12:52.766 INFO:teuthology.orchestra.run.smithi043.stderr: 76650K .......... .......... .......... .......... .......... 45% 6.93M 10s 2024-03-30T13:12:52.773 INFO:teuthology.orchestra.run.smithi043.stderr: 76700K .......... .......... .......... .......... .......... 45% 6.81M 10s 2024-03-30T13:12:52.782 INFO:teuthology.orchestra.run.smithi043.stderr: 76750K .......... .......... .......... .......... .......... 45% 5.66M 10s 2024-03-30T13:12:52.788 INFO:teuthology.orchestra.run.smithi043.stderr: 76800K .......... .......... .......... .......... .......... 45% 7.88M 10s 2024-03-30T13:12:52.796 INFO:teuthology.orchestra.run.smithi043.stderr: 76850K .......... .......... .......... .......... .......... 45% 6.68M 10s 2024-03-30T13:12:52.803 INFO:teuthology.orchestra.run.smithi043.stderr: 76900K .......... .......... .......... .......... .......... 45% 6.60M 10s 2024-03-30T13:12:52.811 INFO:teuthology.orchestra.run.smithi043.stderr: 76950K .......... .......... .......... .......... .......... 45% 5.74M 10s 2024-03-30T13:12:52.818 INFO:teuthology.orchestra.run.smithi043.stderr: 77000K .......... .......... .......... .......... .......... 45% 6.84M 10s 2024-03-30T13:12:52.826 INFO:teuthology.orchestra.run.smithi043.stderr: 77050K .......... .......... .......... .......... .......... 45% 6.92M 10s 2024-03-30T13:12:52.833 INFO:teuthology.orchestra.run.smithi043.stderr: 77100K .......... .......... .......... .......... .......... 45% 6.90M 10s 2024-03-30T13:12:52.841 INFO:teuthology.orchestra.run.smithi043.stderr: 77150K .......... .......... .......... .......... .......... 45% 6.03M 10s 2024-03-30T13:12:52.848 INFO:teuthology.orchestra.run.smithi043.stderr: 77200K .......... .......... .......... .......... .......... 45% 6.82M 10s 2024-03-30T13:12:52.855 INFO:teuthology.orchestra.run.smithi043.stderr: 77250K .......... .......... .......... .......... .......... 45% 6.76M 10s 2024-03-30T13:12:52.862 INFO:teuthology.orchestra.run.smithi043.stderr: 77300K .......... .......... .......... .......... .......... 45% 6.79M 10s 2024-03-30T13:12:52.871 INFO:teuthology.orchestra.run.smithi043.stderr: 77350K .......... .......... .......... .......... .......... 45% 5.78M 10s 2024-03-30T13:12:52.878 INFO:teuthology.orchestra.run.smithi043.stderr: 77400K .......... .......... .......... .......... .......... 45% 6.60M 10s 2024-03-30T13:12:52.886 INFO:teuthology.orchestra.run.smithi043.stderr: 77450K .......... .......... .......... .......... .......... 45% 6.85M 10s 2024-03-30T13:12:52.893 INFO:teuthology.orchestra.run.smithi043.stderr: 77500K .......... .......... .......... .......... .......... 46% 6.35M 10s 2024-03-30T13:12:52.902 INFO:teuthology.orchestra.run.smithi043.stderr: 77550K .......... .......... .......... .......... .......... 46% 5.48M 10s 2024-03-30T13:12:52.910 INFO:teuthology.orchestra.run.smithi043.stderr: 77600K .......... .......... .......... .......... .......... 46% 6.43M 10s 2024-03-30T13:12:52.918 INFO:teuthology.orchestra.run.smithi043.stderr: 77650K .......... .......... .......... .......... .......... 46% 6.03M 10s 2024-03-30T13:12:52.926 INFO:teuthology.orchestra.run.smithi043.stderr: 77700K .......... .......... .......... .......... .......... 46% 6.14M 10s 2024-03-30T13:12:52.935 INFO:teuthology.orchestra.run.smithi043.stderr: 77750K .......... .......... .......... .......... .......... 46% 5.36M 10s 2024-03-30T13:12:52.942 INFO:teuthology.orchestra.run.smithi043.stderr: 77800K .......... .......... .......... .......... .......... 46% 6.65M 10s 2024-03-30T13:12:52.947 INFO:teuthology.orchestra.run.smithi043.stderr: 77850K .......... .......... .......... .......... .......... 46% 9.58M 10s 2024-03-30T13:12:52.952 INFO:teuthology.orchestra.run.smithi043.stderr: 77900K .......... .......... .......... .......... .......... 46% 9.45M 10s 2024-03-30T13:12:52.958 INFO:teuthology.orchestra.run.smithi043.stderr: 77950K .......... .......... .......... .......... .......... 46% 8.70M 10s 2024-03-30T13:12:52.963 INFO:teuthology.orchestra.run.smithi043.stderr: 78000K .......... .......... .......... .......... .......... 46% 9.73M 10s 2024-03-30T13:12:52.968 INFO:teuthology.orchestra.run.smithi043.stderr: 78050K .......... .......... .......... .......... .......... 46% 10.8M 10s 2024-03-30T13:12:52.972 INFO:teuthology.orchestra.run.smithi043.stderr: 78100K .......... .......... .......... .......... .......... 46% 9.02M 10s 2024-03-30T13:12:52.978 INFO:teuthology.orchestra.run.smithi043.stderr: 78150K .......... .......... .......... .......... .......... 46% 8.69M 10s 2024-03-30T13:12:52.982 INFO:teuthology.orchestra.run.smithi043.stderr: 78200K .......... .......... .......... .......... .......... 46% 10.8M 10s 2024-03-30T13:12:52.987 INFO:teuthology.orchestra.run.smithi043.stderr: 78250K .......... .......... .......... .......... .......... 46% 9.84M 10s 2024-03-30T13:12:52.992 INFO:teuthology.orchestra.run.smithi043.stderr: 78300K .......... .......... .......... .......... .......... 46% 10.7M 10s 2024-03-30T13:12:52.997 INFO:teuthology.orchestra.run.smithi043.stderr: 78350K .......... .......... .......... .......... .......... 46% 9.08M 10s 2024-03-30T13:12:53.002 INFO:teuthology.orchestra.run.smithi043.stderr: 78400K .......... .......... .......... .......... .......... 46% 10.1M 10s 2024-03-30T13:12:53.007 INFO:teuthology.orchestra.run.smithi043.stderr: 78450K .......... .......... .......... .......... .......... 46% 9.71M 10s 2024-03-30T13:12:53.012 INFO:teuthology.orchestra.run.smithi043.stderr: 78500K .......... .......... .......... .......... .......... 46% 9.98M 10s 2024-03-30T13:12:53.018 INFO:teuthology.orchestra.run.smithi043.stderr: 78550K .......... .......... .......... .......... .......... 46% 8.27M 10s 2024-03-30T13:12:53.024 INFO:teuthology.orchestra.run.smithi043.stderr: 78600K .......... .......... .......... .......... .......... 46% 8.96M 10s 2024-03-30T13:12:53.029 INFO:teuthology.orchestra.run.smithi043.stderr: 78650K .......... .......... .......... .......... .......... 46% 9.10M 10s 2024-03-30T13:12:53.035 INFO:teuthology.orchestra.run.smithi043.stderr: 78700K .......... .......... .......... .......... .......... 46% 8.67M 10s 2024-03-30T13:12:53.040 INFO:teuthology.orchestra.run.smithi043.stderr: 78750K .......... .......... .......... .......... .......... 46% 8.40M 10s 2024-03-30T13:12:53.045 INFO:teuthology.orchestra.run.smithi043.stderr: 78800K .......... .......... .......... .......... .......... 46% 10.2M 10s 2024-03-30T13:12:53.051 INFO:teuthology.orchestra.run.smithi043.stderr: 78850K .......... .......... .......... .......... .......... 46% 9.37M 10s 2024-03-30T13:12:53.056 INFO:teuthology.orchestra.run.smithi043.stderr: 78900K .......... .......... .......... .......... .......... 46% 8.47M 10s 2024-03-30T13:12:53.062 INFO:teuthology.orchestra.run.smithi043.stderr: 78950K .......... .......... .......... .......... .......... 46% 8.25M 10s 2024-03-30T13:12:53.067 INFO:teuthology.orchestra.run.smithi043.stderr: 79000K .......... .......... .......... .......... .......... 46% 9.70M 10s 2024-03-30T13:12:53.072 INFO:teuthology.orchestra.run.smithi043.stderr: 79050K .......... .......... .......... .......... .......... 46% 9.41M 10s 2024-03-30T13:12:53.078 INFO:teuthology.orchestra.run.smithi043.stderr: 79100K .......... .......... .......... .......... .......... 46% 9.22M 10s 2024-03-30T13:12:53.084 INFO:teuthology.orchestra.run.smithi043.stderr: 79150K .......... .......... .......... .......... .......... 46% 7.83M 10s 2024-03-30T13:12:53.089 INFO:teuthology.orchestra.run.smithi043.stderr: 79200K .......... .......... .......... .......... .......... 47% 9.11M 10s 2024-03-30T13:12:53.094 INFO:teuthology.orchestra.run.smithi043.stderr: 79250K .......... .......... .......... .......... .......... 47% 9.63M 10s 2024-03-30T13:12:53.100 INFO:teuthology.orchestra.run.smithi043.stderr: 79300K .......... .......... .......... .......... .......... 47% 9.18M 10s 2024-03-30T13:12:53.106 INFO:teuthology.orchestra.run.smithi043.stderr: 79350K .......... .......... .......... .......... .......... 47% 7.85M 10s 2024-03-30T13:12:53.111 INFO:teuthology.orchestra.run.smithi043.stderr: 79400K .......... .......... .......... .......... .......... 47% 9.57M 10s 2024-03-30T13:12:53.116 INFO:teuthology.orchestra.run.smithi043.stderr: 79450K .......... .......... .......... .......... .......... 47% 9.48M 10s 2024-03-30T13:12:53.122 INFO:teuthology.orchestra.run.smithi043.stderr: 79500K .......... .......... .......... .......... .......... 47% 9.28M 10s 2024-03-30T13:12:53.130 INFO:teuthology.orchestra.run.smithi043.stderr: 79550K .......... .......... .......... .......... .......... 47% 7.41M 10s 2024-03-30T13:12:53.133 INFO:teuthology.orchestra.run.smithi043.stderr: 79600K .......... .......... .......... .......... .......... 47% 9.22M 10s 2024-03-30T13:12:53.138 INFO:teuthology.orchestra.run.smithi043.stderr: 79650K .......... .......... .......... .......... .......... 47% 10.4M 10s 2024-03-30T13:12:53.143 INFO:teuthology.orchestra.run.smithi043.stderr: 79700K .......... .......... .......... .......... .......... 47% 9.78M 10s 2024-03-30T13:12:53.149 INFO:teuthology.orchestra.run.smithi043.stderr: 79750K .......... .......... .......... .......... .......... 47% 8.28M 10s 2024-03-30T13:12:53.154 INFO:teuthology.orchestra.run.smithi043.stderr: 79800K .......... .......... .......... .......... .......... 47% 9.64M 10s 2024-03-30T13:12:53.159 INFO:teuthology.orchestra.run.smithi043.stderr: 79850K .......... .......... .......... .......... .......... 47% 9.75M 10s 2024-03-30T13:12:53.165 INFO:teuthology.orchestra.run.smithi043.stderr: 79900K .......... .......... .......... .......... .......... 47% 9.85M 10s 2024-03-30T13:12:53.170 INFO:teuthology.orchestra.run.smithi043.stderr: 79950K .......... .......... .......... .......... .......... 47% 8.22M 10s 2024-03-30T13:12:53.175 INFO:teuthology.orchestra.run.smithi043.stderr: 80000K .......... .......... .......... .......... .......... 47% 9.28M 10s 2024-03-30T13:12:53.180 INFO:teuthology.orchestra.run.smithi043.stderr: 80050K .......... .......... .......... .......... .......... 47% 9.29M 10s 2024-03-30T13:12:53.186 INFO:teuthology.orchestra.run.smithi043.stderr: 80100K .......... .......... .......... .......... .......... 47% 8.88M 10s 2024-03-30T13:12:53.192 INFO:teuthology.orchestra.run.smithi043.stderr: 80150K .......... .......... .......... .......... .......... 47% 8.40M 10s 2024-03-30T13:12:53.197 INFO:teuthology.orchestra.run.smithi043.stderr: 80200K .......... .......... .......... .......... .......... 47% 9.70M 10s 2024-03-30T13:12:53.202 INFO:teuthology.orchestra.run.smithi043.stderr: 80250K .......... .......... .......... .......... .......... 47% 9.53M 10s 2024-03-30T13:12:53.207 INFO:teuthology.orchestra.run.smithi043.stderr: 80300K .......... .......... .......... .......... .......... 47% 9.55M 10s 2024-03-30T13:12:53.213 INFO:teuthology.orchestra.run.smithi043.stderr: 80350K .......... .......... .......... .......... .......... 47% 7.84M 10s 2024-03-30T13:12:53.219 INFO:teuthology.orchestra.run.smithi043.stderr: 80400K .......... .......... .......... .......... .......... 47% 8.11M 10s 2024-03-30T13:12:53.225 INFO:teuthology.orchestra.run.smithi043.stderr: 80450K .......... .......... .......... .......... .......... 47% 8.90M 10s 2024-03-30T13:12:53.231 INFO:teuthology.orchestra.run.smithi043.stderr: 80500K .......... .......... .......... .......... .......... 47% 8.29M 10s 2024-03-30T13:12:53.238 INFO:teuthology.orchestra.run.smithi043.stderr: 80550K .......... .......... .......... .......... .......... 47% 6.80M 10s 2024-03-30T13:12:53.244 INFO:teuthology.orchestra.run.smithi043.stderr: 80600K .......... .......... .......... .......... .......... 47% 8.24M 10s 2024-03-30T13:12:53.249 INFO:teuthology.orchestra.run.smithi043.stderr: 80650K .......... .......... .......... .......... .......... 47% 8.99M 10s 2024-03-30T13:12:53.254 INFO:teuthology.orchestra.run.smithi043.stderr: 80700K .......... .......... .......... .......... .......... 47% 9.33M 10s 2024-03-30T13:12:53.261 INFO:teuthology.orchestra.run.smithi043.stderr: 80750K .......... .......... .......... .......... .......... 47% 7.49M 10s 2024-03-30T13:12:53.266 INFO:teuthology.orchestra.run.smithi043.stderr: 80800K .......... .......... .......... .......... .......... 47% 9.00M 10s 2024-03-30T13:12:53.272 INFO:teuthology.orchestra.run.smithi043.stderr: 80850K .......... .......... .......... .......... .......... 47% 9.07M 10s 2024-03-30T13:12:53.277 INFO:teuthology.orchestra.run.smithi043.stderr: 80900K .......... .......... .......... .......... .......... 48% 9.08M 10s 2024-03-30T13:12:53.284 INFO:teuthology.orchestra.run.smithi043.stderr: 80950K .......... .......... .......... .......... .......... 48% 7.09M 10s 2024-03-30T13:12:53.290 INFO:teuthology.orchestra.run.smithi043.stderr: 81000K .......... .......... .......... .......... .......... 48% 8.70M 10s 2024-03-30T13:12:53.296 INFO:teuthology.orchestra.run.smithi043.stderr: 81050K .......... .......... .......... .......... .......... 48% 8.00M 10s 2024-03-30T13:12:53.302 INFO:teuthology.orchestra.run.smithi043.stderr: 81100K .......... .......... .......... .......... .......... 48% 7.66M 10s 2024-03-30T13:12:53.309 INFO:teuthology.orchestra.run.smithi043.stderr: 81150K .......... .......... .......... .......... .......... 48% 6.65M 10s 2024-03-30T13:12:53.316 INFO:teuthology.orchestra.run.smithi043.stderr: 81200K .......... .......... .......... .......... .......... 48% 7.46M 10s 2024-03-30T13:12:53.323 INFO:teuthology.orchestra.run.smithi043.stderr: 81250K .......... .......... .......... .......... .......... 48% 7.11M 10s 2024-03-30T13:12:53.330 INFO:teuthology.orchestra.run.smithi043.stderr: 81300K .......... .......... .......... .......... .......... 48% 7.40M 10s 2024-03-30T13:12:53.337 INFO:teuthology.orchestra.run.smithi043.stderr: 81350K .......... .......... .......... .......... .......... 48% 6.80M 10s 2024-03-30T13:12:53.342 INFO:teuthology.orchestra.run.smithi043.stderr: 81400K .......... .......... .......... .......... .......... 48% 8.26M 10s 2024-03-30T13:12:53.348 INFO:teuthology.orchestra.run.smithi043.stderr: 81450K .......... .......... .......... .......... .......... 48% 8.43M 10s 2024-03-30T13:12:53.354 INFO:teuthology.orchestra.run.smithi043.stderr: 81500K .......... .......... .......... .......... .......... 48% 8.02M 10s 2024-03-30T13:12:53.362 INFO:teuthology.orchestra.run.smithi043.stderr: 81550K .......... .......... .......... .......... .......... 48% 6.76M 10s 2024-03-30T13:12:53.368 INFO:teuthology.orchestra.run.smithi043.stderr: 81600K .......... .......... .......... .......... .......... 48% 7.97M 10s 2024-03-30T13:12:53.374 INFO:teuthology.orchestra.run.smithi043.stderr: 81650K .......... .......... .......... .......... .......... 48% 8.17M 10s 2024-03-30T13:12:53.380 INFO:teuthology.orchestra.run.smithi043.stderr: 81700K .......... .......... .......... .......... .......... 48% 7.94M 10s 2024-03-30T13:12:53.387 INFO:teuthology.orchestra.run.smithi043.stderr: 81750K .......... .......... .......... .......... .......... 48% 6.73M 10s 2024-03-30T13:12:53.393 INFO:teuthology.orchestra.run.smithi043.stderr: 81800K .......... .......... .......... .......... .......... 48% 7.93M 10s 2024-03-30T13:12:53.400 INFO:teuthology.orchestra.run.smithi043.stderr: 81850K .......... .......... .......... .......... .......... 48% 7.94M 10s 2024-03-30T13:12:53.406 INFO:teuthology.orchestra.run.smithi043.stderr: 81900K .......... .......... .......... .......... .......... 48% 7.96M 10s 2024-03-30T13:12:53.411 INFO:teuthology.orchestra.run.smithi043.stderr: 81950K .......... .......... .......... .......... .......... 48% 9.01M 10s 2024-03-30T13:12:53.416 INFO:teuthology.orchestra.run.smithi043.stderr: 82000K .......... .......... .......... .......... .......... 48% 9.25M 10s 2024-03-30T13:12:53.422 INFO:teuthology.orchestra.run.smithi043.stderr: 82050K .......... .......... .......... .......... .......... 48% 9.03M 10s 2024-03-30T13:12:53.427 INFO:teuthology.orchestra.run.smithi043.stderr: 82100K .......... .......... .......... .......... .......... 48% 9.17M 10s 2024-03-30T13:12:53.432 INFO:teuthology.orchestra.run.smithi043.stderr: 82150K .......... .......... .......... .......... .......... 48% 9.33M 10s 2024-03-30T13:12:53.436 INFO:teuthology.orchestra.run.smithi043.stderr: 82200K .......... .......... .......... .......... .......... 48% 11.8M 10s 2024-03-30T13:12:53.441 INFO:teuthology.orchestra.run.smithi043.stderr: 82250K .......... .......... .......... .......... .......... 48% 11.1M 10s 2024-03-30T13:12:53.445 INFO:teuthology.orchestra.run.smithi043.stderr: 82300K .......... .......... .......... .......... .......... 48% 10.6M 10s 2024-03-30T13:12:53.451 INFO:teuthology.orchestra.run.smithi043.stderr: 82350K .......... .......... .......... .......... .......... 48% 8.60M 10s 2024-03-30T13:12:53.456 INFO:teuthology.orchestra.run.smithi043.stderr: 82400K .......... .......... .......... .......... .......... 48% 9.48M 10s 2024-03-30T13:12:53.461 INFO:teuthology.orchestra.run.smithi043.stderr: 82450K .......... .......... .......... .......... .......... 48% 10.3M 10s 2024-03-30T13:12:53.466 INFO:teuthology.orchestra.run.smithi043.stderr: 82500K .......... .......... .......... .......... .......... 48% 10.0M 10s 2024-03-30T13:12:53.471 INFO:teuthology.orchestra.run.smithi043.stderr: 82550K .......... .......... .......... .......... .......... 49% 8.50M 10s 2024-03-30T13:12:53.476 INFO:teuthology.orchestra.run.smithi043.stderr: 82600K .......... .......... .......... .......... .......... 49% 9.92M 10s 2024-03-30T13:12:53.481 INFO:teuthology.orchestra.run.smithi043.stderr: 82650K .......... .......... .......... .......... .......... 49% 9.88M 10s 2024-03-30T13:12:53.486 INFO:teuthology.orchestra.run.smithi043.stderr: 82700K .......... .......... .......... .......... .......... 49% 9.75M 10s 2024-03-30T13:12:53.492 INFO:teuthology.orchestra.run.smithi043.stderr: 82750K .......... .......... .......... .......... .......... 49% 8.64M 10s 2024-03-30T13:12:53.497 INFO:teuthology.orchestra.run.smithi043.stderr: 82800K .......... .......... .......... .......... .......... 49% 9.11M 10s 2024-03-30T13:12:53.503 INFO:teuthology.orchestra.run.smithi043.stderr: 82850K .......... .......... .......... .......... .......... 49% 9.14M 10s 2024-03-30T13:12:53.507 INFO:teuthology.orchestra.run.smithi043.stderr: 82900K .......... .......... .......... .......... .......... 49% 11.2M 10s 2024-03-30T13:12:53.512 INFO:teuthology.orchestra.run.smithi043.stderr: 82950K .......... .......... .......... .......... .......... 49% 9.80M 10s 2024-03-30T13:12:53.518 INFO:teuthology.orchestra.run.smithi043.stderr: 83000K .......... .......... .......... .......... .......... 49% 8.28M 10s 2024-03-30T13:12:53.524 INFO:teuthology.orchestra.run.smithi043.stderr: 83050K .......... .......... .......... .......... .......... 49% 8.20M 10s 2024-03-30T13:12:53.530 INFO:teuthology.orchestra.run.smithi043.stderr: 83100K .......... .......... .......... .......... .......... 49% 8.29M 10s 2024-03-30T13:12:53.537 INFO:teuthology.orchestra.run.smithi043.stderr: 83150K .......... .......... .......... .......... .......... 49% 7.29M 10s 2024-03-30T13:12:53.542 INFO:teuthology.orchestra.run.smithi043.stderr: 83200K .......... .......... .......... .......... .......... 49% 8.45M 10s 2024-03-30T13:12:53.548 INFO:teuthology.orchestra.run.smithi043.stderr: 83250K .......... .......... .......... .......... .......... 49% 8.12M 10s 2024-03-30T13:12:53.554 INFO:teuthology.orchestra.run.smithi043.stderr: 83300K .......... .......... .......... .......... .......... 49% 8.19M 10s 2024-03-30T13:12:53.561 INFO:teuthology.orchestra.run.smithi043.stderr: 83350K .......... .......... .......... .......... .......... 49% 6.97M 10s 2024-03-30T13:12:53.566 INFO:teuthology.orchestra.run.smithi043.stderr: 83400K .......... .......... .......... .......... .......... 49% 10.7M 10s 2024-03-30T13:12:53.572 INFO:teuthology.orchestra.run.smithi043.stderr: 83450K .......... .......... .......... .......... .......... 49% 8.13M 10s 2024-03-30T13:12:53.578 INFO:teuthology.orchestra.run.smithi043.stderr: 83500K .......... .......... .......... .......... .......... 49% 7.81M 10s 2024-03-30T13:12:53.585 INFO:teuthology.orchestra.run.smithi043.stderr: 83550K .......... .......... .......... .......... .......... 49% 6.85M 10s 2024-03-30T13:12:53.592 INFO:teuthology.orchestra.run.smithi043.stderr: 83600K .......... .......... .......... .......... .......... 49% 7.34M 10s 2024-03-30T13:12:53.599 INFO:teuthology.orchestra.run.smithi043.stderr: 83650K .......... .......... .......... .......... .......... 49% 7.55M 10s 2024-03-30T13:12:53.605 INFO:teuthology.orchestra.run.smithi043.stderr: 83700K .......... .......... .......... .......... .......... 49% 7.54M 10s 2024-03-30T13:12:53.613 INFO:teuthology.orchestra.run.smithi043.stderr: 83750K .......... .......... .......... .......... .......... 49% 6.49M 10s 2024-03-30T13:12:53.619 INFO:teuthology.orchestra.run.smithi043.stderr: 83800K .......... .......... .......... .......... .......... 49% 7.53M 10s 2024-03-30T13:12:53.626 INFO:teuthology.orchestra.run.smithi043.stderr: 83850K .......... .......... .......... .......... .......... 49% 7.60M 10s 2024-03-30T13:12:53.632 INFO:teuthology.orchestra.run.smithi043.stderr: 83900K .......... .......... .......... .......... .......... 49% 7.58M 10s 2024-03-30T13:12:53.640 INFO:teuthology.orchestra.run.smithi043.stderr: 83950K .......... .......... .......... .......... .......... 49% 6.35M 9s 2024-03-30T13:12:53.645 INFO:teuthology.orchestra.run.smithi043.stderr: 84000K .......... .......... .......... .......... .......... 49% 8.79M 9s 2024-03-30T13:12:53.650 INFO:teuthology.orchestra.run.smithi043.stderr: 84050K .......... .......... .......... .......... .......... 49% 10.1M 9s 2024-03-30T13:12:53.655 INFO:teuthology.orchestra.run.smithi043.stderr: 84100K .......... .......... .......... .......... .......... 49% 10.2M 9s 2024-03-30T13:12:53.661 INFO:teuthology.orchestra.run.smithi043.stderr: 84150K .......... .......... .......... .......... .......... 49% 7.58M 9s 2024-03-30T13:12:53.668 INFO:teuthology.orchestra.run.smithi043.stderr: 84200K .......... .......... .......... .......... .......... 49% 7.44M 9s 2024-03-30T13:12:53.675 INFO:teuthology.orchestra.run.smithi043.stderr: 84250K .......... .......... .......... .......... .......... 50% 7.19M 9s 2024-03-30T13:12:53.681 INFO:teuthology.orchestra.run.smithi043.stderr: 84300K .......... .......... .......... .......... .......... 50% 7.50M 9s 2024-03-30T13:12:53.689 INFO:teuthology.orchestra.run.smithi043.stderr: 84350K .......... .......... .......... .......... .......... 50% 6.42M 9s 2024-03-30T13:12:53.695 INFO:teuthology.orchestra.run.smithi043.stderr: 84400K .......... .......... .......... .......... .......... 50% 7.55M 9s 2024-03-30T13:12:53.702 INFO:teuthology.orchestra.run.smithi043.stderr: 84450K .......... .......... .......... .......... .......... 50% 7.54M 9s 2024-03-30T13:12:53.708 INFO:teuthology.orchestra.run.smithi043.stderr: 84500K .......... .......... .......... .......... .......... 50% 7.56M 9s 2024-03-30T13:12:53.715 INFO:teuthology.orchestra.run.smithi043.stderr: 84550K .......... .......... .......... .......... .......... 50% 6.98M 9s 2024-03-30T13:12:53.721 INFO:teuthology.orchestra.run.smithi043.stderr: 84600K .......... .......... .......... .......... .......... 50% 8.08M 9s 2024-03-30T13:12:53.727 INFO:teuthology.orchestra.run.smithi043.stderr: 84650K .......... .......... .......... .......... .......... 50% 8.23M 9s 2024-03-30T13:12:53.733 INFO:teuthology.orchestra.run.smithi043.stderr: 84700K .......... .......... .......... .......... .......... 50% 8.90M 9s 2024-03-30T13:12:53.739 INFO:teuthology.orchestra.run.smithi043.stderr: 84750K .......... .......... .......... .......... .......... 50% 7.52M 9s 2024-03-30T13:12:53.746 INFO:teuthology.orchestra.run.smithi043.stderr: 84800K .......... .......... .......... .......... .......... 50% 8.71M 9s 2024-03-30T13:12:53.750 INFO:teuthology.orchestra.run.smithi043.stderr: 84850K .......... .......... .......... .......... .......... 50% 9.19M 9s 2024-03-30T13:12:53.756 INFO:teuthology.orchestra.run.smithi043.stderr: 84900K .......... .......... .......... .......... .......... 50% 8.17M 9s 2024-03-30T13:12:53.762 INFO:teuthology.orchestra.run.smithi043.stderr: 84950K .......... .......... .......... .......... .......... 50% 7.53M 9s 2024-03-30T13:12:53.768 INFO:teuthology.orchestra.run.smithi043.stderr: 85000K .......... .......... .......... .......... .......... 50% 8.80M 9s 2024-03-30T13:12:53.773 INFO:teuthology.orchestra.run.smithi043.stderr: 85050K .......... .......... .......... .......... .......... 50% 8.85M 9s 2024-03-30T13:12:53.779 INFO:teuthology.orchestra.run.smithi043.stderr: 85100K .......... .......... .......... .......... .......... 50% 8.91M 9s 2024-03-30T13:12:53.786 INFO:teuthology.orchestra.run.smithi043.stderr: 85150K .......... .......... .......... .......... .......... 50% 6.94M 9s 2024-03-30T13:12:53.792 INFO:teuthology.orchestra.run.smithi043.stderr: 85200K .......... .......... .......... .......... .......... 50% 7.60M 9s 2024-03-30T13:12:53.799 INFO:teuthology.orchestra.run.smithi043.stderr: 85250K .......... .......... .......... .......... .......... 50% 7.59M 9s 2024-03-30T13:12:53.806 INFO:teuthology.orchestra.run.smithi043.stderr: 85300K .......... .......... .......... .......... .......... 50% 7.17M 9s 2024-03-30T13:12:53.814 INFO:teuthology.orchestra.run.smithi043.stderr: 85350K .......... .......... .......... .......... .......... 50% 6.16M 9s 2024-03-30T13:12:53.820 INFO:teuthology.orchestra.run.smithi043.stderr: 85400K .......... .......... .......... .......... .......... 50% 7.30M 9s 2024-03-30T13:12:53.827 INFO:teuthology.orchestra.run.smithi043.stderr: 85450K .......... .......... .......... .......... .......... 50% 7.65M 9s 2024-03-30T13:12:53.833 INFO:teuthology.orchestra.run.smithi043.stderr: 85500K .......... .......... .......... .......... .......... 50% 7.18M 9s 2024-03-30T13:12:53.841 INFO:teuthology.orchestra.run.smithi043.stderr: 85550K .......... .......... .......... .......... .......... 50% 6.20M 9s 2024-03-30T13:12:53.848 INFO:teuthology.orchestra.run.smithi043.stderr: 85600K .......... .......... .......... .......... .......... 50% 7.45M 9s 2024-03-30T13:12:53.855 INFO:teuthology.orchestra.run.smithi043.stderr: 85650K .......... .......... .......... .......... .......... 50% 7.32M 9s 2024-03-30T13:12:53.862 INFO:teuthology.orchestra.run.smithi043.stderr: 85700K .......... .......... .......... .......... .......... 50% 7.26M 9s 2024-03-30T13:12:53.869 INFO:teuthology.orchestra.run.smithi043.stderr: 85750K .......... .......... .......... .......... .......... 50% 6.14M 9s 2024-03-30T13:12:53.876 INFO:teuthology.orchestra.run.smithi043.stderr: 85800K .......... .......... .......... .......... .......... 50% 7.22M 9s 2024-03-30T13:12:53.883 INFO:teuthology.orchestra.run.smithi043.stderr: 85850K .......... .......... .......... .......... .......... 50% 7.11M 9s 2024-03-30T13:12:53.890 INFO:teuthology.orchestra.run.smithi043.stderr: 85900K .......... .......... .......... .......... .......... 50% 7.00M 9s 2024-03-30T13:12:53.898 INFO:teuthology.orchestra.run.smithi043.stderr: 85950K .......... .......... .......... .......... .......... 51% 6.29M 9s 2024-03-30T13:12:53.903 INFO:teuthology.orchestra.run.smithi043.stderr: 86000K .......... .......... .......... .......... .......... 51% 8.43M 9s 2024-03-30T13:12:53.908 INFO:teuthology.orchestra.run.smithi043.stderr: 86050K .......... .......... .......... .......... .......... 51% 11.6M 9s 2024-03-30T13:12:53.913 INFO:teuthology.orchestra.run.smithi043.stderr: 86100K .......... .......... .......... .......... .......... 51% 8.80M 9s 2024-03-30T13:12:53.918 INFO:teuthology.orchestra.run.smithi043.stderr: 86150K .......... .......... .......... .......... .......... 51% 8.76M 9s 2024-03-30T13:12:53.922 INFO:teuthology.orchestra.run.smithi043.stderr: 86200K .......... .......... .......... .......... .......... 51% 12.3M 9s 2024-03-30T13:12:53.927 INFO:teuthology.orchestra.run.smithi043.stderr: 86250K .......... .......... .......... .......... .......... 51% 11.7M 9s 2024-03-30T13:12:53.932 INFO:teuthology.orchestra.run.smithi043.stderr: 86300K .......... .......... .......... .......... .......... 51% 9.83M 9s 2024-03-30T13:12:53.937 INFO:teuthology.orchestra.run.smithi043.stderr: 86350K .......... .......... .......... .......... .......... 51% 9.55M 9s 2024-03-30T13:12:53.941 INFO:teuthology.orchestra.run.smithi043.stderr: 86400K .......... .......... .......... .......... .......... 51% 10.9M 9s 2024-03-30T13:12:53.946 INFO:teuthology.orchestra.run.smithi043.stderr: 86450K .......... .......... .......... .......... .......... 51% 9.87M 9s 2024-03-30T13:12:53.954 INFO:teuthology.orchestra.run.smithi043.stderr: 86500K .......... .......... .......... .......... .......... 51% 9.79M 9s 2024-03-30T13:12:53.958 INFO:teuthology.orchestra.run.smithi043.stderr: 86550K .......... .......... .......... .......... .......... 51% 7.61M 9s 2024-03-30T13:12:53.963 INFO:teuthology.orchestra.run.smithi043.stderr: 86600K .......... .......... .......... .......... .......... 51% 9.20M 9s 2024-03-30T13:12:53.968 INFO:teuthology.orchestra.run.smithi043.stderr: 86650K .......... .......... .......... .......... .......... 51% 9.37M 9s 2024-03-30T13:12:53.973 INFO:teuthology.orchestra.run.smithi043.stderr: 86700K .......... .......... .......... .......... .......... 51% 9.25M 9s 2024-03-30T13:12:53.979 INFO:teuthology.orchestra.run.smithi043.stderr: 86750K .......... .......... .......... .......... .......... 51% 8.04M 9s 2024-03-30T13:12:53.984 INFO:teuthology.orchestra.run.smithi043.stderr: 86800K .......... .......... .......... .......... .......... 51% 10.1M 9s 2024-03-30T13:12:53.990 INFO:teuthology.orchestra.run.smithi043.stderr: 86850K .......... .......... .......... .......... .......... 51% 9.67M 9s 2024-03-30T13:12:53.995 INFO:teuthology.orchestra.run.smithi043.stderr: 86900K .......... .......... .......... .......... .......... 51% 9.16M 9s 2024-03-30T13:12:54.001 INFO:teuthology.orchestra.run.smithi043.stderr: 86950K .......... .......... .......... .......... .......... 51% 7.79M 9s 2024-03-30T13:12:54.006 INFO:teuthology.orchestra.run.smithi043.stderr: 87000K .......... .......... .......... .......... .......... 51% 9.01M 9s 2024-03-30T13:12:54.012 INFO:teuthology.orchestra.run.smithi043.stderr: 87050K .......... .......... .......... .......... .......... 51% 8.76M 9s 2024-03-30T13:12:54.018 INFO:teuthology.orchestra.run.smithi043.stderr: 87100K .......... .......... .......... .......... .......... 51% 8.11M 9s 2024-03-30T13:12:54.025 INFO:teuthology.orchestra.run.smithi043.stderr: 87150K .......... .......... .......... .......... .......... 51% 7.18M 9s 2024-03-30T13:12:54.030 INFO:teuthology.orchestra.run.smithi043.stderr: 87200K .......... .......... .......... .......... .......... 51% 8.91M 9s 2024-03-30T13:12:54.036 INFO:teuthology.orchestra.run.smithi043.stderr: 87250K .......... .......... .......... .......... .......... 51% 9.04M 9s 2024-03-30T13:12:54.041 INFO:teuthology.orchestra.run.smithi043.stderr: 87300K .......... .......... .......... .......... .......... 51% 8.88M 9s 2024-03-30T13:12:54.048 INFO:teuthology.orchestra.run.smithi043.stderr: 87350K .......... .......... .......... .......... .......... 51% 7.49M 9s 2024-03-30T13:12:54.054 INFO:teuthology.orchestra.run.smithi043.stderr: 87400K .......... .......... .......... .......... .......... 51% 8.31M 9s 2024-03-30T13:12:54.060 INFO:teuthology.orchestra.run.smithi043.stderr: 87450K .......... .......... .......... .......... .......... 51% 8.28M 9s 2024-03-30T13:12:54.066 INFO:teuthology.orchestra.run.smithi043.stderr: 87500K .......... .......... .......... .......... .......... 51% 8.32M 9s 2024-03-30T13:12:54.072 INFO:teuthology.orchestra.run.smithi043.stderr: 87550K .......... .......... .......... .......... .......... 51% 6.94M 9s 2024-03-30T13:12:54.079 INFO:teuthology.orchestra.run.smithi043.stderr: 87600K .......... .......... .......... .......... .......... 51% 8.22M 9s 2024-03-30T13:12:54.084 INFO:teuthology.orchestra.run.smithi043.stderr: 87650K .......... .......... .......... .......... .......... 52% 8.12M 9s 2024-03-30T13:12:54.089 INFO:teuthology.orchestra.run.smithi043.stderr: 87700K .......... .......... .......... .......... .......... 52% 10.5M 9s 2024-03-30T13:12:54.096 INFO:teuthology.orchestra.run.smithi043.stderr: 87750K .......... .......... .......... .......... .......... 52% 7.82M 9s 2024-03-30T13:12:54.100 INFO:teuthology.orchestra.run.smithi043.stderr: 87800K .......... .......... .......... .......... .......... 52% 9.76M 9s 2024-03-30T13:12:54.106 INFO:teuthology.orchestra.run.smithi043.stderr: 87850K .......... .......... .......... .......... .......... 52% 8.82M 9s 2024-03-30T13:12:54.111 INFO:teuthology.orchestra.run.smithi043.stderr: 87900K .......... .......... .......... .......... .......... 52% 9.84M 9s 2024-03-30T13:12:54.116 INFO:teuthology.orchestra.run.smithi043.stderr: 87950K .......... .......... .......... .......... .......... 52% 8.80M 9s 2024-03-30T13:12:54.121 INFO:teuthology.orchestra.run.smithi043.stderr: 88000K .......... .......... .......... .......... .......... 52% 10.1M 9s 2024-03-30T13:12:54.127 INFO:teuthology.orchestra.run.smithi043.stderr: 88050K .......... .......... .......... .......... .......... 52% 8.47M 9s 2024-03-30T13:12:54.133 INFO:teuthology.orchestra.run.smithi043.stderr: 88100K .......... .......... .......... .......... .......... 52% 8.52M 9s 2024-03-30T13:12:54.140 INFO:teuthology.orchestra.run.smithi043.stderr: 88150K .......... .......... .......... .......... .......... 52% 7.03M 9s 2024-03-30T13:12:54.146 INFO:teuthology.orchestra.run.smithi043.stderr: 88200K .......... .......... .......... .......... .......... 52% 7.98M 9s 2024-03-30T13:12:54.153 INFO:teuthology.orchestra.run.smithi043.stderr: 88250K .......... .......... .......... .......... .......... 52% 7.23M 9s 2024-03-30T13:12:54.159 INFO:teuthology.orchestra.run.smithi043.stderr: 88300K .......... .......... .......... .......... .......... 52% 7.49M 9s 2024-03-30T13:12:54.167 INFO:teuthology.orchestra.run.smithi043.stderr: 88350K .......... .......... .......... .......... .......... 52% 6.64M 9s 2024-03-30T13:12:54.173 INFO:teuthology.orchestra.run.smithi043.stderr: 88400K .......... .......... .......... .......... .......... 52% 7.41M 9s 2024-03-30T13:12:54.180 INFO:teuthology.orchestra.run.smithi043.stderr: 88450K .......... .......... .......... .......... .......... 52% 7.30M 9s 2024-03-30T13:12:54.187 INFO:teuthology.orchestra.run.smithi043.stderr: 88500K .......... .......... .......... .......... .......... 52% 7.31M 9s 2024-03-30T13:12:54.193 INFO:teuthology.orchestra.run.smithi043.stderr: 88550K .......... .......... .......... .......... .......... 52% 6.95M 9s 2024-03-30T13:12:54.199 INFO:teuthology.orchestra.run.smithi043.stderr: 88600K .......... .......... .......... .......... .......... 52% 8.27M 9s 2024-03-30T13:12:54.205 INFO:teuthology.orchestra.run.smithi043.stderr: 88650K .......... .......... .......... .......... .......... 52% 8.40M 9s 2024-03-30T13:12:54.211 INFO:teuthology.orchestra.run.smithi043.stderr: 88700K .......... .......... .......... .......... .......... 52% 7.86M 9s 2024-03-30T13:12:54.218 INFO:teuthology.orchestra.run.smithi043.stderr: 88750K .......... .......... .......... .......... .......... 52% 7.31M 9s 2024-03-30T13:12:54.224 INFO:teuthology.orchestra.run.smithi043.stderr: 88800K .......... .......... .......... .......... .......... 52% 8.62M 9s 2024-03-30T13:12:54.230 INFO:teuthology.orchestra.run.smithi043.stderr: 88850K .......... .......... .......... .......... .......... 52% 8.29M 9s 2024-03-30T13:12:54.236 INFO:teuthology.orchestra.run.smithi043.stderr: 88900K .......... .......... .......... .......... .......... 52% 7.72M 9s 2024-03-30T13:12:54.243 INFO:teuthology.orchestra.run.smithi043.stderr: 88950K .......... .......... .......... .......... .......... 52% 6.91M 9s 2024-03-30T13:12:54.249 INFO:teuthology.orchestra.run.smithi043.stderr: 89000K .......... .......... .......... .......... .......... 52% 8.04M 9s 2024-03-30T13:12:54.255 INFO:teuthology.orchestra.run.smithi043.stderr: 89050K .......... .......... .......... .......... .......... 52% 8.15M 9s 2024-03-30T13:12:54.262 INFO:teuthology.orchestra.run.smithi043.stderr: 89100K .......... .......... .......... .......... .......... 52% 7.68M 9s 2024-03-30T13:12:54.269 INFO:teuthology.orchestra.run.smithi043.stderr: 89150K .......... .......... .......... .......... .......... 52% 6.69M 9s 2024-03-30T13:12:54.275 INFO:teuthology.orchestra.run.smithi043.stderr: 89200K .......... .......... .......... .......... .......... 52% 8.02M 9s 2024-03-30T13:12:54.282 INFO:teuthology.orchestra.run.smithi043.stderr: 89250K .......... .......... .......... .......... .......... 52% 7.39M 9s 2024-03-30T13:12:54.289 INFO:teuthology.orchestra.run.smithi043.stderr: 89300K .......... .......... .......... .......... .......... 53% 7.03M 9s 2024-03-30T13:12:54.296 INFO:teuthology.orchestra.run.smithi043.stderr: 89350K .......... .......... .......... .......... .......... 53% 6.30M 9s 2024-03-30T13:12:54.302 INFO:teuthology.orchestra.run.smithi043.stderr: 89400K .......... .......... .......... .......... .......... 53% 7.98M 9s 2024-03-30T13:12:54.308 INFO:teuthology.orchestra.run.smithi043.stderr: 89450K .......... .......... .......... .......... .......... 53% 7.99M 9s 2024-03-30T13:12:54.314 INFO:teuthology.orchestra.run.smithi043.stderr: 89500K .......... .......... .......... .......... .......... 53% 8.44M 9s 2024-03-30T13:12:54.320 INFO:teuthology.orchestra.run.smithi043.stderr: 89550K .......... .......... .......... .......... .......... 53% 8.11M 9s 2024-03-30T13:12:54.326 INFO:teuthology.orchestra.run.smithi043.stderr: 89600K .......... .......... .......... .......... .......... 53% 9.47M 9s 2024-03-30T13:12:54.330 INFO:teuthology.orchestra.run.smithi043.stderr: 89650K .......... .......... .......... .......... .......... 53% 10.3M 9s 2024-03-30T13:12:54.335 INFO:teuthology.orchestra.run.smithi043.stderr: 89700K .......... .......... .......... .......... .......... 53% 9.48M 9s 2024-03-30T13:12:54.343 INFO:teuthology.orchestra.run.smithi043.stderr: 89750K .......... .......... .......... .......... .......... 53% 6.67M 9s 2024-03-30T13:12:54.349 INFO:teuthology.orchestra.run.smithi043.stderr: 89800K .......... .......... .......... .......... .......... 53% 7.48M 9s 2024-03-30T13:12:54.356 INFO:teuthology.orchestra.run.smithi043.stderr: 89850K .......... .......... .......... .......... .......... 53% 7.15M 9s 2024-03-30T13:12:54.363 INFO:teuthology.orchestra.run.smithi043.stderr: 89900K .......... .......... .......... .......... .......... 53% 6.79M 9s 2024-03-30T13:12:54.372 INFO:teuthology.orchestra.run.smithi043.stderr: 89950K .......... .......... .......... .......... .......... 53% 5.83M 9s 2024-03-30T13:12:54.378 INFO:teuthology.orchestra.run.smithi043.stderr: 90000K .......... .......... .......... .......... .......... 53% 7.05M 9s 2024-03-30T13:12:54.384 INFO:teuthology.orchestra.run.smithi043.stderr: 90050K .......... .......... .......... .......... .......... 53% 8.02M 9s 2024-03-30T13:12:54.390 INFO:teuthology.orchestra.run.smithi043.stderr: 90100K .......... .......... .......... .......... .......... 53% 8.17M 9s 2024-03-30T13:12:54.395 INFO:teuthology.orchestra.run.smithi043.stderr: 90150K .......... .......... .......... .......... .......... 53% 9.56M 9s 2024-03-30T13:12:54.400 INFO:teuthology.orchestra.run.smithi043.stderr: 90200K .......... .......... .......... .......... .......... 53% 11.3M 9s 2024-03-30T13:12:54.404 INFO:teuthology.orchestra.run.smithi043.stderr: 90250K .......... .......... .......... .......... .......... 53% 11.0M 9s 2024-03-30T13:12:54.408 INFO:teuthology.orchestra.run.smithi043.stderr: 90300K .......... .......... .......... .......... .......... 53% 11.7M 9s 2024-03-30T13:12:54.413 INFO:teuthology.orchestra.run.smithi043.stderr: 90350K .......... .......... .......... .......... .......... 53% 10.3M 9s 2024-03-30T13:12:54.417 INFO:teuthology.orchestra.run.smithi043.stderr: 90400K .......... .......... .......... .......... .......... 53% 12.3M 9s 2024-03-30T13:12:54.421 INFO:teuthology.orchestra.run.smithi043.stderr: 90450K .......... .......... .......... .......... .......... 53% 11.6M 9s 2024-03-30T13:12:54.425 INFO:teuthology.orchestra.run.smithi043.stderr: 90500K .......... .......... .......... .......... .......... 53% 12.0M 9s 2024-03-30T13:12:54.430 INFO:teuthology.orchestra.run.smithi043.stderr: 90550K .......... .......... .......... .......... .......... 53% 9.93M 9s 2024-03-30T13:12:54.434 INFO:teuthology.orchestra.run.smithi043.stderr: 90600K .......... .......... .......... .......... .......... 53% 11.4M 9s 2024-03-30T13:12:54.439 INFO:teuthology.orchestra.run.smithi043.stderr: 90650K .......... .......... .......... .......... .......... 53% 11.2M 9s 2024-03-30T13:12:54.443 INFO:teuthology.orchestra.run.smithi043.stderr: 90700K .......... .......... .......... .......... .......... 53% 12.4M 9s 2024-03-30T13:12:54.448 INFO:teuthology.orchestra.run.smithi043.stderr: 90750K .......... .......... .......... .......... .......... 53% 9.45M 9s 2024-03-30T13:12:54.453 INFO:teuthology.orchestra.run.smithi043.stderr: 90800K .......... .......... .......... .......... .......... 53% 10.8M 9s 2024-03-30T13:12:54.457 INFO:teuthology.orchestra.run.smithi043.stderr: 90850K .......... .......... .......... .......... .......... 53% 9.58M 9s 2024-03-30T13:12:54.462 INFO:teuthology.orchestra.run.smithi043.stderr: 90900K .......... .......... .......... .......... .......... 53% 9.69M 9s 2024-03-30T13:12:54.468 INFO:teuthology.orchestra.run.smithi043.stderr: 90950K .......... .......... .......... .......... .......... 53% 8.61M 9s 2024-03-30T13:12:54.474 INFO:teuthology.orchestra.run.smithi043.stderr: 91000K .......... .......... .......... .......... .......... 54% 9.71M 9s 2024-03-30T13:12:54.478 INFO:teuthology.orchestra.run.smithi043.stderr: 91050K .......... .......... .......... .......... .......... 54% 9.76M 9s 2024-03-30T13:12:54.483 INFO:teuthology.orchestra.run.smithi043.stderr: 91100K .......... .......... .......... .......... .......... 54% 9.82M 9s 2024-03-30T13:12:54.489 INFO:teuthology.orchestra.run.smithi043.stderr: 91150K .......... .......... .......... .......... .......... 54% 8.32M 9s 2024-03-30T13:12:54.494 INFO:teuthology.orchestra.run.smithi043.stderr: 91200K .......... .......... .......... .......... .......... 54% 11.1M 9s 2024-03-30T13:12:54.499 INFO:teuthology.orchestra.run.smithi043.stderr: 91250K .......... .......... .......... .......... .......... 54% 9.64M 9s 2024-03-30T13:12:54.503 INFO:teuthology.orchestra.run.smithi043.stderr: 91300K .......... .......... .......... .......... .......... 54% 10.3M 9s 2024-03-30T13:12:54.509 INFO:teuthology.orchestra.run.smithi043.stderr: 91350K .......... .......... .......... .......... .......... 54% 8.74M 9s 2024-03-30T13:12:54.514 INFO:teuthology.orchestra.run.smithi043.stderr: 91400K .......... .......... .......... .......... .......... 54% 9.29M 9s 2024-03-30T13:12:54.520 INFO:teuthology.orchestra.run.smithi043.stderr: 91450K .......... .......... .......... .......... .......... 54% 8.20M 9s 2024-03-30T13:12:54.526 INFO:teuthology.orchestra.run.smithi043.stderr: 91500K .......... .......... .......... .......... .......... 54% 8.67M 9s 2024-03-30T13:12:54.533 INFO:teuthology.orchestra.run.smithi043.stderr: 91550K .......... .......... .......... .......... .......... 54% 7.83M 9s 2024-03-30T13:12:54.538 INFO:teuthology.orchestra.run.smithi043.stderr: 91600K .......... .......... .......... .......... .......... 54% 8.76M 9s 2024-03-30T13:12:54.544 INFO:teuthology.orchestra.run.smithi043.stderr: 91650K .......... .......... .......... .......... .......... 54% 8.76M 9s 2024-03-30T13:12:54.548 INFO:teuthology.orchestra.run.smithi043.stderr: 91700K .......... .......... .......... .......... .......... 54% 9.23M 9s 2024-03-30T13:12:54.555 INFO:teuthology.orchestra.run.smithi043.stderr: 91750K .......... .......... .......... .......... .......... 54% 8.20M 9s 2024-03-30T13:12:54.560 INFO:teuthology.orchestra.run.smithi043.stderr: 91800K .......... .......... .......... .......... .......... 54% 9.27M 9s 2024-03-30T13:12:54.565 INFO:teuthology.orchestra.run.smithi043.stderr: 91850K .......... .......... .......... .......... .......... 54% 9.42M 9s 2024-03-30T13:12:54.570 INFO:teuthology.orchestra.run.smithi043.stderr: 91900K .......... .......... .......... .......... .......... 54% 9.34M 9s 2024-03-30T13:12:54.577 INFO:teuthology.orchestra.run.smithi043.stderr: 91950K .......... .......... .......... .......... .......... 54% 7.49M 9s 2024-03-30T13:12:54.582 INFO:teuthology.orchestra.run.smithi043.stderr: 92000K .......... .......... .......... .......... .......... 54% 9.75M 9s 2024-03-30T13:12:54.588 INFO:teuthology.orchestra.run.smithi043.stderr: 92050K .......... .......... .......... .......... .......... 54% 8.03M 9s 2024-03-30T13:12:54.594 INFO:teuthology.orchestra.run.smithi043.stderr: 92100K .......... .......... .......... .......... .......... 54% 8.35M 9s 2024-03-30T13:12:54.600 INFO:teuthology.orchestra.run.smithi043.stderr: 92150K .......... .......... .......... .......... .......... 54% 7.67M 9s 2024-03-30T13:12:54.606 INFO:teuthology.orchestra.run.smithi043.stderr: 92200K .......... .......... .......... .......... .......... 54% 8.39M 9s 2024-03-30T13:12:54.613 INFO:teuthology.orchestra.run.smithi043.stderr: 92250K .......... .......... .......... .......... .......... 54% 7.73M 9s 2024-03-30T13:12:54.618 INFO:teuthology.orchestra.run.smithi043.stderr: 92300K .......... .......... .......... .......... .......... 54% 7.98M 9s 2024-03-30T13:12:54.625 INFO:teuthology.orchestra.run.smithi043.stderr: 92350K .......... .......... .......... .......... .......... 54% 7.24M 9s 2024-03-30T13:12:54.631 INFO:teuthology.orchestra.run.smithi043.stderr: 92400K .......... .......... .......... .......... .......... 54% 8.14M 9s 2024-03-30T13:12:54.637 INFO:teuthology.orchestra.run.smithi043.stderr: 92450K .......... .......... .......... .......... .......... 54% 8.19M 9s 2024-03-30T13:12:54.643 INFO:teuthology.orchestra.run.smithi043.stderr: 92500K .......... .......... .......... .......... .......... 54% 8.09M 9s 2024-03-30T13:12:54.650 INFO:teuthology.orchestra.run.smithi043.stderr: 92550K .......... .......... .......... .......... .......... 54% 7.12M 9s 2024-03-30T13:12:54.656 INFO:teuthology.orchestra.run.smithi043.stderr: 92600K .......... .......... .......... .......... .......... 54% 8.15M 9s 2024-03-30T13:12:54.662 INFO:teuthology.orchestra.run.smithi043.stderr: 92650K .......... .......... .......... .......... .......... 54% 7.70M 9s 2024-03-30T13:12:54.669 INFO:teuthology.orchestra.run.smithi043.stderr: 92700K .......... .......... .......... .......... .......... 55% 7.46M 9s 2024-03-30T13:12:54.676 INFO:teuthology.orchestra.run.smithi043.stderr: 92750K .......... .......... .......... .......... .......... 55% 6.40M 9s 2024-03-30T13:12:54.683 INFO:teuthology.orchestra.run.smithi043.stderr: 92800K .......... .......... .......... .......... .......... 55% 7.37M 9s 2024-03-30T13:12:54.689 INFO:teuthology.orchestra.run.smithi043.stderr: 92850K .......... .......... .......... .......... .......... 55% 7.88M 9s 2024-03-30T13:12:54.695 INFO:teuthology.orchestra.run.smithi043.stderr: 92900K .......... .......... .......... .......... .......... 55% 7.98M 9s 2024-03-30T13:12:54.703 INFO:teuthology.orchestra.run.smithi043.stderr: 92950K .......... .......... .......... .......... .......... 55% 6.50M 9s 2024-03-30T13:12:54.709 INFO:teuthology.orchestra.run.smithi043.stderr: 93000K .......... .......... .......... .......... .......... 55% 7.71M 9s 2024-03-30T13:12:54.716 INFO:teuthology.orchestra.run.smithi043.stderr: 93050K .......... .......... .......... .......... .......... 55% 7.57M 9s 2024-03-30T13:12:54.723 INFO:teuthology.orchestra.run.smithi043.stderr: 93100K .......... .......... .......... .......... .......... 55% 7.05M 9s 2024-03-30T13:12:54.731 INFO:teuthology.orchestra.run.smithi043.stderr: 93150K .......... .......... .......... .......... .......... 55% 5.64M 9s 2024-03-30T13:12:54.738 INFO:teuthology.orchestra.run.smithi043.stderr: 93200K .......... .......... .......... .......... .......... 55% 7.51M 9s 2024-03-30T13:12:54.745 INFO:teuthology.orchestra.run.smithi043.stderr: 93250K .......... .......... .......... .......... .......... 55% 7.08M 9s 2024-03-30T13:12:54.751 INFO:teuthology.orchestra.run.smithi043.stderr: 93300K .......... .......... .......... .......... .......... 55% 7.27M 8s 2024-03-30T13:12:54.760 INFO:teuthology.orchestra.run.smithi043.stderr: 93350K .......... .......... .......... .......... .......... 55% 5.76M 8s 2024-03-30T13:12:54.767 INFO:teuthology.orchestra.run.smithi043.stderr: 93400K .......... .......... .......... .......... .......... 55% 7.35M 8s 2024-03-30T13:12:54.773 INFO:teuthology.orchestra.run.smithi043.stderr: 93450K .......... .......... .......... .......... .......... 55% 7.12M 8s 2024-03-30T13:12:54.780 INFO:teuthology.orchestra.run.smithi043.stderr: 93500K .......... .......... .......... .......... .......... 55% 7.75M 8s 2024-03-30T13:12:54.787 INFO:teuthology.orchestra.run.smithi043.stderr: 93550K .......... .......... .......... .......... .......... 55% 6.86M 8s 2024-03-30T13:12:54.794 INFO:teuthology.orchestra.run.smithi043.stderr: 93600K .......... .......... .......... .......... .......... 55% 7.21M 8s 2024-03-30T13:12:54.800 INFO:teuthology.orchestra.run.smithi043.stderr: 93650K .......... .......... .......... .......... .......... 55% 7.08M 8s 2024-03-30T13:12:54.808 INFO:teuthology.orchestra.run.smithi043.stderr: 93700K .......... .......... .......... .......... .......... 55% 7.00M 8s 2024-03-30T13:12:54.816 INFO:teuthology.orchestra.run.smithi043.stderr: 93750K .......... .......... .......... .......... .......... 55% 5.91M 8s 2024-03-30T13:12:54.823 INFO:teuthology.orchestra.run.smithi043.stderr: 93800K .......... .......... .......... .......... .......... 55% 6.87M 8s 2024-03-30T13:12:54.830 INFO:teuthology.orchestra.run.smithi043.stderr: 93850K .......... .......... .......... .......... .......... 55% 6.87M 8s 2024-03-30T13:12:54.837 INFO:teuthology.orchestra.run.smithi043.stderr: 93900K .......... .......... .......... .......... .......... 55% 6.92M 8s 2024-03-30T13:12:54.845 INFO:teuthology.orchestra.run.smithi043.stderr: 93950K .......... .......... .......... .......... .......... 55% 6.09M 8s 2024-03-30T13:12:54.852 INFO:teuthology.orchestra.run.smithi043.stderr: 94000K .......... .......... .......... .......... .......... 55% 7.20M 8s 2024-03-30T13:12:54.859 INFO:teuthology.orchestra.run.smithi043.stderr: 94050K .......... .......... .......... .......... .......... 55% 7.21M 8s 2024-03-30T13:12:54.866 INFO:teuthology.orchestra.run.smithi043.stderr: 94100K .......... .......... .......... .......... .......... 55% 7.07M 8s 2024-03-30T13:12:54.874 INFO:teuthology.orchestra.run.smithi043.stderr: 94150K .......... .......... .......... .......... .......... 55% 6.08M 8s 2024-03-30T13:12:54.879 INFO:teuthology.orchestra.run.smithi043.stderr: 94200K .......... .......... .......... .......... .......... 55% 8.98M 8s 2024-03-30T13:12:54.882 INFO:teuthology.orchestra.run.smithi043.stderr: 94250K .......... .......... .......... .......... .......... 55% 12.6M 8s 2024-03-30T13:12:54.888 INFO:teuthology.orchestra.run.smithi043.stderr: 94300K .......... .......... .......... .......... .......... 55% 8.83M 8s 2024-03-30T13:12:54.894 INFO:teuthology.orchestra.run.smithi043.stderr: 94350K .......... .......... .......... .......... .......... 56% 7.73M 8s 2024-03-30T13:12:54.899 INFO:teuthology.orchestra.run.smithi043.stderr: 94400K .......... .......... .......... .......... .......... 56% 11.8M 8s 2024-03-30T13:12:54.903 INFO:teuthology.orchestra.run.smithi043.stderr: 94450K .......... .......... .......... .......... .......... 56% 10.1M 8s 2024-03-30T13:12:54.908 INFO:teuthology.orchestra.run.smithi043.stderr: 94500K .......... .......... .......... .......... .......... 56% 11.6M 8s 2024-03-30T13:12:54.913 INFO:teuthology.orchestra.run.smithi043.stderr: 94550K .......... .......... .......... .......... .......... 56% 9.21M 8s 2024-03-30T13:12:54.917 INFO:teuthology.orchestra.run.smithi043.stderr: 94600K .......... .......... .......... .......... .......... 56% 10.9M 8s 2024-03-30T13:12:54.922 INFO:teuthology.orchestra.run.smithi043.stderr: 94650K .......... .......... .......... .......... .......... 56% 10.1M 8s 2024-03-30T13:12:54.927 INFO:teuthology.orchestra.run.smithi043.stderr: 94700K .......... .......... .......... .......... .......... 56% 10.3M 8s 2024-03-30T13:12:54.933 INFO:teuthology.orchestra.run.smithi043.stderr: 94750K .......... .......... .......... .......... .......... 56% 9.11M 8s 2024-03-30T13:12:54.937 INFO:teuthology.orchestra.run.smithi043.stderr: 94800K .......... .......... .......... .......... .......... 56% 9.81M 8s 2024-03-30T13:12:54.942 INFO:teuthology.orchestra.run.smithi043.stderr: 94850K .......... .......... .......... .......... .......... 56% 9.86M 8s 2024-03-30T13:12:54.948 INFO:teuthology.orchestra.run.smithi043.stderr: 94900K .......... .......... .......... .......... .......... 56% 9.46M 8s 2024-03-30T13:12:54.953 INFO:teuthology.orchestra.run.smithi043.stderr: 94950K .......... .......... .......... .......... .......... 56% 9.14M 8s 2024-03-30T13:12:54.957 INFO:teuthology.orchestra.run.smithi043.stderr: 95000K .......... .......... .......... .......... .......... 56% 10.7M 8s 2024-03-30T13:12:54.962 INFO:teuthology.orchestra.run.smithi043.stderr: 95050K .......... .......... .......... .......... .......... 56% 10.5M 8s 2024-03-30T13:12:54.968 INFO:teuthology.orchestra.run.smithi043.stderr: 95100K .......... .......... .......... .......... .......... 56% 8.85M 8s 2024-03-30T13:12:54.973 INFO:teuthology.orchestra.run.smithi043.stderr: 95150K .......... .......... .......... .......... .......... 56% 8.41M 8s 2024-03-30T13:12:54.978 INFO:teuthology.orchestra.run.smithi043.stderr: 95200K .......... .......... .......... .......... .......... 56% 9.42M 8s 2024-03-30T13:12:54.983 INFO:teuthology.orchestra.run.smithi043.stderr: 95250K .......... .......... .......... .......... .......... 56% 10.6M 8s 2024-03-30T13:12:54.988 INFO:teuthology.orchestra.run.smithi043.stderr: 95300K .......... .......... .......... .......... .......... 56% 10.1M 8s 2024-03-30T13:12:54.993 INFO:teuthology.orchestra.run.smithi043.stderr: 95350K .......... .......... .......... .......... .......... 56% 8.95M 8s 2024-03-30T13:12:54.999 INFO:teuthology.orchestra.run.smithi043.stderr: 95400K .......... .......... .......... .......... .......... 56% 9.32M 8s 2024-03-30T13:12:55.004 INFO:teuthology.orchestra.run.smithi043.stderr: 95450K .......... .......... .......... .......... .......... 56% 9.85M 8s 2024-03-30T13:12:55.009 INFO:teuthology.orchestra.run.smithi043.stderr: 95500K .......... .......... .......... .......... .......... 56% 9.29M 8s 2024-03-30T13:12:55.015 INFO:teuthology.orchestra.run.smithi043.stderr: 95550K .......... .......... .......... .......... .......... 56% 8.33M 8s 2024-03-30T13:12:55.019 INFO:teuthology.orchestra.run.smithi043.stderr: 95600K .......... .......... .......... .......... .......... 56% 10.1M 8s 2024-03-30T13:12:55.024 INFO:teuthology.orchestra.run.smithi043.stderr: 95650K .......... .......... .......... .......... .......... 56% 10.6M 8s 2024-03-30T13:12:55.030 INFO:teuthology.orchestra.run.smithi043.stderr: 95700K .......... .......... .......... .......... .......... 56% 9.01M 8s 2024-03-30T13:12:55.036 INFO:teuthology.orchestra.run.smithi043.stderr: 95750K .......... .......... .......... .......... .......... 56% 7.93M 8s 2024-03-30T13:12:55.041 INFO:teuthology.orchestra.run.smithi043.stderr: 95800K .......... .......... .......... .......... .......... 56% 9.49M 8s 2024-03-30T13:12:55.046 INFO:teuthology.orchestra.run.smithi043.stderr: 95850K .......... .......... .......... .......... .......... 56% 9.94M 8s 2024-03-30T13:12:55.051 INFO:teuthology.orchestra.run.smithi043.stderr: 95900K .......... .......... .......... .......... .......... 56% 8.92M 8s 2024-03-30T13:12:55.058 INFO:teuthology.orchestra.run.smithi043.stderr: 95950K .......... .......... .......... .......... .......... 56% 7.72M 8s 2024-03-30T13:12:55.064 INFO:teuthology.orchestra.run.smithi043.stderr: 96000K .......... .......... .......... .......... .......... 56% 7.82M 8s 2024-03-30T13:12:55.070 INFO:teuthology.orchestra.run.smithi043.stderr: 96050K .......... .......... .......... .......... .......... 57% 8.04M 8s 2024-03-30T13:12:55.076 INFO:teuthology.orchestra.run.smithi043.stderr: 96100K .......... .......... .......... .......... .......... 57% 8.42M 8s 2024-03-30T13:12:55.083 INFO:teuthology.orchestra.run.smithi043.stderr: 96150K .......... .......... .......... .......... .......... 57% 6.97M 8s 2024-03-30T13:12:55.089 INFO:teuthology.orchestra.run.smithi043.stderr: 96200K .......... .......... .......... .......... .......... 57% 7.82M 8s 2024-03-30T13:12:55.095 INFO:teuthology.orchestra.run.smithi043.stderr: 96250K .......... .......... .......... .......... .......... 57% 8.11M 8s 2024-03-30T13:12:55.101 INFO:teuthology.orchestra.run.smithi043.stderr: 96300K .......... .......... .......... .......... .......... 57% 7.87M 8s 2024-03-30T13:12:55.108 INFO:teuthology.orchestra.run.smithi043.stderr: 96350K .......... .......... .......... .......... .......... 57% 7.29M 8s 2024-03-30T13:12:55.114 INFO:teuthology.orchestra.run.smithi043.stderr: 96400K .......... .......... .......... .......... .......... 57% 8.63M 8s 2024-03-30T13:12:55.119 INFO:teuthology.orchestra.run.smithi043.stderr: 96450K .......... .......... .......... .......... .......... 57% 9.15M 8s 2024-03-30T13:12:55.125 INFO:teuthology.orchestra.run.smithi043.stderr: 96500K .......... .......... .......... .......... .......... 57% 8.89M 8s 2024-03-30T13:12:55.132 INFO:teuthology.orchestra.run.smithi043.stderr: 96550K .......... .......... .......... .......... .......... 57% 6.67M 8s 2024-03-30T13:12:55.137 INFO:teuthology.orchestra.run.smithi043.stderr: 96600K .......... .......... .......... .......... .......... 57% 8.80M 8s 2024-03-30T13:12:55.143 INFO:teuthology.orchestra.run.smithi043.stderr: 96650K .......... .......... .......... .......... .......... 57% 9.35M 8s 2024-03-30T13:12:55.148 INFO:teuthology.orchestra.run.smithi043.stderr: 96700K .......... .......... .......... .......... .......... 57% 9.20M 8s 2024-03-30T13:12:55.154 INFO:teuthology.orchestra.run.smithi043.stderr: 96750K .......... .......... .......... .......... .......... 57% 7.71M 8s 2024-03-30T13:12:55.160 INFO:teuthology.orchestra.run.smithi043.stderr: 96800K .......... .......... .......... .......... .......... 57% 9.03M 8s 2024-03-30T13:12:55.165 INFO:teuthology.orchestra.run.smithi043.stderr: 96850K .......... .......... .......... .......... .......... 57% 8.99M 8s 2024-03-30T13:12:55.170 INFO:teuthology.orchestra.run.smithi043.stderr: 96900K .......... .......... .......... .......... .......... 57% 8.96M 8s 2024-03-30T13:12:55.177 INFO:teuthology.orchestra.run.smithi043.stderr: 96950K .......... .......... .......... .......... .......... 57% 7.89M 8s 2024-03-30T13:12:55.186 INFO:teuthology.orchestra.run.smithi043.stderr: 97000K .......... .......... .......... .......... .......... 57% 8.96M 8s 2024-03-30T13:12:55.189 INFO:teuthology.orchestra.run.smithi043.stderr: 97050K .......... .......... .......... .......... .......... 57% 7.56M 8s 2024-03-30T13:12:55.194 INFO:teuthology.orchestra.run.smithi043.stderr: 97100K .......... .......... .......... .......... .......... 57% 8.49M 8s 2024-03-30T13:12:55.201 INFO:teuthology.orchestra.run.smithi043.stderr: 97150K .......... .......... .......... .......... .......... 57% 7.11M 8s 2024-03-30T13:12:55.207 INFO:teuthology.orchestra.run.smithi043.stderr: 97200K .......... .......... .......... .......... .......... 57% 8.09M 8s 2024-03-30T13:12:55.213 INFO:teuthology.orchestra.run.smithi043.stderr: 97250K .......... .......... .......... .......... .......... 57% 7.92M 8s 2024-03-30T13:12:55.220 INFO:teuthology.orchestra.run.smithi043.stderr: 97300K .......... .......... .......... .......... .......... 57% 7.72M 8s 2024-03-30T13:12:55.228 INFO:teuthology.orchestra.run.smithi043.stderr: 97350K .......... .......... .......... .......... .......... 57% 6.37M 8s 2024-03-30T13:12:55.234 INFO:teuthology.orchestra.run.smithi043.stderr: 97400K .......... .......... .......... .......... .......... 57% 7.34M 8s 2024-03-30T13:12:55.240 INFO:teuthology.orchestra.run.smithi043.stderr: 97450K .......... .......... .......... .......... .......... 57% 8.21M 8s 2024-03-30T13:12:55.246 INFO:teuthology.orchestra.run.smithi043.stderr: 97500K .......... .......... .......... .......... .......... 57% 7.78M 8s 2024-03-30T13:12:55.253 INFO:teuthology.orchestra.run.smithi043.stderr: 97550K .......... .......... .......... .......... .......... 57% 7.29M 8s 2024-03-30T13:12:55.259 INFO:teuthology.orchestra.run.smithi043.stderr: 97600K .......... .......... .......... .......... .......... 57% 7.90M 8s 2024-03-30T13:12:55.265 INFO:teuthology.orchestra.run.smithi043.stderr: 97650K .......... .......... .......... .......... .......... 57% 7.91M 8s 2024-03-30T13:12:55.272 INFO:teuthology.orchestra.run.smithi043.stderr: 97700K .......... .......... .......... .......... .......... 57% 7.95M 8s 2024-03-30T13:12:55.280 INFO:teuthology.orchestra.run.smithi043.stderr: 97750K .......... .......... .......... .......... .......... 58% 6.10M 8s 2024-03-30T13:12:55.286 INFO:teuthology.orchestra.run.smithi043.stderr: 97800K .......... .......... .......... .......... .......... 58% 7.49M 8s 2024-03-30T13:12:55.293 INFO:teuthology.orchestra.run.smithi043.stderr: 97850K .......... .......... .......... .......... .......... 58% 6.91M 8s 2024-03-30T13:12:55.300 INFO:teuthology.orchestra.run.smithi043.stderr: 97900K .......... .......... .......... .......... .......... 58% 6.55M 8s 2024-03-30T13:12:55.308 INFO:teuthology.orchestra.run.smithi043.stderr: 97950K .......... .......... .......... .......... .......... 58% 6.23M 8s 2024-03-30T13:12:55.315 INFO:teuthology.orchestra.run.smithi043.stderr: 98000K .......... .......... .......... .......... .......... 58% 7.53M 8s 2024-03-30T13:12:55.322 INFO:teuthology.orchestra.run.smithi043.stderr: 98050K .......... .......... .......... .......... .......... 58% 7.14M 8s 2024-03-30T13:12:55.329 INFO:teuthology.orchestra.run.smithi043.stderr: 98100K .......... .......... .......... .......... .......... 58% 6.99M 8s 2024-03-30T13:12:55.337 INFO:teuthology.orchestra.run.smithi043.stderr: 98150K .......... .......... .......... .......... .......... 58% 5.95M 8s 2024-03-30T13:12:55.344 INFO:teuthology.orchestra.run.smithi043.stderr: 98200K .......... .......... .......... .......... .......... 58% 7.10M 8s 2024-03-30T13:12:55.351 INFO:teuthology.orchestra.run.smithi043.stderr: 98250K .......... .......... .......... .......... .......... 58% 6.50M 8s 2024-03-30T13:12:55.358 INFO:teuthology.orchestra.run.smithi043.stderr: 98300K .......... .......... .......... .......... .......... 58% 7.61M 8s 2024-03-30T13:12:55.364 INFO:teuthology.orchestra.run.smithi043.stderr: 98350K .......... .......... .......... .......... .......... 58% 7.66M 8s 2024-03-30T13:12:55.370 INFO:teuthology.orchestra.run.smithi043.stderr: 98400K .......... .......... .......... .......... .......... 58% 8.46M 8s 2024-03-30T13:12:55.374 INFO:teuthology.orchestra.run.smithi043.stderr: 98450K .......... .......... .......... .......... .......... 58% 10.8M 8s 2024-03-30T13:12:55.379 INFO:teuthology.orchestra.run.smithi043.stderr: 98500K .......... .......... .......... .......... .......... 58% 10.6M 8s 2024-03-30T13:12:55.385 INFO:teuthology.orchestra.run.smithi043.stderr: 98550K .......... .......... .......... .......... .......... 58% 8.82M 8s 2024-03-30T13:12:55.389 INFO:teuthology.orchestra.run.smithi043.stderr: 98600K .......... .......... .......... .......... .......... 58% 9.63M 8s 2024-03-30T13:12:55.394 INFO:teuthology.orchestra.run.smithi043.stderr: 98650K .......... .......... .......... .......... .......... 58% 9.97M 8s 2024-03-30T13:12:55.399 INFO:teuthology.orchestra.run.smithi043.stderr: 98700K .......... .......... .......... .......... .......... 58% 10.3M 8s 2024-03-30T13:12:55.405 INFO:teuthology.orchestra.run.smithi043.stderr: 98750K .......... .......... .......... .......... .......... 58% 8.83M 8s 2024-03-30T13:12:55.409 INFO:teuthology.orchestra.run.smithi043.stderr: 98800K .......... .......... .......... .......... .......... 58% 10.0M 8s 2024-03-30T13:12:55.414 INFO:teuthology.orchestra.run.smithi043.stderr: 98850K .......... .......... .......... .......... .......... 58% 10.3M 8s 2024-03-30T13:12:55.419 INFO:teuthology.orchestra.run.smithi043.stderr: 98900K .......... .......... .......... .......... .......... 58% 10.2M 8s 2024-03-30T13:12:55.425 INFO:teuthology.orchestra.run.smithi043.stderr: 98950K .......... .......... .......... .......... .......... 58% 8.04M 8s 2024-03-30T13:12:55.431 INFO:teuthology.orchestra.run.smithi043.stderr: 99000K .......... .......... .......... .......... .......... 58% 8.34M 8s 2024-03-30T13:12:55.436 INFO:teuthology.orchestra.run.smithi043.stderr: 99050K .......... .......... .......... .......... .......... 58% 9.06M 8s 2024-03-30T13:12:55.441 INFO:teuthology.orchestra.run.smithi043.stderr: 99100K .......... .......... .......... .......... .......... 58% 9.65M 8s 2024-03-30T13:12:55.446 INFO:teuthology.orchestra.run.smithi043.stderr: 99150K .......... .......... .......... .......... .......... 58% 9.44M 8s 2024-03-30T13:12:55.451 INFO:teuthology.orchestra.run.smithi043.stderr: 99200K .......... .......... .......... .......... .......... 58% 10.8M 8s 2024-03-30T13:12:55.455 INFO:teuthology.orchestra.run.smithi043.stderr: 99250K .......... .......... .......... .......... .......... 58% 10.9M 8s 2024-03-30T13:12:55.460 INFO:teuthology.orchestra.run.smithi043.stderr: 99300K .......... .......... .......... .......... .......... 58% 10.5M 8s 2024-03-30T13:12:55.466 INFO:teuthology.orchestra.run.smithi043.stderr: 99350K .......... .......... .......... .......... .......... 58% 8.63M 8s 2024-03-30T13:12:55.471 INFO:teuthology.orchestra.run.smithi043.stderr: 99400K .......... .......... .......... .......... .......... 58% 9.04M 8s 2024-03-30T13:12:55.476 INFO:teuthology.orchestra.run.smithi043.stderr: 99450K .......... .......... .......... .......... .......... 59% 9.23M 8s 2024-03-30T13:12:55.482 INFO:teuthology.orchestra.run.smithi043.stderr: 99500K .......... .......... .......... .......... .......... 59% 9.62M 8s 2024-03-30T13:12:55.488 INFO:teuthology.orchestra.run.smithi043.stderr: 99550K .......... .......... .......... .......... .......... 59% 7.78M 8s 2024-03-30T13:12:55.493 INFO:teuthology.orchestra.run.smithi043.stderr: 99600K .......... .......... .......... .......... .......... 59% 9.44M 8s 2024-03-30T13:12:55.499 INFO:teuthology.orchestra.run.smithi043.stderr: 99650K .......... .......... .......... .......... .......... 59% 8.49M 8s 2024-03-30T13:12:55.504 INFO:teuthology.orchestra.run.smithi043.stderr: 99700K .......... .......... .......... .......... .......... 59% 9.21M 8s 2024-03-30T13:12:55.510 INFO:teuthology.orchestra.run.smithi043.stderr: 99750K .......... .......... .......... .......... .......... 59% 8.26M 8s 2024-03-30T13:12:55.515 INFO:teuthology.orchestra.run.smithi043.stderr: 99800K .......... .......... .......... .......... .......... 59% 9.11M 8s 2024-03-30T13:12:55.520 INFO:teuthology.orchestra.run.smithi043.stderr: 99850K .......... .......... .......... .......... .......... 59% 9.59M 8s 2024-03-30T13:12:55.526 INFO:teuthology.orchestra.run.smithi043.stderr: 99900K .......... .......... .......... .......... .......... 59% 9.28M 8s 2024-03-30T13:12:55.532 INFO:teuthology.orchestra.run.smithi043.stderr: 99950K .......... .......... .......... .......... .......... 59% 7.99M 8s 2024-03-30T13:12:55.537 INFO:teuthology.orchestra.run.smithi043.stderr:100000K .......... .......... .......... .......... .......... 59% 9.71M 8s 2024-03-30T13:12:55.542 INFO:teuthology.orchestra.run.smithi043.stderr:100050K .......... .......... .......... .......... .......... 59% 9.77M 8s 2024-03-30T13:12:55.547 INFO:teuthology.orchestra.run.smithi043.stderr:100100K .......... .......... .......... .......... .......... 59% 9.14M 8s 2024-03-30T13:12:55.553 INFO:teuthology.orchestra.run.smithi043.stderr:100150K .......... .......... .......... .......... .......... 59% 7.99M 8s 2024-03-30T13:12:55.559 INFO:teuthology.orchestra.run.smithi043.stderr:100200K .......... .......... .......... .......... .......... 59% 8.69M 8s 2024-03-30T13:12:55.565 INFO:teuthology.orchestra.run.smithi043.stderr:100250K .......... .......... .......... .......... .......... 59% 8.64M 8s 2024-03-30T13:12:55.570 INFO:teuthology.orchestra.run.smithi043.stderr:100300K .......... .......... .......... .......... .......... 59% 8.96M 8s 2024-03-30T13:12:55.577 INFO:teuthology.orchestra.run.smithi043.stderr:100350K .......... .......... .......... .......... .......... 59% 7.45M 8s 2024-03-30T13:12:55.582 INFO:teuthology.orchestra.run.smithi043.stderr:100400K .......... .......... .......... .......... .......... 59% 8.82M 8s 2024-03-30T13:12:55.588 INFO:teuthology.orchestra.run.smithi043.stderr:100450K .......... .......... .......... .......... .......... 59% 8.62M 8s 2024-03-30T13:12:55.593 INFO:teuthology.orchestra.run.smithi043.stderr:100500K .......... .......... .......... .......... .......... 59% 8.71M 8s 2024-03-30T13:12:55.600 INFO:teuthology.orchestra.run.smithi043.stderr:100550K .......... .......... .......... .......... .......... 59% 7.48M 8s 2024-03-30T13:12:55.606 INFO:teuthology.orchestra.run.smithi043.stderr:100600K .......... .......... .......... .......... .......... 59% 8.20M 8s 2024-03-30T13:12:55.612 INFO:teuthology.orchestra.run.smithi043.stderr:100650K .......... .......... .......... .......... .......... 59% 8.15M 8s 2024-03-30T13:12:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:100700K .......... .......... .......... .......... .......... 59% 8.60M 8s 2024-03-30T13:12:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:100750K .......... .......... .......... .......... .......... 59% 6.76M 8s 2024-03-30T13:12:55.631 INFO:teuthology.orchestra.run.smithi043.stderr:100800K .......... .......... .......... .......... .......... 59% 7.64M 8s 2024-03-30T13:12:55.637 INFO:teuthology.orchestra.run.smithi043.stderr:100850K .......... .......... .......... .......... .......... 59% 7.89M 8s 2024-03-30T13:12:55.644 INFO:teuthology.orchestra.run.smithi043.stderr:100900K .......... .......... .......... .......... .......... 59% 7.92M 8s 2024-03-30T13:12:55.651 INFO:teuthology.orchestra.run.smithi043.stderr:100950K .......... .......... .......... .......... .......... 59% 6.84M 8s 2024-03-30T13:12:55.657 INFO:teuthology.orchestra.run.smithi043.stderr:101000K .......... .......... .......... .......... .......... 59% 8.02M 8s 2024-03-30T13:12:55.663 INFO:teuthology.orchestra.run.smithi043.stderr:101050K .......... .......... .......... .......... .......... 59% 7.76M 8s 2024-03-30T13:12:55.669 INFO:teuthology.orchestra.run.smithi043.stderr:101100K .......... .......... .......... .......... .......... 60% 8.06M 8s 2024-03-30T13:12:55.676 INFO:teuthology.orchestra.run.smithi043.stderr:101150K .......... .......... .......... .......... .......... 60% 6.89M 8s 2024-03-30T13:12:55.682 INFO:teuthology.orchestra.run.smithi043.stderr:101200K .......... .......... .......... .......... .......... 60% 7.99M 8s 2024-03-30T13:12:55.689 INFO:teuthology.orchestra.run.smithi043.stderr:101250K .......... .......... .......... .......... .......... 60% 7.75M 8s 2024-03-30T13:12:55.695 INFO:teuthology.orchestra.run.smithi043.stderr:101300K .......... .......... .......... .......... .......... 60% 8.14M 8s 2024-03-30T13:12:55.702 INFO:teuthology.orchestra.run.smithi043.stderr:101350K .......... .......... .......... .......... .......... 60% 6.53M 8s 2024-03-30T13:12:55.709 INFO:teuthology.orchestra.run.smithi043.stderr:101400K .......... .......... .......... .......... .......... 60% 7.69M 8s 2024-03-30T13:12:55.715 INFO:teuthology.orchestra.run.smithi043.stderr:101450K .......... .......... .......... .......... .......... 60% 7.75M 8s 2024-03-30T13:12:55.721 INFO:teuthology.orchestra.run.smithi043.stderr:101500K .......... .......... .......... .......... .......... 60% 7.56M 8s 2024-03-30T13:12:55.729 INFO:teuthology.orchestra.run.smithi043.stderr:101550K .......... .......... .......... .......... .......... 60% 6.58M 8s 2024-03-30T13:12:55.735 INFO:teuthology.orchestra.run.smithi043.stderr:101600K .......... .......... .......... .......... .......... 60% 7.44M 8s 2024-03-30T13:12:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:101650K .......... .......... .......... .......... .......... 60% 7.94M 8s 2024-03-30T13:12:55.748 INFO:teuthology.orchestra.run.smithi043.stderr:101700K .......... .......... .......... .......... .......... 60% 7.56M 8s 2024-03-30T13:12:55.756 INFO:teuthology.orchestra.run.smithi043.stderr:101750K .......... .......... .......... .......... .......... 60% 6.38M 8s 2024-03-30T13:12:55.762 INFO:teuthology.orchestra.run.smithi043.stderr:101800K .......... .......... .......... .......... .......... 60% 7.35M 8s 2024-03-30T13:12:55.769 INFO:teuthology.orchestra.run.smithi043.stderr:101850K .......... .......... .......... .......... .......... 60% 7.35M 8s 2024-03-30T13:12:55.775 INFO:teuthology.orchestra.run.smithi043.stderr:101900K .......... .......... .......... .......... .......... 60% 7.35M 8s 2024-03-30T13:12:55.783 INFO:teuthology.orchestra.run.smithi043.stderr:101950K .......... .......... .......... .......... .......... 60% 6.27M 8s 2024-03-30T13:12:55.790 INFO:teuthology.orchestra.run.smithi043.stderr:102000K .......... .......... .......... .......... .......... 60% 7.45M 8s 2024-03-30T13:12:55.796 INFO:teuthology.orchestra.run.smithi043.stderr:102050K .......... .......... .......... .......... .......... 60% 7.44M 8s 2024-03-30T13:12:55.803 INFO:teuthology.orchestra.run.smithi043.stderr:102100K .......... .......... .......... .......... .......... 60% 7.52M 8s 2024-03-30T13:12:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:102150K .......... .......... .......... .......... .......... 60% 6.96M 8s 2024-03-30T13:12:55.816 INFO:teuthology.orchestra.run.smithi043.stderr:102200K .......... .......... .......... .......... .......... 60% 7.86M 8s 2024-03-30T13:12:55.822 INFO:teuthology.orchestra.run.smithi043.stderr:102250K .......... .......... .......... .......... .......... 60% 8.08M 8s 2024-03-30T13:12:55.828 INFO:teuthology.orchestra.run.smithi043.stderr:102300K .......... .......... .......... .......... .......... 60% 7.95M 8s 2024-03-30T13:12:55.835 INFO:teuthology.orchestra.run.smithi043.stderr:102350K .......... .......... .......... .......... .......... 60% 7.09M 8s 2024-03-30T13:12:55.842 INFO:teuthology.orchestra.run.smithi043.stderr:102400K .......... .......... .......... .......... .......... 60% 7.72M 8s 2024-03-30T13:12:55.845 INFO:teuthology.orchestra.run.smithi043.stderr:102450K .......... .......... .......... .......... .......... 60% 11.3M 8s 2024-03-30T13:12:55.850 INFO:teuthology.orchestra.run.smithi043.stderr:102500K .......... .......... .......... .......... .......... 60% 12.6M 7s 2024-03-30T13:12:55.854 INFO:teuthology.orchestra.run.smithi043.stderr:102550K .......... .......... .......... .......... .......... 60% 9.87M 7s 2024-03-30T13:12:55.858 INFO:teuthology.orchestra.run.smithi043.stderr:102600K .......... .......... .......... .......... .......... 60% 13.2M 7s 2024-03-30T13:12:55.862 INFO:teuthology.orchestra.run.smithi043.stderr:102650K .......... .......... .......... .......... .......... 60% 13.1M 7s 2024-03-30T13:12:55.866 INFO:teuthology.orchestra.run.smithi043.stderr:102700K .......... .......... .......... .......... .......... 60% 12.3M 7s 2024-03-30T13:12:55.871 INFO:teuthology.orchestra.run.smithi043.stderr:102750K .......... .......... .......... .......... .......... 60% 9.88M 7s 2024-03-30T13:12:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:102800K .......... .......... .......... .......... .......... 61% 8.65M 7s 2024-03-30T13:12:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:102850K .......... .......... .......... .......... .......... 61% 8.42M 7s 2024-03-30T13:12:55.887 INFO:teuthology.orchestra.run.smithi043.stderr:102900K .......... .......... .......... .......... .......... 61% 10.1M 7s 2024-03-30T13:12:55.892 INFO:teuthology.orchestra.run.smithi043.stderr:102950K .......... .......... .......... .......... .......... 61% 10.0M 7s 2024-03-30T13:12:55.896 INFO:teuthology.orchestra.run.smithi043.stderr:103000K .......... .......... .......... .......... .......... 61% 11.6M 7s 2024-03-30T13:12:55.901 INFO:teuthology.orchestra.run.smithi043.stderr:103050K .......... .......... .......... .......... .......... 61% 11.1M 7s 2024-03-30T13:12:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:103100K .......... .......... .......... .......... .......... 61% 11.5M 7s 2024-03-30T13:12:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:103150K .......... .......... .......... .......... .......... 61% 9.70M 7s 2024-03-30T13:12:55.914 INFO:teuthology.orchestra.run.smithi043.stderr:103200K .......... .......... .......... .......... .......... 61% 11.4M 7s 2024-03-30T13:12:55.918 INFO:teuthology.orchestra.run.smithi043.stderr:103250K .......... .......... .......... .......... .......... 61% 11.1M 7s 2024-03-30T13:12:55.923 INFO:teuthology.orchestra.run.smithi043.stderr:103300K .......... .......... .......... .......... .......... 61% 10.3M 7s 2024-03-30T13:12:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:103350K .......... .......... .......... .......... .......... 61% 9.60M 7s 2024-03-30T13:12:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:103400K .......... .......... .......... .......... .......... 61% 11.3M 7s 2024-03-30T13:12:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:103450K .......... .......... .......... .......... .......... 61% 10.6M 7s 2024-03-30T13:12:55.942 INFO:teuthology.orchestra.run.smithi043.stderr:103500K .......... .......... .......... .......... .......... 61% 10.4M 7s 2024-03-30T13:12:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:103550K .......... .......... .......... .......... .......... 61% 9.20M 7s 2024-03-30T13:12:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:103600K .......... .......... .......... .......... .......... 61% 10.8M 7s 2024-03-30T13:12:55.956 INFO:teuthology.orchestra.run.smithi043.stderr:103650K .......... .......... .......... .......... .......... 61% 11.2M 7s 2024-03-30T13:12:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:103700K .......... .......... .......... .......... .......... 61% 11.0M 7s 2024-03-30T13:12:55.966 INFO:teuthology.orchestra.run.smithi043.stderr:103750K .......... .......... .......... .......... .......... 61% 9.41M 7s 2024-03-30T13:12:55.970 INFO:teuthology.orchestra.run.smithi043.stderr:103800K .......... .......... .......... .......... .......... 61% 11.1M 7s 2024-03-30T13:12:55.974 INFO:teuthology.orchestra.run.smithi043.stderr:103850K .......... .......... .......... .......... .......... 61% 10.9M 7s 2024-03-30T13:12:55.979 INFO:teuthology.orchestra.run.smithi043.stderr:103900K .......... .......... .......... .......... .......... 61% 11.1M 7s 2024-03-30T13:12:55.984 INFO:teuthology.orchestra.run.smithi043.stderr:103950K .......... .......... .......... .......... .......... 61% 9.21M 7s 2024-03-30T13:12:55.989 INFO:teuthology.orchestra.run.smithi043.stderr:104000K .......... .......... .......... .......... .......... 61% 10.6M 7s 2024-03-30T13:12:55.994 INFO:teuthology.orchestra.run.smithi043.stderr:104050K .......... .......... .......... .......... .......... 61% 9.70M 7s 2024-03-30T13:12:56.002 INFO:teuthology.orchestra.run.smithi043.stderr:104100K .......... .......... .......... .......... .......... 61% 9.74M 7s 2024-03-30T13:12:56.005 INFO:teuthology.orchestra.run.smithi043.stderr:104150K .......... .......... .......... .......... .......... 61% 8.42M 7s 2024-03-30T13:12:56.010 INFO:teuthology.orchestra.run.smithi043.stderr:104200K .......... .......... .......... .......... .......... 61% 9.21M 7s 2024-03-30T13:12:56.015 INFO:teuthology.orchestra.run.smithi043.stderr:104250K .......... .......... .......... .......... .......... 61% 9.40M 7s 2024-03-30T13:12:56.021 INFO:teuthology.orchestra.run.smithi043.stderr:104300K .......... .......... .......... .......... .......... 61% 9.12M 7s 2024-03-30T13:12:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:104350K .......... .......... .......... .......... .......... 61% 7.73M 7s 2024-03-30T13:12:56.032 INFO:teuthology.orchestra.run.smithi043.stderr:104400K .......... .......... .......... .......... .......... 61% 9.21M 7s 2024-03-30T13:12:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:104450K .......... .......... .......... .......... .......... 61% 9.67M 7s 2024-03-30T13:12:56.043 INFO:teuthology.orchestra.run.smithi043.stderr:104500K .......... .......... .......... .......... .......... 62% 8.84M 7s 2024-03-30T13:12:56.049 INFO:teuthology.orchestra.run.smithi043.stderr:104550K .......... .......... .......... .......... .......... 62% 7.98M 7s 2024-03-30T13:12:56.055 INFO:teuthology.orchestra.run.smithi043.stderr:104600K .......... .......... .......... .......... .......... 62% 8.66M 7s 2024-03-30T13:12:56.060 INFO:teuthology.orchestra.run.smithi043.stderr:104650K .......... .......... .......... .......... .......... 62% 8.74M 7s 2024-03-30T13:12:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:104700K .......... .......... .......... .......... .......... 62% 8.59M 7s 2024-03-30T13:12:56.073 INFO:teuthology.orchestra.run.smithi043.stderr:104750K .......... .......... .......... .......... .......... 62% 7.50M 7s 2024-03-30T13:12:56.078 INFO:teuthology.orchestra.run.smithi043.stderr:104800K .......... .......... .......... .......... .......... 62% 8.95M 7s 2024-03-30T13:12:56.084 INFO:teuthology.orchestra.run.smithi043.stderr:104850K .......... .......... .......... .......... .......... 62% 7.66M 7s 2024-03-30T13:12:56.090 INFO:teuthology.orchestra.run.smithi043.stderr:104900K .......... .......... .......... .......... .......... 62% 8.19M 7s 2024-03-30T13:12:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:104950K .......... .......... .......... .......... .......... 62% 7.62M 7s 2024-03-30T13:12:56.103 INFO:teuthology.orchestra.run.smithi043.stderr:105000K .......... .......... .......... .......... .......... 62% 8.55M 7s 2024-03-30T13:12:56.109 INFO:teuthology.orchestra.run.smithi043.stderr:105050K .......... .......... .......... .......... .......... 62% 8.07M 7s 2024-03-30T13:12:56.115 INFO:teuthology.orchestra.run.smithi043.stderr:105100K .......... .......... .......... .......... .......... 62% 8.15M 7s 2024-03-30T13:12:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:105150K .......... .......... .......... .......... .......... 62% 7.02M 7s 2024-03-30T13:12:56.128 INFO:teuthology.orchestra.run.smithi043.stderr:105200K .......... .......... .......... .......... .......... 62% 7.85M 7s 2024-03-30T13:12:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:105250K .......... .......... .......... .......... .......... 62% 7.56M 7s 2024-03-30T13:12:56.140 INFO:teuthology.orchestra.run.smithi043.stderr:105300K .......... .......... .......... .......... .......... 62% 8.56M 7s 2024-03-30T13:12:56.147 INFO:teuthology.orchestra.run.smithi043.stderr:105350K .......... .......... .......... .......... .......... 62% 7.25M 7s 2024-03-30T13:12:56.153 INFO:teuthology.orchestra.run.smithi043.stderr:105400K .......... .......... .......... .......... .......... 62% 7.81M 7s 2024-03-30T13:12:56.160 INFO:teuthology.orchestra.run.smithi043.stderr:105450K .......... .......... .......... .......... .......... 62% 6.75M 7s 2024-03-30T13:12:56.168 INFO:teuthology.orchestra.run.smithi043.stderr:105500K .......... .......... .......... .......... .......... 62% 6.48M 7s 2024-03-30T13:12:56.176 INFO:teuthology.orchestra.run.smithi043.stderr:105550K .......... .......... .......... .......... .......... 62% 5.77M 7s 2024-03-30T13:12:56.184 INFO:teuthology.orchestra.run.smithi043.stderr:105600K .......... .......... .......... .......... .......... 62% 6.53M 7s 2024-03-30T13:12:56.191 INFO:teuthology.orchestra.run.smithi043.stderr:105650K .......... .......... .......... .......... .......... 62% 6.58M 7s 2024-03-30T13:12:56.198 INFO:teuthology.orchestra.run.smithi043.stderr:105700K .......... .......... .......... .......... .......... 62% 6.73M 7s 2024-03-30T13:12:56.211 INFO:teuthology.orchestra.run.smithi043.stderr:105750K .......... .......... .......... .......... .......... 62% 5.66M 7s 2024-03-30T13:12:56.214 INFO:teuthology.orchestra.run.smithi043.stderr:105800K .......... .......... .......... .......... .......... 62% 6.61M 7s 2024-03-30T13:12:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:105850K .......... .......... .......... .......... .......... 62% 6.43M 7s 2024-03-30T13:12:56.230 INFO:teuthology.orchestra.run.smithi043.stderr:105900K .......... .......... .......... .......... .......... 62% 6.48M 7s 2024-03-30T13:12:56.238 INFO:teuthology.orchestra.run.smithi043.stderr:105950K .......... .......... .......... .......... .......... 62% 5.57M 7s 2024-03-30T13:12:56.245 INFO:teuthology.orchestra.run.smithi043.stderr:106000K .......... .......... .......... .......... .......... 62% 7.52M 7s 2024-03-30T13:12:56.250 INFO:teuthology.orchestra.run.smithi043.stderr:106050K .......... .......... .......... .......... .......... 62% 8.93M 7s 2024-03-30T13:12:56.256 INFO:teuthology.orchestra.run.smithi043.stderr:106100K .......... .......... .......... .......... .......... 62% 8.08M 7s 2024-03-30T13:12:56.264 INFO:teuthology.orchestra.run.smithi043.stderr:106150K .......... .......... .......... .......... .......... 63% 6.62M 7s 2024-03-30T13:12:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:106200K .......... .......... .......... .......... .......... 63% 7.51M 7s 2024-03-30T13:12:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:106250K .......... .......... .......... .......... .......... 63% 7.15M 7s 2024-03-30T13:12:56.284 INFO:teuthology.orchestra.run.smithi043.stderr:106300K .......... .......... .......... .......... .......... 63% 7.18M 7s 2024-03-30T13:12:56.291 INFO:teuthology.orchestra.run.smithi043.stderr:106350K .......... .......... .......... .......... .......... 63% 6.46M 7s 2024-03-30T13:12:56.297 INFO:teuthology.orchestra.run.smithi043.stderr:106400K .......... .......... .......... .......... .......... 63% 8.05M 7s 2024-03-30T13:12:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:106450K .......... .......... .......... .......... .......... 63% 8.14M 7s 2024-03-30T13:12:56.308 INFO:teuthology.orchestra.run.smithi043.stderr:106500K .......... .......... .......... .......... .......... 63% 10.4M 7s 2024-03-30T13:12:56.314 INFO:teuthology.orchestra.run.smithi043.stderr:106550K .......... .......... .......... .......... .......... 63% 7.85M 7s 2024-03-30T13:12:56.319 INFO:teuthology.orchestra.run.smithi043.stderr:106600K .......... .......... .......... .......... .......... 63% 8.89M 7s 2024-03-30T13:12:56.325 INFO:teuthology.orchestra.run.smithi043.stderr:106650K .......... .......... .......... .......... .......... 63% 9.32M 7s 2024-03-30T13:12:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:106700K .......... .......... .......... .......... .......... 63% 10.8M 7s 2024-03-30T13:12:56.335 INFO:teuthology.orchestra.run.smithi043.stderr:106750K .......... .......... .......... .......... .......... 63% 8.89M 7s 2024-03-30T13:12:56.339 INFO:teuthology.orchestra.run.smithi043.stderr:106800K .......... .......... .......... .......... .......... 63% 12.0M 7s 2024-03-30T13:12:56.343 INFO:teuthology.orchestra.run.smithi043.stderr:106850K .......... .......... .......... .......... .......... 63% 11.6M 7s 2024-03-30T13:12:56.347 INFO:teuthology.orchestra.run.smithi043.stderr:106900K .......... .......... .......... .......... .......... 63% 11.6M 7s 2024-03-30T13:12:56.352 INFO:teuthology.orchestra.run.smithi043.stderr:106950K .......... .......... .......... .......... .......... 63% 9.41M 7s 2024-03-30T13:12:56.356 INFO:teuthology.orchestra.run.smithi043.stderr:107000K .......... .......... .......... .......... .......... 63% 11.8M 7s 2024-03-30T13:12:56.361 INFO:teuthology.orchestra.run.smithi043.stderr:107050K .......... .......... .......... .......... .......... 63% 11.4M 7s 2024-03-30T13:12:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:107100K .......... .......... .......... .......... .......... 63% 11.2M 7s 2024-03-30T13:12:56.370 INFO:teuthology.orchestra.run.smithi043.stderr:107150K .......... .......... .......... .......... .......... 63% 10.4M 7s 2024-03-30T13:12:56.374 INFO:teuthology.orchestra.run.smithi043.stderr:107200K .......... .......... .......... .......... .......... 63% 11.3M 7s 2024-03-30T13:12:56.378 INFO:teuthology.orchestra.run.smithi043.stderr:107250K .......... .......... .......... .......... .......... 63% 11.5M 7s 2024-03-30T13:12:56.383 INFO:teuthology.orchestra.run.smithi043.stderr:107300K .......... .......... .......... .......... .......... 63% 11.1M 7s 2024-03-30T13:12:56.388 INFO:teuthology.orchestra.run.smithi043.stderr:107350K .......... .......... .......... .......... .......... 63% 9.46M 7s 2024-03-30T13:12:56.393 INFO:teuthology.orchestra.run.smithi043.stderr:107400K .......... .......... .......... .......... .......... 63% 9.94M 7s 2024-03-30T13:12:56.398 INFO:teuthology.orchestra.run.smithi043.stderr:107450K .......... .......... .......... .......... .......... 63% 9.69M 7s 2024-03-30T13:12:56.403 INFO:teuthology.orchestra.run.smithi043.stderr:107500K .......... .......... .......... .......... .......... 63% 10.6M 7s 2024-03-30T13:12:56.408 INFO:teuthology.orchestra.run.smithi043.stderr:107550K .......... .......... .......... .......... .......... 63% 8.67M 7s 2024-03-30T13:12:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:107600K .......... .......... .......... .......... .......... 63% 10.1M 7s 2024-03-30T13:12:56.418 INFO:teuthology.orchestra.run.smithi043.stderr:107650K .......... .......... .......... .......... .......... 63% 10.1M 7s 2024-03-30T13:12:56.423 INFO:teuthology.orchestra.run.smithi043.stderr:107700K .......... .......... .......... .......... .......... 63% 9.33M 7s 2024-03-30T13:12:56.428 INFO:teuthology.orchestra.run.smithi043.stderr:107750K .......... .......... .......... .......... .......... 63% 9.21M 7s 2024-03-30T13:12:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:107800K .......... .......... .......... .......... .......... 63% 9.99M 7s 2024-03-30T13:12:56.438 INFO:teuthology.orchestra.run.smithi043.stderr:107850K .......... .......... .......... .......... .......... 64% 10.5M 7s 2024-03-30T13:12:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:107900K .......... .......... .......... .......... .......... 64% 10.1M 7s 2024-03-30T13:12:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:107950K .......... .......... .......... .......... .......... 64% 7.93M 7s 2024-03-30T13:12:56.454 INFO:teuthology.orchestra.run.smithi043.stderr:108000K .......... .......... .......... .......... .......... 64% 9.08M 7s 2024-03-30T13:12:56.459 INFO:teuthology.orchestra.run.smithi043.stderr:108050K .......... .......... .......... .......... .......... 64% 9.57M 7s 2024-03-30T13:12:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:108100K .......... .......... .......... .......... .......... 64% 9.82M 7s 2024-03-30T13:12:56.470 INFO:teuthology.orchestra.run.smithi043.stderr:108150K .......... .......... .......... .......... .......... 64% 8.45M 7s 2024-03-30T13:12:56.475 INFO:teuthology.orchestra.run.smithi043.stderr:108200K .......... .......... .......... .......... .......... 64% 9.47M 7s 2024-03-30T13:12:56.480 INFO:teuthology.orchestra.run.smithi043.stderr:108250K .......... .......... .......... .......... .......... 64% 9.71M 7s 2024-03-30T13:12:56.485 INFO:teuthology.orchestra.run.smithi043.stderr:108300K .......... .......... .......... .......... .......... 64% 9.95M 7s 2024-03-30T13:12:56.491 INFO:teuthology.orchestra.run.smithi043.stderr:108350K .......... .......... .......... .......... .......... 64% 8.09M 7s 2024-03-30T13:12:56.497 INFO:teuthology.orchestra.run.smithi043.stderr:108400K .......... .......... .......... .......... .......... 64% 9.41M 7s 2024-03-30T13:12:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:108450K .......... .......... .......... .......... .......... 64% 9.29M 7s 2024-03-30T13:12:56.507 INFO:teuthology.orchestra.run.smithi043.stderr:108500K .......... .......... .......... .......... .......... 64% 8.78M 7s 2024-03-30T13:12:56.513 INFO:teuthology.orchestra.run.smithi043.stderr:108550K .......... .......... .......... .......... .......... 64% 8.11M 7s 2024-03-30T13:12:56.519 INFO:teuthology.orchestra.run.smithi043.stderr:108600K .......... .......... .......... .......... .......... 64% 9.14M 7s 2024-03-30T13:12:56.524 INFO:teuthology.orchestra.run.smithi043.stderr:108650K .......... .......... .......... .......... .......... 64% 9.24M 7s 2024-03-30T13:12:56.530 INFO:teuthology.orchestra.run.smithi043.stderr:108700K .......... .......... .......... .......... .......... 64% 8.74M 7s 2024-03-30T13:12:56.536 INFO:teuthology.orchestra.run.smithi043.stderr:108750K .......... .......... .......... .......... .......... 64% 7.86M 7s 2024-03-30T13:12:56.541 INFO:teuthology.orchestra.run.smithi043.stderr:108800K .......... .......... .......... .......... .......... 64% 9.00M 7s 2024-03-30T13:12:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:108850K .......... .......... .......... .......... .......... 64% 9.67M 7s 2024-03-30T13:12:56.552 INFO:teuthology.orchestra.run.smithi043.stderr:108900K .......... .......... .......... .......... .......... 64% 9.42M 7s 2024-03-30T13:12:56.558 INFO:teuthology.orchestra.run.smithi043.stderr:108950K .......... .......... .......... .......... .......... 64% 8.11M 7s 2024-03-30T13:12:56.564 INFO:teuthology.orchestra.run.smithi043.stderr:109000K .......... .......... .......... .......... .......... 64% 8.27M 7s 2024-03-30T13:12:56.570 INFO:teuthology.orchestra.run.smithi043.stderr:109050K .......... .......... .......... .......... .......... 64% 7.97M 7s 2024-03-30T13:12:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:109100K .......... .......... .......... .......... .......... 64% 7.77M 7s 2024-03-30T13:12:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:109150K .......... .......... .......... .......... .......... 64% 7.09M 7s 2024-03-30T13:12:56.589 INFO:teuthology.orchestra.run.smithi043.stderr:109200K .......... .......... .......... .......... .......... 64% 8.10M 7s 2024-03-30T13:12:56.595 INFO:teuthology.orchestra.run.smithi043.stderr:109250K .......... .......... .......... .......... .......... 64% 8.06M 7s 2024-03-30T13:12:56.601 INFO:teuthology.orchestra.run.smithi043.stderr:109300K .......... .......... .......... .......... .......... 64% 7.82M 7s 2024-03-30T13:12:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:109350K .......... .......... .......... .......... .......... 64% 6.95M 7s 2024-03-30T13:12:56.614 INFO:teuthology.orchestra.run.smithi043.stderr:109400K .......... .......... .......... .......... .......... 64% 8.77M 7s 2024-03-30T13:12:56.619 INFO:teuthology.orchestra.run.smithi043.stderr:109450K .......... .......... .......... .......... .......... 64% 8.77M 7s 2024-03-30T13:12:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:109500K .......... .......... .......... .......... .......... 64% 8.82M 7s 2024-03-30T13:12:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:109550K .......... .......... .......... .......... .......... 65% 7.13M 7s 2024-03-30T13:12:56.637 INFO:teuthology.orchestra.run.smithi043.stderr:109600K .......... .......... .......... .......... .......... 65% 8.42M 7s 2024-03-30T13:12:56.643 INFO:teuthology.orchestra.run.smithi043.stderr:109650K .......... .......... .......... .......... .......... 65% 8.48M 7s 2024-03-30T13:12:56.649 INFO:teuthology.orchestra.run.smithi043.stderr:109700K .......... .......... .......... .......... .......... 65% 8.47M 7s 2024-03-30T13:12:56.656 INFO:teuthology.orchestra.run.smithi043.stderr:109750K .......... .......... .......... .......... .......... 65% 6.70M 7s 2024-03-30T13:12:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:109800K .......... .......... .......... .......... .......... 65% 8.06M 7s 2024-03-30T13:12:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:109850K .......... .......... .......... .......... .......... 65% 8.27M 7s 2024-03-30T13:12:56.674 INFO:teuthology.orchestra.run.smithi043.stderr:109900K .......... .......... .......... .......... .......... 65% 7.99M 7s 2024-03-30T13:12:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:109950K .......... .......... .......... .......... .......... 65% 6.73M 7s 2024-03-30T13:12:56.684 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:56.687 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:12:56.689 INFO:teuthology.orchestra.run.smithi043.stderr:110000K .......... .......... .......... .......... .......... 65% 7.29M 7s 2024-03-30T13:12:56.694 INFO:teuthology.orchestra.run.smithi043.stderr:110050K .......... .......... .......... .......... .......... 65% 7.80M 7s 2024-03-30T13:12:56.700 INFO:teuthology.orchestra.run.smithi043.stderr:110100K .......... .......... .......... .......... .......... 65% 8.11M 7s 2024-03-30T13:12:56.707 INFO:teuthology.orchestra.run.smithi043.stderr:110150K .......... .......... .......... .......... .......... 65% 8.03M 7s 2024-03-30T13:12:56.712 INFO:teuthology.orchestra.run.smithi043.stderr:110200K .......... .......... .......... .......... .......... 65% 9.17M 7s 2024-03-30T13:12:56.717 INFO:teuthology.orchestra.run.smithi043.stderr:110250K .......... .......... .......... .......... .......... 65% 9.21M 7s 2024-03-30T13:12:56.723 INFO:teuthology.orchestra.run.smithi043.stderr:110300K .......... .......... .......... .......... .......... 65% 8.47M 7s 2024-03-30T13:12:56.730 INFO:teuthology.orchestra.run.smithi043.stderr:110350K .......... .......... .......... .......... .......... 65% 7.11M 7s 2024-03-30T13:12:56.736 INFO:teuthology.orchestra.run.smithi043.stderr:110400K .......... .......... .......... .......... .......... 65% 8.07M 7s 2024-03-30T13:12:56.742 INFO:teuthology.orchestra.run.smithi043.stderr:110450K .......... .......... .......... .......... .......... 65% 8.30M 7s 2024-03-30T13:12:56.748 INFO:teuthology.orchestra.run.smithi043.stderr:110500K .......... .......... .......... .......... .......... 65% 8.10M 7s 2024-03-30T13:12:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:110550K .......... .......... .......... .......... .......... 65% 7.04M 7s 2024-03-30T13:12:56.760 INFO:teuthology.orchestra.run.smithi043.stderr:110600K .......... .......... .......... .......... .......... 65% 8.51M 7s 2024-03-30T13:12:56.765 INFO:teuthology.orchestra.run.smithi043.stderr:110650K .......... .......... .......... .......... .......... 65% 11.9M 7s 2024-03-30T13:12:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:110700K .......... .......... .......... .......... .......... 65% 12.4M 7s 2024-03-30T13:12:56.773 INFO:teuthology.orchestra.run.smithi043.stderr:110750K .......... .......... .......... .......... .......... 65% 9.57M 7s 2024-03-30T13:12:56.778 INFO:teuthology.orchestra.run.smithi043.stderr:110800K .......... .......... .......... .......... .......... 65% 11.3M 7s 2024-03-30T13:12:56.781 INFO:teuthology.orchestra.run.smithi043.stderr:110850K .......... .......... .......... .......... .......... 65% 14.6M 7s 2024-03-30T13:12:56.783 INFO:teuthology.orchestra.run.smithi043.stderr:110900K .......... .......... .......... .......... .......... 65% 18.1M 7s 2024-03-30T13:12:56.787 INFO:teuthology.orchestra.run.smithi043.stderr:110950K .......... .......... .......... .......... .......... 65% 15.6M 7s 2024-03-30T13:12:56.789 INFO:teuthology.orchestra.run.smithi043.stderr:111000K .......... .......... .......... .......... .......... 65% 17.6M 7s 2024-03-30T13:12:56.792 INFO:teuthology.orchestra.run.smithi043.stderr:111050K .......... .......... .......... .......... .......... 65% 17.9M 7s 2024-03-30T13:12:56.795 INFO:teuthology.orchestra.run.smithi043.stderr:111100K .......... .......... .......... .......... .......... 65% 17.3M 7s 2024-03-30T13:12:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:111150K .......... .......... .......... .......... .......... 65% 14.8M 6s 2024-03-30T13:12:56.801 INFO:teuthology.orchestra.run.smithi043.stderr:111200K .......... .......... .......... .......... .......... 65% 17.4M 6s 2024-03-30T13:12:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:111250K .......... .......... .......... .......... .......... 66% 14.7M 6s 2024-03-30T13:12:56.808 INFO:teuthology.orchestra.run.smithi043.stderr:111300K .......... .......... .......... .......... .......... 66% 13.5M 6s 2024-03-30T13:12:56.814 INFO:teuthology.orchestra.run.smithi043.stderr:111350K .......... .......... .......... .......... .......... 66% 11.7M 6s 2024-03-30T13:12:56.817 INFO:teuthology.orchestra.run.smithi043.stderr:111400K .......... .......... .......... .......... .......... 66% 11.7M 6s 2024-03-30T13:12:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:111450K .......... .......... .......... .......... .......... 66% 11.2M 6s 2024-03-30T13:12:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:111500K .......... .......... .......... .......... .......... 66% 11.8M 6s 2024-03-30T13:12:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:111550K .......... .......... .......... .......... .......... 66% 9.97M 6s 2024-03-30T13:12:56.835 INFO:teuthology.orchestra.run.smithi043.stderr:111600K .......... .......... .......... .......... .......... 66% 11.4M 6s 2024-03-30T13:12:56.839 INFO:teuthology.orchestra.run.smithi043.stderr:111650K .......... .......... .......... .......... .......... 66% 11.7M 6s 2024-03-30T13:12:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:111700K .......... .......... .......... .......... .......... 66% 11.9M 6s 2024-03-30T13:12:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:111750K .......... .......... .......... .......... .......... 66% 8.86M 6s 2024-03-30T13:12:56.853 INFO:teuthology.orchestra.run.smithi043.stderr:111800K .......... .......... .......... .......... .......... 66% 9.99M 6s 2024-03-30T13:12:56.858 INFO:teuthology.orchestra.run.smithi043.stderr:111850K .......... .......... .......... .......... .......... 66% 10.2M 6s 2024-03-30T13:12:56.862 INFO:teuthology.orchestra.run.smithi043.stderr:111900K .......... .......... .......... .......... .......... 66% 10.8M 6s 2024-03-30T13:12:56.868 INFO:teuthology.orchestra.run.smithi043.stderr:111950K .......... .......... .......... .......... .......... 66% 8.98M 6s 2024-03-30T13:12:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:112000K .......... .......... .......... .......... .......... 66% 10.7M 6s 2024-03-30T13:12:56.877 INFO:teuthology.orchestra.run.smithi043.stderr:112050K .......... .......... .......... .......... .......... 66% 11.5M 6s 2024-03-30T13:12:56.881 INFO:teuthology.orchestra.run.smithi043.stderr:112100K .......... .......... .......... .......... .......... 66% 10.3M 6s 2024-03-30T13:12:56.887 INFO:teuthology.orchestra.run.smithi043.stderr:112150K .......... .......... .......... .......... .......... 66% 8.87M 6s 2024-03-30T13:12:56.891 INFO:teuthology.orchestra.run.smithi043.stderr:112200K .......... .......... .......... .......... .......... 66% 10.7M 6s 2024-03-30T13:12:56.896 INFO:teuthology.orchestra.run.smithi043.stderr:112250K .......... .......... .......... .......... .......... 66% 10.1M 6s 2024-03-30T13:12:56.901 INFO:teuthology.orchestra.run.smithi043.stderr:112300K .......... .......... .......... .......... .......... 66% 9.81M 6s 2024-03-30T13:12:56.908 INFO:teuthology.orchestra.run.smithi043.stderr:112350K .......... .......... .......... .......... .......... 66% 8.19M 6s 2024-03-30T13:12:56.912 INFO:teuthology.orchestra.run.smithi043.stderr:112400K .......... .......... .......... .......... .......... 66% 9.27M 6s 2024-03-30T13:12:56.918 INFO:teuthology.orchestra.run.smithi043.stderr:112450K .......... .......... .......... .......... .......... 66% 9.20M 6s 2024-03-30T13:12:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:112500K .......... .......... .......... .......... .......... 66% 9.29M 6s 2024-03-30T13:12:56.930 INFO:teuthology.orchestra.run.smithi043.stderr:112550K .......... .......... .......... .......... .......... 66% 7.35M 6s 2024-03-30T13:12:56.935 INFO:teuthology.orchestra.run.smithi043.stderr:112600K .......... .......... .......... .......... .......... 66% 9.02M 6s 2024-03-30T13:12:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:112650K .......... .......... .......... .......... .......... 66% 9.41M 6s 2024-03-30T13:12:56.946 INFO:teuthology.orchestra.run.smithi043.stderr:112700K .......... .......... .......... .......... .......... 66% 8.50M 6s 2024-03-30T13:12:56.954 INFO:teuthology.orchestra.run.smithi043.stderr:112750K .......... .......... .......... .......... .......... 66% 6.63M 6s 2024-03-30T13:12:56.960 INFO:teuthology.orchestra.run.smithi043.stderr:112800K .......... .......... .......... .......... .......... 66% 7.77M 6s 2024-03-30T13:12:56.966 INFO:teuthology.orchestra.run.smithi043.stderr:112850K .......... .......... .......... .......... .......... 66% 8.01M 6s 2024-03-30T13:12:56.972 INFO:teuthology.orchestra.run.smithi043.stderr:112900K .......... .......... .......... .......... .......... 67% 8.07M 6s 2024-03-30T13:12:56.979 INFO:teuthology.orchestra.run.smithi043.stderr:112950K .......... .......... .......... .......... .......... 67% 7.05M 6s 2024-03-30T13:12:56.985 INFO:teuthology.orchestra.run.smithi043.stderr:113000K .......... .......... .......... .......... .......... 67% 7.90M 6s 2024-03-30T13:12:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:113050K .......... .......... .......... .......... .......... 67% 8.19M 6s 2024-03-30T13:12:56.998 INFO:teuthology.orchestra.run.smithi043.stderr:113100K .......... .......... .......... .......... .......... 67% 7.68M 6s 2024-03-30T13:12:57.005 INFO:teuthology.orchestra.run.smithi043.stderr:113150K .......... .......... .......... .......... .......... 67% 6.47M 6s 2024-03-30T13:12:57.012 INFO:teuthology.orchestra.run.smithi043.stderr:113200K .......... .......... .......... .......... .......... 67% 7.23M 6s 2024-03-30T13:12:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:113250K .......... .......... .......... .......... .......... 67% 7.17M 6s 2024-03-30T13:12:57.026 INFO:teuthology.orchestra.run.smithi043.stderr:113300K .......... .......... .......... .......... .......... 67% 7.32M 6s 2024-03-30T13:12:57.033 INFO:teuthology.orchestra.run.smithi043.stderr:113350K .......... .......... .......... .......... .......... 67% 6.31M 6s 2024-03-30T13:12:57.040 INFO:teuthology.orchestra.run.smithi043.stderr:113400K .......... .......... .......... .......... .......... 67% 7.22M 6s 2024-03-30T13:12:57.047 INFO:teuthology.orchestra.run.smithi043.stderr:113450K .......... .......... .......... .......... .......... 67% 7.33M 6s 2024-03-30T13:12:57.053 INFO:teuthology.orchestra.run.smithi043.stderr:113500K .......... .......... .......... .......... .......... 67% 7.72M 6s 2024-03-30T13:12:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:113550K .......... .......... .......... .......... .......... 67% 6.22M 6s 2024-03-30T13:12:57.067 INFO:teuthology.orchestra.run.smithi043.stderr:113600K .......... .......... .......... .......... .......... 67% 7.34M 6s 2024-03-30T13:12:57.074 INFO:teuthology.orchestra.run.smithi043.stderr:113650K .......... .......... .......... .......... .......... 67% 7.19M 6s 2024-03-30T13:12:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:113700K .......... .......... .......... .......... .......... 67% 7.12M 6s 2024-03-30T13:12:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:113750K .......... .......... .......... .......... .......... 67% 6.05M 6s 2024-03-30T13:12:57.096 INFO:teuthology.orchestra.run.smithi043.stderr:113800K .......... .......... .......... .......... .......... 67% 7.36M 6s 2024-03-30T13:12:57.102 INFO:teuthology.orchestra.run.smithi043.stderr:113850K .......... .......... .......... .......... .......... 67% 7.88M 6s 2024-03-30T13:12:57.108 INFO:teuthology.orchestra.run.smithi043.stderr:113900K .......... .......... .......... .......... .......... 67% 8.08M 6s 2024-03-30T13:12:57.115 INFO:teuthology.orchestra.run.smithi043.stderr:113950K .......... .......... .......... .......... .......... 67% 6.65M 6s 2024-03-30T13:12:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:114000K .......... .......... .......... .......... .......... 67% 7.74M 6s 2024-03-30T13:12:57.128 INFO:teuthology.orchestra.run.smithi043.stderr:114050K .......... .......... .......... .......... .......... 67% 7.57M 6s 2024-03-30T13:12:57.134 INFO:teuthology.orchestra.run.smithi043.stderr:114100K .......... .......... .......... .......... .......... 67% 8.23M 6s 2024-03-30T13:12:57.141 INFO:teuthology.orchestra.run.smithi043.stderr:114150K .......... .......... .......... .......... .......... 67% 6.56M 6s 2024-03-30T13:12:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:114200K .......... .......... .......... .......... .......... 67% 7.92M 6s 2024-03-30T13:12:57.154 INFO:teuthology.orchestra.run.smithi043.stderr:114250K .......... .......... .......... .......... .......... 67% 7.94M 6s 2024-03-30T13:12:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:114300K .......... .......... .......... .......... .......... 67% 7.78M 6s 2024-03-30T13:12:57.168 INFO:teuthology.orchestra.run.smithi043.stderr:114350K .......... .......... .......... .......... .......... 67% 6.19M 6s 2024-03-30T13:12:57.174 INFO:teuthology.orchestra.run.smithi043.stderr:114400K .......... .......... .......... .......... .......... 67% 7.79M 6s 2024-03-30T13:12:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:114450K .......... .......... .......... .......... .......... 67% 7.95M 6s 2024-03-30T13:12:57.187 INFO:teuthology.orchestra.run.smithi043.stderr:114500K .......... .......... .......... .......... .......... 67% 7.92M 6s 2024-03-30T13:12:57.194 INFO:teuthology.orchestra.run.smithi043.stderr:114550K .......... .......... .......... .......... .......... 67% 6.81M 6s 2024-03-30T13:12:57.200 INFO:teuthology.orchestra.run.smithi043.stderr:114600K .......... .......... .......... .......... .......... 68% 7.72M 6s 2024-03-30T13:12:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:114650K .......... .......... .......... .......... .......... 68% 8.00M 6s 2024-03-30T13:12:57.210 INFO:teuthology.orchestra.run.smithi043.stderr:114700K .......... .......... .......... .......... .......... 68% 11.1M 6s 2024-03-30T13:12:57.216 INFO:teuthology.orchestra.run.smithi043.stderr:114750K .......... .......... .......... .......... .......... 68% 8.01M 6s 2024-03-30T13:12:57.221 INFO:teuthology.orchestra.run.smithi043.stderr:114800K .......... .......... .......... .......... .......... 68% 9.90M 6s 2024-03-30T13:12:57.225 INFO:teuthology.orchestra.run.smithi043.stderr:114850K .......... .......... .......... .......... .......... 68% 12.7M 6s 2024-03-30T13:12:57.230 INFO:teuthology.orchestra.run.smithi043.stderr:114900K .......... .......... .......... .......... .......... 68% 12.4M 6s 2024-03-30T13:12:57.234 INFO:teuthology.orchestra.run.smithi043.stderr:114950K .......... .......... .......... .......... .......... 68% 10.5M 6s 2024-03-30T13:12:57.238 INFO:teuthology.orchestra.run.smithi043.stderr:115000K .......... .......... .......... .......... .......... 68% 10.7M 6s 2024-03-30T13:12:57.242 INFO:teuthology.orchestra.run.smithi043.stderr:115050K .......... .......... .......... .......... .......... 68% 11.4M 6s 2024-03-30T13:12:57.247 INFO:teuthology.orchestra.run.smithi043.stderr:115100K .......... .......... .......... .......... .......... 68% 10.5M 6s 2024-03-30T13:12:57.253 INFO:teuthology.orchestra.run.smithi043.stderr:115150K .......... .......... .......... .......... .......... 68% 8.83M 6s 2024-03-30T13:12:57.257 INFO:teuthology.orchestra.run.smithi043.stderr:115200K .......... .......... .......... .......... .......... 68% 10.2M 6s 2024-03-30T13:12:57.262 INFO:teuthology.orchestra.run.smithi043.stderr:115250K .......... .......... .......... .......... .......... 68% 10.1M 6s 2024-03-30T13:12:57.267 INFO:teuthology.orchestra.run.smithi043.stderr:115300K .......... .......... .......... .......... .......... 68% 9.45M 6s 2024-03-30T13:12:57.273 INFO:teuthology.orchestra.run.smithi043.stderr:115350K .......... .......... .......... .......... .......... 68% 8.65M 6s 2024-03-30T13:12:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:115400K .......... .......... .......... .......... .......... 68% 9.68M 6s 2024-03-30T13:12:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:115450K .......... .......... .......... .......... .......... 68% 10.2M 6s 2024-03-30T13:12:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:115500K .......... .......... .......... .......... .......... 68% 9.58M 6s 2024-03-30T13:12:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:115550K .......... .......... .......... .......... .......... 68% 8.44M 6s 2024-03-30T13:12:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:115600K .......... .......... .......... .......... .......... 68% 9.83M 6s 2024-03-30T13:12:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:115650K .......... .......... .......... .......... .......... 68% 10.2M 6s 2024-03-30T13:12:57.307 INFO:teuthology.orchestra.run.smithi043.stderr:115700K .......... .......... .......... .......... .......... 68% 12.2M 6s 2024-03-30T13:12:57.311 INFO:teuthology.orchestra.run.smithi043.stderr:115750K .......... .......... .......... .......... .......... 68% 11.6M 6s 2024-03-30T13:12:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:115800K .......... .......... .......... .......... .......... 68% 13.6M 6s 2024-03-30T13:12:57.320 INFO:teuthology.orchestra.run.smithi043.stderr:115850K .......... .......... .......... .......... .......... 68% 11.1M 6s 2024-03-30T13:12:57.324 INFO:teuthology.orchestra.run.smithi043.stderr:115900K .......... .......... .......... .......... .......... 68% 11.8M 6s 2024-03-30T13:12:57.329 INFO:teuthology.orchestra.run.smithi043.stderr:115950K .......... .......... .......... .......... .......... 68% 9.25M 6s 2024-03-30T13:12:57.334 INFO:teuthology.orchestra.run.smithi043.stderr:116000K .......... .......... .......... .......... .......... 68% 10.1M 6s 2024-03-30T13:12:57.339 INFO:teuthology.orchestra.run.smithi043.stderr:116050K .......... .......... .......... .......... .......... 68% 10.2M 6s 2024-03-30T13:12:57.344 INFO:teuthology.orchestra.run.smithi043.stderr:116100K .......... .......... .......... .......... .......... 68% 9.69M 6s 2024-03-30T13:12:57.349 INFO:teuthology.orchestra.run.smithi043.stderr:116150K .......... .......... .......... .......... .......... 68% 8.60M 6s 2024-03-30T13:12:57.354 INFO:teuthology.orchestra.run.smithi043.stderr:116200K .......... .......... .......... .......... .......... 68% 10.3M 6s 2024-03-30T13:12:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:116250K .......... .......... .......... .......... .......... 68% 10.2M 6s 2024-03-30T13:12:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:116300K .......... .......... .......... .......... .......... 69% 10.6M 6s 2024-03-30T13:12:57.369 INFO:teuthology.orchestra.run.smithi043.stderr:116350K .......... .......... .......... .......... .......... 69% 9.46M 6s 2024-03-30T13:12:57.373 INFO:teuthology.orchestra.run.smithi043.stderr:116400K .......... .......... .......... .......... .......... 69% 10.5M 6s 2024-03-30T13:12:57.378 INFO:teuthology.orchestra.run.smithi043.stderr:116450K .......... .......... .......... .......... .......... 69% 9.47M 6s 2024-03-30T13:12:57.384 INFO:teuthology.orchestra.run.smithi043.stderr:116500K .......... .......... .......... .......... .......... 69% 9.25M 6s 2024-03-30T13:12:57.390 INFO:teuthology.orchestra.run.smithi043.stderr:116550K .......... .......... .......... .......... .......... 69% 7.61M 6s 2024-03-30T13:12:57.396 INFO:teuthology.orchestra.run.smithi043.stderr:116600K .......... .......... .......... .......... .......... 69% 9.57M 6s 2024-03-30T13:12:57.401 INFO:teuthology.orchestra.run.smithi043.stderr:116650K .......... .......... .......... .......... .......... 69% 8.77M 6s 2024-03-30T13:12:57.407 INFO:teuthology.orchestra.run.smithi043.stderr:116700K .......... .......... .......... .......... .......... 69% 8.43M 6s 2024-03-30T13:12:57.413 INFO:teuthology.orchestra.run.smithi043.stderr:116750K .......... .......... .......... .......... .......... 69% 7.90M 6s 2024-03-30T13:12:57.418 INFO:teuthology.orchestra.run.smithi043.stderr:116800K .......... .......... .......... .......... .......... 69% 9.27M 6s 2024-03-30T13:12:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:116850K .......... .......... .......... .......... .......... 69% 9.05M 6s 2024-03-30T13:12:57.429 INFO:teuthology.orchestra.run.smithi043.stderr:116900K .......... .......... .......... .......... .......... 69% 9.26M 6s 2024-03-30T13:12:57.435 INFO:teuthology.orchestra.run.smithi043.stderr:116950K .......... .......... .......... .......... .......... 69% 7.46M 6s 2024-03-30T13:12:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:117000K .......... .......... .......... .......... .......... 69% 9.09M 6s 2024-03-30T13:12:57.446 INFO:teuthology.orchestra.run.smithi043.stderr:117050K .......... .......... .......... .......... .......... 69% 9.17M 6s 2024-03-30T13:12:57.452 INFO:teuthology.orchestra.run.smithi043.stderr:117100K .......... .......... .......... .......... .......... 69% 8.60M 6s 2024-03-30T13:12:57.458 INFO:teuthology.orchestra.run.smithi043.stderr:117150K .......... .......... .......... .......... .......... 69% 7.62M 6s 2024-03-30T13:12:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:117200K .......... .......... .......... .......... .......... 69% 8.70M 6s 2024-03-30T13:12:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:117250K .......... .......... .......... .......... .......... 69% 8.83M 6s 2024-03-30T13:12:57.475 INFO:teuthology.orchestra.run.smithi043.stderr:117300K .......... .......... .......... .......... .......... 69% 8.70M 6s 2024-03-30T13:12:57.482 INFO:teuthology.orchestra.run.smithi043.stderr:117350K .......... .......... .......... .......... .......... 69% 7.33M 6s 2024-03-30T13:12:57.487 INFO:teuthology.orchestra.run.smithi043.stderr:117400K .......... .......... .......... .......... .......... 69% 8.94M 6s 2024-03-30T13:12:57.493 INFO:teuthology.orchestra.run.smithi043.stderr:117450K .......... .......... .......... .......... .......... 69% 8.47M 6s 2024-03-30T13:12:57.499 INFO:teuthology.orchestra.run.smithi043.stderr:117500K .......... .......... .......... .......... .......... 69% 8.40M 6s 2024-03-30T13:12:57.505 INFO:teuthology.orchestra.run.smithi043.stderr:117550K .......... .......... .......... .......... .......... 69% 7.39M 6s 2024-03-30T13:12:57.512 INFO:teuthology.orchestra.run.smithi043.stderr:117600K .......... .......... .......... .......... .......... 69% 7.65M 6s 2024-03-30T13:12:57.518 INFO:teuthology.orchestra.run.smithi043.stderr:117650K .......... .......... .......... .......... .......... 69% 7.37M 6s 2024-03-30T13:12:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:117700K .......... .......... .......... .......... .......... 69% 7.29M 6s 2024-03-30T13:12:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:117750K .......... .......... .......... .......... .......... 69% 5.86M 6s 2024-03-30T13:12:57.540 INFO:teuthology.orchestra.run.smithi043.stderr:117800K .......... .......... .......... .......... .......... 69% 7.13M 6s 2024-03-30T13:12:57.547 INFO:teuthology.orchestra.run.smithi043.stderr:117850K .......... .......... .......... .......... .......... 69% 6.98M 6s 2024-03-30T13:12:57.555 INFO:teuthology.orchestra.run.smithi043.stderr:117900K .......... .......... .......... .......... .......... 69% 6.73M 6s 2024-03-30T13:12:57.563 INFO:teuthology.orchestra.run.smithi043.stderr:117950K .......... .......... .......... .......... .......... 70% 5.80M 6s 2024-03-30T13:12:57.570 INFO:teuthology.orchestra.run.smithi043.stderr:118000K .......... .......... .......... .......... .......... 70% 6.80M 6s 2024-03-30T13:12:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:118050K .......... .......... .......... .......... .......... 70% 6.77M 6s 2024-03-30T13:12:57.584 INFO:teuthology.orchestra.run.smithi043.stderr:118100K .......... .......... .......... .......... .......... 70% 7.13M 6s 2024-03-30T13:12:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:118150K .......... .......... .......... .......... .......... 70% 5.77M 6s 2024-03-30T13:12:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:118200K .......... .......... .......... .......... .......... 70% 7.23M 6s 2024-03-30T13:12:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:118250K .......... .......... .......... .......... .......... 70% 7.74M 6s 2024-03-30T13:12:57.612 INFO:teuthology.orchestra.run.smithi043.stderr:118300K .......... .......... .......... .......... .......... 70% 7.61M 6s 2024-03-30T13:12:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:118350K .......... .......... .......... .......... .......... 70% 6.90M 6s 2024-03-30T13:12:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:118400K .......... .......... .......... .......... .......... 70% 7.79M 6s 2024-03-30T13:12:57.633 INFO:teuthology.orchestra.run.smithi043.stderr:118450K .......... .......... .......... .......... .......... 70% 6.49M 6s 2024-03-30T13:12:57.642 INFO:teuthology.orchestra.run.smithi043.stderr:118500K .......... .......... .......... .......... .......... 70% 6.42M 6s 2024-03-30T13:12:57.650 INFO:teuthology.orchestra.run.smithi043.stderr:118550K .......... .......... .......... .......... .......... 70% 5.52M 6s 2024-03-30T13:12:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:118600K .......... .......... .......... .......... .......... 70% 6.51M 6s 2024-03-30T13:12:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:118650K .......... .......... .......... .......... .......... 70% 6.62M 6s 2024-03-30T13:12:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:118700K .......... .......... .......... .......... .......... 70% 6.27M 6s 2024-03-30T13:12:57.681 INFO:teuthology.orchestra.run.smithi043.stderr:118750K .......... .......... .......... .......... .......... 70% 5.55M 6s 2024-03-30T13:12:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:118800K .......... .......... .......... .......... .......... 70% 9.50M 6s 2024-03-30T13:12:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:118850K .......... .......... .......... .......... .......... 70% 10.0M 6s 2024-03-30T13:12:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:118900K .......... .......... .......... .......... .......... 70% 10.3M 6s 2024-03-30T13:12:57.702 INFO:teuthology.orchestra.run.smithi043.stderr:118950K .......... .......... .......... .......... .......... 70% 7.39M 6s 2024-03-30T13:12:57.707 INFO:teuthology.orchestra.run.smithi043.stderr:119000K .......... .......... .......... .......... .......... 70% 10.8M 6s 2024-03-30T13:12:57.713 INFO:teuthology.orchestra.run.smithi043.stderr:119050K .......... .......... .......... .......... .......... 70% 8.74M 6s 2024-03-30T13:12:57.717 INFO:teuthology.orchestra.run.smithi043.stderr:119100K .......... .......... .......... .......... .......... 70% 9.46M 6s 2024-03-30T13:12:57.724 INFO:teuthology.orchestra.run.smithi043.stderr:119150K .......... .......... .......... .......... .......... 70% 7.47M 6s 2024-03-30T13:12:57.729 INFO:teuthology.orchestra.run.smithi043.stderr:119200K .......... .......... .......... .......... .......... 70% 9.64M 6s 2024-03-30T13:12:57.734 INFO:teuthology.orchestra.run.smithi043.stderr:119250K .......... .......... .......... .......... .......... 70% 9.82M 6s 2024-03-30T13:12:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:119300K .......... .......... .......... .......... .......... 70% 9.76M 6s 2024-03-30T13:12:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:119350K .......... .......... .......... .......... .......... 70% 7.50M 6s 2024-03-30T13:12:57.751 INFO:teuthology.orchestra.run.smithi043.stderr:119400K .......... .......... .......... .......... .......... 70% 9.25M 6s 2024-03-30T13:12:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:119450K .......... .......... .......... .......... .......... 70% 9.87M 6s 2024-03-30T13:12:57.761 INFO:teuthology.orchestra.run.smithi043.stderr:119500K .......... .......... .......... .......... .......... 70% 9.44M 6s 2024-03-30T13:12:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:119550K .......... .......... .......... .......... .......... 70% 7.81M 6s 2024-03-30T13:12:57.773 INFO:teuthology.orchestra.run.smithi043.stderr:119600K .......... .......... .......... .......... .......... 70% 8.56M 6s 2024-03-30T13:12:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:119650K .......... .......... .......... .......... .......... 71% 8.74M 6s 2024-03-30T13:12:57.784 INFO:teuthology.orchestra.run.smithi043.stderr:119700K .......... .......... .......... .......... .......... 71% 9.40M 6s 2024-03-30T13:12:57.790 INFO:teuthology.orchestra.run.smithi043.stderr:119750K .......... .......... .......... .......... .......... 71% 7.71M 6s 2024-03-30T13:12:57.795 INFO:teuthology.orchestra.run.smithi043.stderr:119800K .......... .......... .......... .......... .......... 71% 9.91M 6s 2024-03-30T13:12:57.800 INFO:teuthology.orchestra.run.smithi043.stderr:119850K .......... .......... .......... .......... .......... 71% 9.14M 6s 2024-03-30T13:12:57.806 INFO:teuthology.orchestra.run.smithi043.stderr:119900K .......... .......... .......... .......... .......... 71% 9.10M 6s 2024-03-30T13:12:57.812 INFO:teuthology.orchestra.run.smithi043.stderr:119950K .......... .......... .......... .......... .......... 71% 7.29M 6s 2024-03-30T13:12:57.818 INFO:teuthology.orchestra.run.smithi043.stderr:120000K .......... .......... .......... .......... .......... 71% 8.61M 6s 2024-03-30T13:12:57.824 INFO:teuthology.orchestra.run.smithi043.stderr:120050K .......... .......... .......... .......... .......... 71% 8.66M 5s 2024-03-30T13:12:57.829 INFO:teuthology.orchestra.run.smithi043.stderr:120100K .......... .......... .......... .......... .......... 71% 8.61M 5s 2024-03-30T13:12:57.836 INFO:teuthology.orchestra.run.smithi043.stderr:120150K .......... .......... .......... .......... .......... 71% 7.31M 5s 2024-03-30T13:12:57.842 INFO:teuthology.orchestra.run.smithi043.stderr:120200K .......... .......... .......... .......... .......... 71% 8.63M 5s 2024-03-30T13:12:57.847 INFO:teuthology.orchestra.run.smithi043.stderr:120250K .......... .......... .......... .......... .......... 71% 8.56M 5s 2024-03-30T13:12:57.853 INFO:teuthology.orchestra.run.smithi043.stderr:120300K .......... .......... .......... .......... .......... 71% 8.50M 5s 2024-03-30T13:12:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:120350K .......... .......... .......... .......... .......... 71% 7.28M 5s 2024-03-30T13:12:57.866 INFO:teuthology.orchestra.run.smithi043.stderr:120400K .......... .......... .......... .......... .......... 71% 8.53M 5s 2024-03-30T13:12:57.871 INFO:teuthology.orchestra.run.smithi043.stderr:120450K .......... .......... .......... .......... .......... 71% 8.33M 5s 2024-03-30T13:12:57.877 INFO:teuthology.orchestra.run.smithi043.stderr:120500K .......... .......... .......... .......... .......... 71% 8.34M 5s 2024-03-30T13:12:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:120550K .......... .......... .......... .......... .......... 71% 7.41M 5s 2024-03-30T13:12:57.890 INFO:teuthology.orchestra.run.smithi043.stderr:120600K .......... .......... .......... .......... .......... 71% 8.54M 5s 2024-03-30T13:12:57.895 INFO:teuthology.orchestra.run.smithi043.stderr:120650K .......... .......... .......... .......... .......... 71% 8.68M 5s 2024-03-30T13:12:57.901 INFO:teuthology.orchestra.run.smithi043.stderr:120700K .......... .......... .......... .......... .......... 71% 8.83M 5s 2024-03-30T13:12:57.908 INFO:teuthology.orchestra.run.smithi043.stderr:120750K .......... .......... .......... .......... .......... 71% 6.76M 5s 2024-03-30T13:12:57.914 INFO:teuthology.orchestra.run.smithi043.stderr:120800K .......... .......... .......... .......... .......... 71% 7.83M 5s 2024-03-30T13:12:57.920 INFO:teuthology.orchestra.run.smithi043.stderr:120850K .......... .......... .......... .......... .......... 71% 8.15M 5s 2024-03-30T13:12:57.925 INFO:teuthology.orchestra.run.smithi043.stderr:120900K .......... .......... .......... .......... .......... 71% 9.24M 5s 2024-03-30T13:12:57.931 INFO:teuthology.orchestra.run.smithi043.stderr:120950K .......... .......... .......... .......... .......... 71% 8.06M 5s 2024-03-30T13:12:57.937 INFO:teuthology.orchestra.run.smithi043.stderr:121000K .......... .......... .......... .......... .......... 71% 8.87M 5s 2024-03-30T13:12:57.943 INFO:teuthology.orchestra.run.smithi043.stderr:121050K .......... .......... .......... .......... .......... 71% 8.61M 5s 2024-03-30T13:12:57.949 INFO:teuthology.orchestra.run.smithi043.stderr:121100K .......... .......... .......... .......... .......... 71% 8.16M 5s 2024-03-30T13:12:57.955 INFO:teuthology.orchestra.run.smithi043.stderr:121150K .......... .......... .......... .......... .......... 71% 7.33M 5s 2024-03-30T13:12:57.961 INFO:teuthology.orchestra.run.smithi043.stderr:121200K .......... .......... .......... .......... .......... 71% 8.32M 5s 2024-03-30T13:12:57.967 INFO:teuthology.orchestra.run.smithi043.stderr:121250K .......... .......... .......... .......... .......... 71% 8.29M 5s 2024-03-30T13:12:57.973 INFO:teuthology.orchestra.run.smithi043.stderr:121300K .......... .......... .......... .......... .......... 71% 8.36M 5s 2024-03-30T13:12:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:121350K .......... .......... .......... .......... .......... 72% 7.09M 5s 2024-03-30T13:12:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:121400K .......... .......... .......... .......... .......... 72% 8.79M 5s 2024-03-30T13:12:57.991 INFO:teuthology.orchestra.run.smithi043.stderr:121450K .......... .......... .......... .......... .......... 72% 8.73M 5s 2024-03-30T13:12:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:121500K .......... .......... .......... .......... .......... 72% 8.33M 5s 2024-03-30T13:12:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:121550K .......... .......... .......... .......... .......... 72% 7.04M 5s 2024-03-30T13:12:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:121600K .......... .......... .......... .......... .......... 72% 8.56M 5s 2024-03-30T13:12:58.015 INFO:teuthology.orchestra.run.smithi043.stderr:121650K .......... .......... .......... .......... .......... 72% 8.92M 5s 2024-03-30T13:12:58.020 INFO:teuthology.orchestra.run.smithi043.stderr:121700K .......... .......... .......... .......... .......... 72% 8.89M 5s 2024-03-30T13:12:58.027 INFO:teuthology.orchestra.run.smithi043.stderr:121750K .......... .......... .......... .......... .......... 72% 7.68M 5s 2024-03-30T13:12:58.032 INFO:teuthology.orchestra.run.smithi043.stderr:121800K .......... .......... .......... .......... .......... 72% 8.67M 5s 2024-03-30T13:12:58.038 INFO:teuthology.orchestra.run.smithi043.stderr:121850K .......... .......... .......... .......... .......... 72% 8.89M 5s 2024-03-30T13:12:58.044 INFO:teuthology.orchestra.run.smithi043.stderr:121900K .......... .......... .......... .......... .......... 72% 8.66M 5s 2024-03-30T13:12:58.051 INFO:teuthology.orchestra.run.smithi043.stderr:121950K .......... .......... .......... .......... .......... 72% 7.05M 5s 2024-03-30T13:12:58.057 INFO:teuthology.orchestra.run.smithi043.stderr:122000K .......... .......... .......... .......... .......... 72% 7.41M 5s 2024-03-30T13:12:58.064 INFO:teuthology.orchestra.run.smithi043.stderr:122050K .......... .......... .......... .......... .......... 72% 7.55M 5s 2024-03-30T13:12:58.070 INFO:teuthology.orchestra.run.smithi043.stderr:122100K .......... .......... .......... .......... .......... 72% 7.76M 5s 2024-03-30T13:12:58.078 INFO:teuthology.orchestra.run.smithi043.stderr:122150K .......... .......... .......... .......... .......... 72% 5.84M 5s 2024-03-30T13:12:58.085 INFO:teuthology.orchestra.run.smithi043.stderr:122200K .......... .......... .......... .......... .......... 72% 7.41M 5s 2024-03-30T13:12:58.092 INFO:teuthology.orchestra.run.smithi043.stderr:122250K .......... .......... .......... .......... .......... 72% 6.87M 5s 2024-03-30T13:12:58.099 INFO:teuthology.orchestra.run.smithi043.stderr:122300K .......... .......... .......... .......... .......... 72% 7.10M 5s 2024-03-30T13:12:58.107 INFO:teuthology.orchestra.run.smithi043.stderr:122350K .......... .......... .......... .......... .......... 72% 6.09M 5s 2024-03-30T13:12:58.114 INFO:teuthology.orchestra.run.smithi043.stderr:122400K .......... .......... .......... .......... .......... 72% 6.96M 5s 2024-03-30T13:12:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:122450K .......... .......... .......... .......... .......... 72% 6.87M 5s 2024-03-30T13:12:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:122500K .......... .......... .......... .......... .......... 72% 6.61M 5s 2024-03-30T13:12:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:122550K .......... .......... .......... .......... .......... 72% 5.76M 5s 2024-03-30T13:12:58.144 INFO:teuthology.orchestra.run.smithi043.stderr:122600K .......... .......... .......... .......... .......... 72% 7.35M 5s 2024-03-30T13:12:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:122650K .......... .......... .......... .......... .......... 72% 8.04M 5s 2024-03-30T13:12:58.156 INFO:teuthology.orchestra.run.smithi043.stderr:122700K .......... .......... .......... .......... .......... 72% 7.81M 5s 2024-03-30T13:12:58.163 INFO:teuthology.orchestra.run.smithi043.stderr:122750K .......... .......... .......... .......... .......... 72% 6.77M 5s 2024-03-30T13:12:58.169 INFO:teuthology.orchestra.run.smithi043.stderr:122800K .......... .......... .......... .......... .......... 72% 8.05M 5s 2024-03-30T13:12:58.175 INFO:teuthology.orchestra.run.smithi043.stderr:122850K .......... .......... .......... .......... .......... 72% 7.61M 5s 2024-03-30T13:12:58.179 INFO:teuthology.orchestra.run.smithi043.stderr:122900K .......... .......... .......... .......... .......... 72% 10.7M 5s 2024-03-30T13:12:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:122950K .......... .......... .......... .......... .......... 72% 11.4M 5s 2024-03-30T13:12:58.188 INFO:teuthology.orchestra.run.smithi043.stderr:123000K .......... .......... .......... .......... .......... 72% 11.3M 5s 2024-03-30T13:12:58.192 INFO:teuthology.orchestra.run.smithi043.stderr:123050K .......... .......... .......... .......... .......... 73% 11.8M 5s 2024-03-30T13:12:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:123100K .......... .......... .......... .......... .......... 73% 11.9M 5s 2024-03-30T13:12:58.202 INFO:teuthology.orchestra.run.smithi043.stderr:123150K .......... .......... .......... .......... .......... 73% 8.84M 5s 2024-03-30T13:12:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:123200K .......... .......... .......... .......... .......... 73% 9.88M 5s 2024-03-30T13:12:58.213 INFO:teuthology.orchestra.run.smithi043.stderr:123250K .......... .......... .......... .......... .......... 73% 9.14M 5s 2024-03-30T13:12:58.218 INFO:teuthology.orchestra.run.smithi043.stderr:123300K .......... .......... .......... .......... .......... 73% 9.68M 5s 2024-03-30T13:12:58.223 INFO:teuthology.orchestra.run.smithi043.stderr:123350K .......... .......... .......... .......... .......... 73% 8.38M 5s 2024-03-30T13:12:58.228 INFO:teuthology.orchestra.run.smithi043.stderr:123400K .......... .......... .......... .......... .......... 73% 9.94M 5s 2024-03-30T13:12:58.233 INFO:teuthology.orchestra.run.smithi043.stderr:123450K .......... .......... .......... .......... .......... 73% 9.94M 5s 2024-03-30T13:12:58.238 INFO:teuthology.orchestra.run.smithi043.stderr:123500K .......... .......... .......... .......... .......... 73% 9.77M 5s 2024-03-30T13:12:58.244 INFO:teuthology.orchestra.run.smithi043.stderr:123550K .......... .......... .......... .......... .......... 73% 8.58M 5s 2024-03-30T13:12:58.249 INFO:teuthology.orchestra.run.smithi043.stderr:123600K .......... .......... .......... .......... .......... 73% 9.84M 5s 2024-03-30T13:12:58.254 INFO:teuthology.orchestra.run.smithi043.stderr:123650K .......... .......... .......... .......... .......... 73% 9.60M 5s 2024-03-30T13:12:58.259 INFO:teuthology.orchestra.run.smithi043.stderr:123700K .......... .......... .......... .......... .......... 73% 9.52M 5s 2024-03-30T13:12:58.265 INFO:teuthology.orchestra.run.smithi043.stderr:123750K .......... .......... .......... .......... .......... 73% 8.33M 5s 2024-03-30T13:12:58.270 INFO:teuthology.orchestra.run.smithi043.stderr:123800K .......... .......... .......... .......... .......... 73% 9.43M 5s 2024-03-30T13:12:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:123850K .......... .......... .......... .......... .......... 73% 9.98M 5s 2024-03-30T13:12:58.281 INFO:teuthology.orchestra.run.smithi043.stderr:123900K .......... .......... .......... .......... .......... 73% 8.85M 5s 2024-03-30T13:12:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:123950K .......... .......... .......... .......... .......... 73% 8.95M 5s 2024-03-30T13:12:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:124000K .......... .......... .......... .......... .......... 73% 10.6M 5s 2024-03-30T13:12:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:124050K .......... .......... .......... .......... .......... 73% 10.4M 5s 2024-03-30T13:12:58.300 INFO:teuthology.orchestra.run.smithi043.stderr:124100K .......... .......... .......... .......... .......... 73% 9.85M 5s 2024-03-30T13:12:58.306 INFO:teuthology.orchestra.run.smithi043.stderr:124150K .......... .......... .......... .......... .......... 73% 8.71M 5s 2024-03-30T13:12:58.311 INFO:teuthology.orchestra.run.smithi043.stderr:124200K .......... .......... .......... .......... .......... 73% 10.5M 5s 2024-03-30T13:12:58.315 INFO:teuthology.orchestra.run.smithi043.stderr:124250K .......... .......... .......... .......... .......... 73% 10.2M 5s 2024-03-30T13:12:58.320 INFO:teuthology.orchestra.run.smithi043.stderr:124300K .......... .......... .......... .......... .......... 73% 10.8M 5s 2024-03-30T13:12:58.325 INFO:teuthology.orchestra.run.smithi043.stderr:124350K .......... .......... .......... .......... .......... 73% 9.24M 5s 2024-03-30T13:12:58.330 INFO:teuthology.orchestra.run.smithi043.stderr:124400K .......... .......... .......... .......... .......... 73% 10.6M 5s 2024-03-30T13:12:58.334 INFO:teuthology.orchestra.run.smithi043.stderr:124450K .......... .......... .......... .......... .......... 73% 10.2M 5s 2024-03-30T13:12:58.340 INFO:teuthology.orchestra.run.smithi043.stderr:124500K .......... .......... .......... .......... .......... 73% 9.84M 5s 2024-03-30T13:12:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:124550K .......... .......... .......... .......... .......... 73% 7.75M 5s 2024-03-30T13:12:58.352 INFO:teuthology.orchestra.run.smithi043.stderr:124600K .......... .......... .......... .......... .......... 73% 8.34M 5s 2024-03-30T13:12:58.358 INFO:teuthology.orchestra.run.smithi043.stderr:124650K .......... .......... .......... .......... .......... 73% 8.15M 5s 2024-03-30T13:12:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:124700K .......... .......... .......... .......... .......... 74% 8.02M 5s 2024-03-30T13:12:58.371 INFO:teuthology.orchestra.run.smithi043.stderr:124750K .......... .......... .......... .......... .......... 74% 6.78M 5s 2024-03-30T13:12:58.377 INFO:teuthology.orchestra.run.smithi043.stderr:124800K .......... .......... .......... .......... .......... 74% 7.95M 5s 2024-03-30T13:12:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:124850K .......... .......... .......... .......... .......... 74% 7.97M 5s 2024-03-30T13:12:58.390 INFO:teuthology.orchestra.run.smithi043.stderr:124900K .......... .......... .......... .......... .......... 74% 7.18M 5s 2024-03-30T13:12:58.397 INFO:teuthology.orchestra.run.smithi043.stderr:124950K .......... .......... .......... .......... .......... 74% 6.77M 5s 2024-03-30T13:12:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:125000K .......... .......... .......... .......... .......... 74% 7.67M 5s 2024-03-30T13:12:58.410 INFO:teuthology.orchestra.run.smithi043.stderr:125050K .......... .......... .......... .......... .......... 74% 7.33M 5s 2024-03-30T13:12:58.417 INFO:teuthology.orchestra.run.smithi043.stderr:125100K .......... .......... .......... .......... .......... 74% 7.74M 5s 2024-03-30T13:12:58.425 INFO:teuthology.orchestra.run.smithi043.stderr:125150K .......... .......... .......... .......... .......... 74% 6.00M 5s 2024-03-30T13:12:58.432 INFO:teuthology.orchestra.run.smithi043.stderr:125200K .......... .......... .......... .......... .......... 74% 7.04M 5s 2024-03-30T13:12:58.438 INFO:teuthology.orchestra.run.smithi043.stderr:125250K .......... .......... .......... .......... .......... 74% 7.42M 5s 2024-03-30T13:12:58.443 INFO:teuthology.orchestra.run.smithi043.stderr:125300K .......... .......... .......... .......... .......... 74% 9.38M 5s 2024-03-30T13:12:58.449 INFO:teuthology.orchestra.run.smithi043.stderr:125350K .......... .......... .......... .......... .......... 74% 8.19M 5s 2024-03-30T13:12:58.455 INFO:teuthology.orchestra.run.smithi043.stderr:125400K .......... .......... .......... .......... .......... 74% 8.79M 5s 2024-03-30T13:12:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:125450K .......... .......... .......... .......... .......... 74% 10.1M 5s 2024-03-30T13:12:58.465 INFO:teuthology.orchestra.run.smithi043.stderr:125500K .......... .......... .......... .......... .......... 74% 9.01M 5s 2024-03-30T13:12:58.471 INFO:teuthology.orchestra.run.smithi043.stderr:125550K .......... .......... .......... .......... .......... 74% 7.88M 5s 2024-03-30T13:12:58.476 INFO:teuthology.orchestra.run.smithi043.stderr:125600K .......... .......... .......... .......... .......... 74% 9.55M 5s 2024-03-30T13:12:58.482 INFO:teuthology.orchestra.run.smithi043.stderr:125650K .......... .......... .......... .......... .......... 74% 9.40M 5s 2024-03-30T13:12:58.487 INFO:teuthology.orchestra.run.smithi043.stderr:125700K .......... .......... .......... .......... .......... 74% 9.27M 5s 2024-03-30T13:12:58.493 INFO:teuthology.orchestra.run.smithi043.stderr:125750K .......... .......... .......... .......... .......... 74% 7.97M 5s 2024-03-30T13:12:58.498 INFO:teuthology.orchestra.run.smithi043.stderr:125800K .......... .......... .......... .......... .......... 74% 9.34M 5s 2024-03-30T13:12:58.504 INFO:teuthology.orchestra.run.smithi043.stderr:125850K .......... .......... .......... .......... .......... 74% 8.97M 5s 2024-03-30T13:12:58.509 INFO:teuthology.orchestra.run.smithi043.stderr:125900K .......... .......... .......... .......... .......... 74% 9.11M 5s 2024-03-30T13:12:58.516 INFO:teuthology.orchestra.run.smithi043.stderr:125950K .......... .......... .......... .......... .......... 74% 7.13M 5s 2024-03-30T13:12:58.522 INFO:teuthology.orchestra.run.smithi043.stderr:126000K .......... .......... .......... .......... .......... 74% 8.77M 5s 2024-03-30T13:12:58.527 INFO:teuthology.orchestra.run.smithi043.stderr:126050K .......... .......... .......... .......... .......... 74% 8.58M 5s 2024-03-30T13:12:58.533 INFO:teuthology.orchestra.run.smithi043.stderr:126100K .......... .......... .......... .......... .......... 74% 8.55M 5s 2024-03-30T13:12:58.540 INFO:teuthology.orchestra.run.smithi043.stderr:126150K .......... .......... .......... .......... .......... 74% 7.44M 5s 2024-03-30T13:12:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:126200K .......... .......... .......... .......... .......... 74% 8.46M 5s 2024-03-30T13:12:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:126250K .......... .......... .......... .......... .......... 74% 8.49M 5s 2024-03-30T13:12:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:126300K .......... .......... .......... .......... .......... 74% 8.81M 5s 2024-03-30T13:12:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:126350K .......... .......... .......... .......... .......... 74% 7.42M 5s 2024-03-30T13:12:58.569 INFO:teuthology.orchestra.run.smithi043.stderr:126400K .......... .......... .......... .......... .......... 75% 8.84M 5s 2024-03-30T13:12:58.575 INFO:teuthology.orchestra.run.smithi043.stderr:126450K .......... .......... .......... .......... .......... 75% 7.99M 5s 2024-03-30T13:12:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:126500K .......... .......... .......... .......... .......... 75% 7.93M 5s 2024-03-30T13:12:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:126550K .......... .......... .......... .......... .......... 75% 6.70M 5s 2024-03-30T13:12:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:126600K .......... .......... .......... .......... .......... 75% 8.37M 5s 2024-03-30T13:12:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:126650K .......... .......... .......... .......... .......... 75% 8.29M 5s 2024-03-30T13:12:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:126700K .......... .......... .......... .......... .......... 75% 8.91M 5s 2024-03-30T13:12:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:126750K .......... .......... .......... .......... .......... 75% 7.37M 5s 2024-03-30T13:12:58.618 INFO:teuthology.orchestra.run.smithi043.stderr:126800K .......... .......... .......... .......... .......... 75% 8.05M 5s 2024-03-30T13:12:58.624 INFO:teuthology.orchestra.run.smithi043.stderr:126850K .......... .......... .......... .......... .......... 75% 8.08M 5s 2024-03-30T13:12:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:126900K .......... .......... .......... .......... .......... 75% 8.36M 5s 2024-03-30T13:12:58.636 INFO:teuthology.orchestra.run.smithi043.stderr:126950K .......... .......... .......... .......... .......... 75% 7.59M 5s 2024-03-30T13:12:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:127000K .......... .......... .......... .......... .......... 75% 13.5M 5s 2024-03-30T13:12:58.644 INFO:teuthology.orchestra.run.smithi043.stderr:127050K .......... .......... .......... .......... .......... 75% 13.4M 5s 2024-03-30T13:12:58.647 INFO:teuthology.orchestra.run.smithi043.stderr:127100K .......... .......... .......... .......... .......... 75% 12.7M 5s 2024-03-30T13:12:58.652 INFO:teuthology.orchestra.run.smithi043.stderr:127150K .......... .......... .......... .......... .......... 75% 10.6M 5s 2024-03-30T13:12:58.656 INFO:teuthology.orchestra.run.smithi043.stderr:127200K .......... .......... .......... .......... .......... 75% 11.9M 5s 2024-03-30T13:12:58.661 INFO:teuthology.orchestra.run.smithi043.stderr:127250K .......... .......... .......... .......... .......... 75% 10.8M 5s 2024-03-30T13:12:58.666 INFO:teuthology.orchestra.run.smithi043.stderr:127300K .......... .......... .......... .......... .......... 75% 10.3M 5s 2024-03-30T13:12:58.671 INFO:teuthology.orchestra.run.smithi043.stderr:127350K .......... .......... .......... .......... .......... 75% 8.63M 5s 2024-03-30T13:12:58.676 INFO:teuthology.orchestra.run.smithi043.stderr:127400K .......... .......... .......... .......... .......... 75% 10.8M 5s 2024-03-30T13:12:58.680 INFO:teuthology.orchestra.run.smithi043.stderr:127450K .......... .......... .......... .......... .......... 75% 11.0M 5s 2024-03-30T13:12:58.685 INFO:teuthology.orchestra.run.smithi043.stderr:127500K .......... .......... .......... .......... .......... 75% 10.7M 5s 2024-03-30T13:12:58.690 INFO:teuthology.orchestra.run.smithi043.stderr:127550K .......... .......... .......... .......... .......... 75% 8.79M 5s 2024-03-30T13:12:58.695 INFO:teuthology.orchestra.run.smithi043.stderr:127600K .......... .......... .......... .......... .......... 75% 9.84M 5s 2024-03-30T13:12:58.700 INFO:teuthology.orchestra.run.smithi043.stderr:127650K .......... .......... .......... .......... .......... 75% 10.5M 5s 2024-03-30T13:12:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:127700K .......... .......... .......... .......... .......... 75% 9.91M 5s 2024-03-30T13:12:58.710 INFO:teuthology.orchestra.run.smithi043.stderr:127750K .......... .......... .......... .......... .......... 75% 8.67M 5s 2024-03-30T13:12:58.715 INFO:teuthology.orchestra.run.smithi043.stderr:127800K .......... .......... .......... .......... .......... 75% 10.8M 5s 2024-03-30T13:12:58.719 INFO:teuthology.orchestra.run.smithi043.stderr:127850K .......... .......... .......... .......... .......... 75% 11.2M 5s 2024-03-30T13:12:58.724 INFO:teuthology.orchestra.run.smithi043.stderr:127900K .......... .......... .......... .......... .......... 75% 11.0M 5s 2024-03-30T13:12:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:127950K .......... .......... .......... .......... .......... 75% 9.03M 5s 2024-03-30T13:12:58.734 INFO:teuthology.orchestra.run.smithi043.stderr:128000K .......... .......... .......... .......... .......... 75% 10.9M 5s 2024-03-30T13:12:58.738 INFO:teuthology.orchestra.run.smithi043.stderr:128050K .......... .......... .......... .......... .......... 75% 10.3M 5s 2024-03-30T13:12:58.743 INFO:teuthology.orchestra.run.smithi043.stderr:128100K .......... .......... .......... .......... .......... 76% 9.66M 5s 2024-03-30T13:12:58.749 INFO:teuthology.orchestra.run.smithi043.stderr:128150K .......... .......... .......... .......... .......... 76% 8.38M 5s 2024-03-30T13:12:58.755 INFO:teuthology.orchestra.run.smithi043.stderr:128200K .......... .......... .......... .......... .......... 76% 8.94M 5s 2024-03-30T13:12:58.760 INFO:teuthology.orchestra.run.smithi043.stderr:128250K .......... .......... .......... .......... .......... 76% 10.0M 5s 2024-03-30T13:12:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:128300K .......... .......... .......... .......... .......... 76% 9.63M 5s 2024-03-30T13:12:58.771 INFO:teuthology.orchestra.run.smithi043.stderr:128350K .......... .......... .......... .......... .......... 76% 7.75M 5s 2024-03-30T13:12:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:128400K .......... .......... .......... .......... .......... 76% 10.2M 5s 2024-03-30T13:12:58.781 INFO:teuthology.orchestra.run.smithi043.stderr:128450K .......... .......... .......... .......... .......... 76% 10.2M 5s 2024-03-30T13:12:58.786 INFO:teuthology.orchestra.run.smithi043.stderr:128500K .......... .......... .......... .......... .......... 76% 9.10M 5s 2024-03-30T13:12:58.792 INFO:teuthology.orchestra.run.smithi043.stderr:128550K .......... .......... .......... .......... .......... 76% 8.29M 5s 2024-03-30T13:12:58.797 INFO:teuthology.orchestra.run.smithi043.stderr:128600K .......... .......... .......... .......... .......... 76% 9.04M 5s 2024-03-30T13:12:58.802 INFO:teuthology.orchestra.run.smithi043.stderr:128650K .......... .......... .......... .......... .......... 76% 9.30M 5s 2024-03-30T13:12:58.808 INFO:teuthology.orchestra.run.smithi043.stderr:128700K .......... .......... .......... .......... .......... 76% 8.98M 5s 2024-03-30T13:12:58.815 INFO:teuthology.orchestra.run.smithi043.stderr:128750K .......... .......... .......... .......... .......... 76% 7.24M 5s 2024-03-30T13:12:58.820 INFO:teuthology.orchestra.run.smithi043.stderr:128800K .......... .......... .......... .......... .......... 76% 8.41M 5s 2024-03-30T13:12:58.826 INFO:teuthology.orchestra.run.smithi043.stderr:128850K .......... .......... .......... .......... .......... 76% 8.56M 4s 2024-03-30T13:12:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:128900K .......... .......... .......... .......... .......... 76% 8.57M 4s 2024-03-30T13:12:58.838 INFO:teuthology.orchestra.run.smithi043.stderr:128950K .......... .......... .......... .......... .......... 76% 7.34M 4s 2024-03-30T13:12:58.844 INFO:teuthology.orchestra.run.smithi043.stderr:129000K .......... .......... .......... .......... .......... 76% 8.75M 4s 2024-03-30T13:12:58.850 INFO:teuthology.orchestra.run.smithi043.stderr:129050K .......... .......... .......... .......... .......... 76% 8.75M 4s 2024-03-30T13:12:58.856 INFO:teuthology.orchestra.run.smithi043.stderr:129100K .......... .......... .......... .......... .......... 76% 8.64M 4s 2024-03-30T13:12:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:129150K .......... .......... .......... .......... .......... 76% 6.93M 4s 2024-03-30T13:12:58.869 INFO:teuthology.orchestra.run.smithi043.stderr:129200K .......... .......... .......... .......... .......... 76% 7.66M 4s 2024-03-30T13:12:58.875 INFO:teuthology.orchestra.run.smithi043.stderr:129250K .......... .......... .......... .......... .......... 76% 7.58M 4s 2024-03-30T13:12:58.881 INFO:teuthology.orchestra.run.smithi043.stderr:129300K .......... .......... .......... .......... .......... 76% 8.02M 4s 2024-03-30T13:12:58.889 INFO:teuthology.orchestra.run.smithi043.stderr:129350K .......... .......... .......... .......... .......... 76% 6.49M 4s 2024-03-30T13:12:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:129400K .......... .......... .......... .......... .......... 76% 7.55M 4s 2024-03-30T13:12:58.902 INFO:teuthology.orchestra.run.smithi043.stderr:129450K .......... .......... .......... .......... .......... 76% 7.83M 4s 2024-03-30T13:12:58.908 INFO:teuthology.orchestra.run.smithi043.stderr:129500K .......... .......... .......... .......... .......... 76% 7.50M 4s 2024-03-30T13:12:58.916 INFO:teuthology.orchestra.run.smithi043.stderr:129550K .......... .......... .......... .......... .......... 76% 6.49M 4s 2024-03-30T13:12:58.922 INFO:teuthology.orchestra.run.smithi043.stderr:129600K .......... .......... .......... .......... .......... 76% 7.42M 4s 2024-03-30T13:12:58.929 INFO:teuthology.orchestra.run.smithi043.stderr:129650K .......... .......... .......... .......... .......... 76% 7.12M 4s 2024-03-30T13:12:58.936 INFO:teuthology.orchestra.run.smithi043.stderr:129700K .......... .......... .......... .......... .......... 76% 7.24M 4s 2024-03-30T13:12:58.943 INFO:teuthology.orchestra.run.smithi043.stderr:129750K .......... .......... .......... .......... .......... 77% 6.59M 4s 2024-03-30T13:12:58.949 INFO:teuthology.orchestra.run.smithi043.stderr:129800K .......... .......... .......... .......... .......... 77% 7.86M 4s 2024-03-30T13:12:58.955 INFO:teuthology.orchestra.run.smithi043.stderr:129850K .......... .......... .......... .......... .......... 77% 8.37M 4s 2024-03-30T13:12:58.962 INFO:teuthology.orchestra.run.smithi043.stderr:129900K .......... .......... .......... .......... .......... 77% 7.38M 4s 2024-03-30T13:12:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:129950K .......... .......... .......... .......... .......... 77% 6.32M 4s 2024-03-30T13:12:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:130000K .......... .......... .......... .......... .......... 77% 7.54M 4s 2024-03-30T13:12:58.983 INFO:teuthology.orchestra.run.smithi043.stderr:130050K .......... .......... .......... .......... .......... 77% 7.47M 4s 2024-03-30T13:12:58.989 INFO:teuthology.orchestra.run.smithi043.stderr:130100K .......... .......... .......... .......... .......... 77% 7.46M 4s 2024-03-30T13:12:58.997 INFO:teuthology.orchestra.run.smithi043.stderr:130150K .......... .......... .......... .......... .......... 77% 5.98M 4s 2024-03-30T13:12:59.003 INFO:teuthology.orchestra.run.smithi043.stderr:130200K .......... .......... .......... .......... .......... 77% 7.94M 4s 2024-03-30T13:12:59.010 INFO:teuthology.orchestra.run.smithi043.stderr:130250K .......... .......... .......... .......... .......... 77% 7.45M 4s 2024-03-30T13:12:59.017 INFO:teuthology.orchestra.run.smithi043.stderr:130300K .......... .......... .......... .......... .......... 77% 7.09M 4s 2024-03-30T13:12:59.025 INFO:teuthology.orchestra.run.smithi043.stderr:130350K .......... .......... .......... .......... .......... 77% 6.28M 4s 2024-03-30T13:12:59.031 INFO:teuthology.orchestra.run.smithi043.stderr:130400K .......... .......... .......... .......... .......... 77% 8.00M 4s 2024-03-30T13:12:59.037 INFO:teuthology.orchestra.run.smithi043.stderr:130450K .......... .......... .......... .......... .......... 77% 8.14M 4s 2024-03-30T13:12:59.043 INFO:teuthology.orchestra.run.smithi043.stderr:130500K .......... .......... .......... .......... .......... 77% 7.69M 4s 2024-03-30T13:12:59.050 INFO:teuthology.orchestra.run.smithi043.stderr:130550K .......... .......... .......... .......... .......... 77% 6.74M 4s 2024-03-30T13:12:59.057 INFO:teuthology.orchestra.run.smithi043.stderr:130600K .......... .......... .......... .......... .......... 77% 7.78M 4s 2024-03-30T13:12:59.063 INFO:teuthology.orchestra.run.smithi043.stderr:130650K .......... .......... .......... .......... .......... 77% 7.58M 4s 2024-03-30T13:12:59.070 INFO:teuthology.orchestra.run.smithi043.stderr:130700K .......... .......... .......... .......... .......... 77% 7.61M 4s 2024-03-30T13:12:59.077 INFO:teuthology.orchestra.run.smithi043.stderr:130750K .......... .......... .......... .......... .......... 77% 6.39M 4s 2024-03-30T13:12:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:130800K .......... .......... .......... .......... .......... 77% 8.26M 4s 2024-03-30T13:12:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:130850K .......... .......... .......... .......... .......... 77% 8.18M 4s 2024-03-30T13:12:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:130900K .......... .......... .......... .......... .......... 77% 8.16M 4s 2024-03-30T13:12:59.103 INFO:teuthology.orchestra.run.smithi043.stderr:130950K .......... .......... .......... .......... .......... 77% 6.26M 4s 2024-03-30T13:12:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:131000K .......... .......... .......... .......... .......... 77% 6.75M 4s 2024-03-30T13:12:59.117 INFO:teuthology.orchestra.run.smithi043.stderr:131050K .......... .......... .......... .......... .......... 77% 7.12M 4s 2024-03-30T13:12:59.122 INFO:teuthology.orchestra.run.smithi043.stderr:131100K .......... .......... .......... .......... .......... 77% 10.1M 4s 2024-03-30T13:12:59.126 INFO:teuthology.orchestra.run.smithi043.stderr:131150K .......... .......... .......... .......... .......... 77% 9.84M 4s 2024-03-30T13:12:59.130 INFO:teuthology.orchestra.run.smithi043.stderr:131200K .......... .......... .......... .......... .......... 77% 12.4M 4s 2024-03-30T13:12:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:131250K .......... .......... .......... .......... .......... 77% 10.6M 4s 2024-03-30T13:12:59.140 INFO:teuthology.orchestra.run.smithi043.stderr:131300K .......... .......... .......... .......... .......... 77% 10.2M 4s 2024-03-30T13:12:59.146 INFO:teuthology.orchestra.run.smithi043.stderr:131350K .......... .......... .......... .......... .......... 77% 8.20M 4s 2024-03-30T13:12:59.151 INFO:teuthology.orchestra.run.smithi043.stderr:131400K .......... .......... .......... .......... .......... 77% 9.27M 4s 2024-03-30T13:12:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:131450K .......... .......... .......... .......... .......... 78% 10.4M 4s 2024-03-30T13:12:59.159 INFO:teuthology.orchestra.run.smithi043.stderr:131500K .......... .......... .......... .......... .......... 78% 12.6M 4s 2024-03-30T13:12:59.164 INFO:teuthology.orchestra.run.smithi043.stderr:131550K .......... .......... .......... .......... .......... 78% 9.85M 4s 2024-03-30T13:12:59.169 INFO:teuthology.orchestra.run.smithi043.stderr:131600K .......... .......... .......... .......... .......... 78% 10.2M 4s 2024-03-30T13:12:59.175 INFO:teuthology.orchestra.run.smithi043.stderr:131650K .......... .......... .......... .......... .......... 78% 8.43M 4s 2024-03-30T13:12:59.181 INFO:teuthology.orchestra.run.smithi043.stderr:131700K .......... .......... .......... .......... .......... 78% 8.46M 4s 2024-03-30T13:12:59.188 INFO:teuthology.orchestra.run.smithi043.stderr:131750K .......... .......... .......... .......... .......... 78% 7.03M 4s 2024-03-30T13:12:59.194 INFO:teuthology.orchestra.run.smithi043.stderr:131800K .......... .......... .......... .......... .......... 78% 8.30M 4s 2024-03-30T13:12:59.200 INFO:teuthology.orchestra.run.smithi043.stderr:131850K .......... .......... .......... .......... .......... 78% 8.55M 4s 2024-03-30T13:12:59.205 INFO:teuthology.orchestra.run.smithi043.stderr:131900K .......... .......... .......... .......... .......... 78% 8.48M 4s 2024-03-30T13:12:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:131950K .......... .......... .......... .......... .......... 78% 8.27M 4s 2024-03-30T13:12:59.215 INFO:teuthology.orchestra.run.smithi043.stderr:132000K .......... .......... .......... .......... .......... 78% 11.7M 4s 2024-03-30T13:12:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:132050K .......... .......... .......... .......... .......... 78% 12.3M 4s 2024-03-30T13:12:59.223 INFO:teuthology.orchestra.run.smithi043.stderr:132100K .......... .......... .......... .......... .......... 78% 12.3M 4s 2024-03-30T13:12:59.228 INFO:teuthology.orchestra.run.smithi043.stderr:132150K .......... .......... .......... .......... .......... 78% 10.0M 4s 2024-03-30T13:12:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:132200K .......... .......... .......... .......... .......... 78% 11.0M 4s 2024-03-30T13:12:59.237 INFO:teuthology.orchestra.run.smithi043.stderr:132250K .......... .......... .......... .......... .......... 78% 11.2M 4s 2024-03-30T13:12:59.242 INFO:teuthology.orchestra.run.smithi043.stderr:132300K .......... .......... .......... .......... .......... 78% 10.5M 4s 2024-03-30T13:12:59.247 INFO:teuthology.orchestra.run.smithi043.stderr:132350K .......... .......... .......... .......... .......... 78% 8.51M 4s 2024-03-30T13:12:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:132400K .......... .......... .......... .......... .......... 78% 10.6M 4s 2024-03-30T13:12:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:132450K .......... .......... .......... .......... .......... 78% 10.5M 4s 2024-03-30T13:12:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:132500K .......... .......... .......... .......... .......... 78% 10.5M 4s 2024-03-30T13:12:59.267 INFO:teuthology.orchestra.run.smithi043.stderr:132550K .......... .......... .......... .......... .......... 78% 9.05M 4s 2024-03-30T13:12:59.272 INFO:teuthology.orchestra.run.smithi043.stderr:132600K .......... .......... .......... .......... .......... 78% 8.53M 4s 2024-03-30T13:12:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:132650K .......... .......... .......... .......... .......... 78% 8.60M 4s 2024-03-30T13:12:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:132700K .......... .......... .......... .......... .......... 78% 8.02M 4s 2024-03-30T13:12:59.291 INFO:teuthology.orchestra.run.smithi043.stderr:132750K .......... .......... .......... .......... .......... 78% 7.11M 4s 2024-03-30T13:12:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:132800K .......... .......... .......... .......... .......... 78% 8.23M 4s 2024-03-30T13:12:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:132850K .......... .......... .......... .......... .......... 78% 8.36M 4s 2024-03-30T13:12:59.309 INFO:teuthology.orchestra.run.smithi043.stderr:132900K .......... .......... .......... .......... .......... 78% 8.19M 4s 2024-03-30T13:12:59.316 INFO:teuthology.orchestra.run.smithi043.stderr:132950K .......... .......... .......... .......... .......... 78% 7.34M 4s 2024-03-30T13:12:59.321 INFO:teuthology.orchestra.run.smithi043.stderr:133000K .......... .......... .......... .......... .......... 78% 8.23M 4s 2024-03-30T13:12:59.327 INFO:teuthology.orchestra.run.smithi043.stderr:133050K .......... .......... .......... .......... .......... 78% 7.97M 4s 2024-03-30T13:12:59.334 INFO:teuthology.orchestra.run.smithi043.stderr:133100K .......... .......... .......... .......... .......... 78% 7.89M 4s 2024-03-30T13:12:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:133150K .......... .......... .......... .......... .......... 79% 6.91M 4s 2024-03-30T13:12:59.347 INFO:teuthology.orchestra.run.smithi043.stderr:133200K .......... .......... .......... .......... .......... 79% 7.92M 4s 2024-03-30T13:12:59.353 INFO:teuthology.orchestra.run.smithi043.stderr:133250K .......... .......... .......... .......... .......... 79% 8.01M 4s 2024-03-30T13:12:59.359 INFO:teuthology.orchestra.run.smithi043.stderr:133300K .......... .......... .......... .......... .......... 79% 7.77M 4s 2024-03-30T13:12:59.366 INFO:teuthology.orchestra.run.smithi043.stderr:133350K .......... .......... .......... .......... .......... 79% 7.39M 4s 2024-03-30T13:12:59.372 INFO:teuthology.orchestra.run.smithi043.stderr:133400K .......... .......... .......... .......... .......... 79% 8.39M 4s 2024-03-30T13:12:59.378 INFO:teuthology.orchestra.run.smithi043.stderr:133450K .......... .......... .......... .......... .......... 79% 7.84M 4s 2024-03-30T13:12:59.384 INFO:teuthology.orchestra.run.smithi043.stderr:133500K .......... .......... .......... .......... .......... 79% 7.92M 4s 2024-03-30T13:12:59.391 INFO:teuthology.orchestra.run.smithi043.stderr:133550K .......... .......... .......... .......... .......... 79% 6.72M 4s 2024-03-30T13:12:59.397 INFO:teuthology.orchestra.run.smithi043.stderr:133600K .......... .......... .......... .......... .......... 79% 8.80M 4s 2024-03-30T13:12:59.402 INFO:teuthology.orchestra.run.smithi043.stderr:133650K .......... .......... .......... .......... .......... 79% 9.03M 4s 2024-03-30T13:12:59.408 INFO:teuthology.orchestra.run.smithi043.stderr:133700K .......... .......... .......... .......... .......... 79% 8.54M 4s 2024-03-30T13:12:59.415 INFO:teuthology.orchestra.run.smithi043.stderr:133750K .......... .......... .......... .......... .......... 79% 7.27M 4s 2024-03-30T13:12:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:133800K .......... .......... .......... .......... .......... 79% 8.31M 4s 2024-03-30T13:12:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:133850K .......... .......... .......... .......... .......... 79% 8.80M 4s 2024-03-30T13:12:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:133900K .......... .......... .......... .......... .......... 79% 8.59M 4s 2024-03-30T13:12:59.439 INFO:teuthology.orchestra.run.smithi043.stderr:133950K .......... .......... .......... .......... .......... 79% 7.27M 4s 2024-03-30T13:12:59.445 INFO:teuthology.orchestra.run.smithi043.stderr:134000K .......... .......... .......... .......... .......... 79% 8.17M 4s 2024-03-30T13:12:59.450 INFO:teuthology.orchestra.run.smithi043.stderr:134050K .......... .......... .......... .......... .......... 79% 8.65M 4s 2024-03-30T13:12:59.456 INFO:teuthology.orchestra.run.smithi043.stderr:134100K .......... .......... .......... .......... .......... 79% 7.97M 4s 2024-03-30T13:12:59.464 INFO:teuthology.orchestra.run.smithi043.stderr:134150K .......... .......... .......... .......... .......... 79% 6.47M 4s 2024-03-30T13:12:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:134200K .......... .......... .......... .......... .......... 79% 7.75M 4s 2024-03-30T13:12:59.477 INFO:teuthology.orchestra.run.smithi043.stderr:134250K .......... .......... .......... .......... .......... 79% 7.72M 4s 2024-03-30T13:12:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:134300K .......... .......... .......... .......... .......... 79% 8.58M 4s 2024-03-30T13:12:59.489 INFO:teuthology.orchestra.run.smithi043.stderr:134350K .......... .......... .......... .......... .......... 79% 7.26M 4s 2024-03-30T13:12:59.495 INFO:teuthology.orchestra.run.smithi043.stderr:134400K .......... .......... .......... .......... .......... 79% 8.62M 4s 2024-03-30T13:12:59.500 INFO:teuthology.orchestra.run.smithi043.stderr:134450K .......... .......... .......... .......... .......... 79% 8.60M 4s 2024-03-30T13:12:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:134500K .......... .......... .......... .......... .......... 79% 8.35M 4s 2024-03-30T13:12:59.513 INFO:teuthology.orchestra.run.smithi043.stderr:134550K .......... .......... .......... .......... .......... 79% 6.77M 4s 2024-03-30T13:12:59.520 INFO:teuthology.orchestra.run.smithi043.stderr:134600K .......... .......... .......... .......... .......... 79% 7.74M 4s 2024-03-30T13:12:59.527 INFO:teuthology.orchestra.run.smithi043.stderr:134650K .......... .......... .......... .......... .......... 79% 6.97M 4s 2024-03-30T13:12:59.534 INFO:teuthology.orchestra.run.smithi043.stderr:134700K .......... .......... .......... .......... .......... 79% 6.96M 4s 2024-03-30T13:12:59.542 INFO:teuthology.orchestra.run.smithi043.stderr:134750K .......... .......... .......... .......... .......... 79% 6.02M 4s 2024-03-30T13:12:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:134800K .......... .......... .......... .......... .......... 79% 7.27M 4s 2024-03-30T13:12:59.555 INFO:teuthology.orchestra.run.smithi043.stderr:134850K .......... .......... .......... .......... .......... 80% 7.14M 4s 2024-03-30T13:12:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:134900K .......... .......... .......... .......... .......... 80% 7.39M 4s 2024-03-30T13:12:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:134950K .......... .......... .......... .......... .......... 80% 5.62M 4s 2024-03-30T13:12:59.577 INFO:teuthology.orchestra.run.smithi043.stderr:135000K .......... .......... .......... .......... .......... 80% 8.11M 4s 2024-03-30T13:12:59.584 INFO:teuthology.orchestra.run.smithi043.stderr:135050K .......... .......... .......... .......... .......... 80% 7.07M 4s 2024-03-30T13:12:59.591 INFO:teuthology.orchestra.run.smithi043.stderr:135100K .......... .......... .......... .......... .......... 80% 7.20M 4s 2024-03-30T13:12:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:135150K .......... .......... .......... .......... .......... 80% 6.74M 4s 2024-03-30T13:12:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:135200K .......... .......... .......... .......... .......... 80% 10.3M 4s 2024-03-30T13:12:59.608 INFO:teuthology.orchestra.run.smithi043.stderr:135250K .......... .......... .......... .......... .......... 80% 9.76M 4s 2024-03-30T13:12:59.613 INFO:teuthology.orchestra.run.smithi043.stderr:135300K .......... .......... .......... .......... .......... 80% 8.19M 4s 2024-03-30T13:12:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:135350K .......... .......... .......... .......... .......... 80% 7.52M 4s 2024-03-30T13:12:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:135400K .......... .......... .......... .......... .......... 80% 10.6M 4s 2024-03-30T13:12:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:135450K .......... .......... .......... .......... .......... 80% 10.6M 4s 2024-03-30T13:12:59.633 INFO:teuthology.orchestra.run.smithi043.stderr:135500K .......... .......... .......... .......... .......... 80% 12.4M 4s 2024-03-30T13:12:59.638 INFO:teuthology.orchestra.run.smithi043.stderr:135550K .......... .......... .......... .......... .......... 80% 9.64M 4s 2024-03-30T13:12:59.642 INFO:teuthology.orchestra.run.smithi043.stderr:135600K .......... .......... .......... .......... .......... 80% 12.4M 4s 2024-03-30T13:12:59.646 INFO:teuthology.orchestra.run.smithi043.stderr:135650K .......... .......... .......... .......... .......... 80% 12.1M 4s 2024-03-30T13:12:59.650 INFO:teuthology.orchestra.run.smithi043.stderr:135700K .......... .......... .......... .......... .......... 80% 12.2M 4s 2024-03-30T13:12:59.655 INFO:teuthology.orchestra.run.smithi043.stderr:135750K .......... .......... .......... .......... .......... 80% 8.90M 4s 2024-03-30T13:12:59.660 INFO:teuthology.orchestra.run.smithi043.stderr:135800K .......... .......... .......... .......... .......... 80% 9.69M 4s 2024-03-30T13:12:59.666 INFO:teuthology.orchestra.run.smithi043.stderr:135850K .......... .......... .......... .......... .......... 80% 8.78M 4s 2024-03-30T13:12:59.672 INFO:teuthology.orchestra.run.smithi043.stderr:135900K .......... .......... .......... .......... .......... 80% 8.31M 4s 2024-03-30T13:12:59.678 INFO:teuthology.orchestra.run.smithi043.stderr:135950K .......... .......... .......... .......... .......... 80% 7.87M 4s 2024-03-30T13:12:59.682 INFO:teuthology.orchestra.run.smithi043.stderr:136000K .......... .......... .......... .......... .......... 80% 11.3M 4s 2024-03-30T13:12:59.689 INFO:teuthology.orchestra.run.smithi043.stderr:136050K .......... .......... .......... .......... .......... 80% 10.7M 4s 2024-03-30T13:12:59.691 INFO:teuthology.orchestra.run.smithi043.stderr:136100K .......... .......... .......... .......... .......... 80% 11.9M 4s 2024-03-30T13:12:59.696 INFO:teuthology.orchestra.run.smithi043.stderr:136150K .......... .......... .......... .......... .......... 80% 9.91M 4s 2024-03-30T13:12:59.701 INFO:teuthology.orchestra.run.smithi043.stderr:136200K .......... .......... .......... .......... .......... 80% 10.5M 4s 2024-03-30T13:12:59.705 INFO:teuthology.orchestra.run.smithi043.stderr:136250K .......... .......... .......... .......... .......... 80% 10.4M 4s 2024-03-30T13:12:59.710 INFO:teuthology.orchestra.run.smithi043.stderr:136300K .......... .......... .......... .......... .......... 80% 10.4M 4s 2024-03-30T13:12:59.715 INFO:teuthology.orchestra.run.smithi043.stderr:136350K .......... .......... .......... .......... .......... 80% 9.34M 4s 2024-03-30T13:12:59.720 INFO:teuthology.orchestra.run.smithi043.stderr:136400K .......... .......... .......... .......... .......... 80% 9.81M 4s 2024-03-30T13:12:59.725 INFO:teuthology.orchestra.run.smithi043.stderr:136450K .......... .......... .......... .......... .......... 80% 11.3M 4s 2024-03-30T13:12:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:136500K .......... .......... .......... .......... .......... 81% 10.2M 4s 2024-03-30T13:12:59.735 INFO:teuthology.orchestra.run.smithi043.stderr:136550K .......... .......... .......... .......... .......... 81% 8.81M 4s 2024-03-30T13:12:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:136600K .......... .......... .......... .......... .......... 81% 10.4M 4s 2024-03-30T13:12:59.744 INFO:teuthology.orchestra.run.smithi043.stderr:136650K .......... .......... .......... .......... .......... 81% 10.6M 4s 2024-03-30T13:12:59.749 INFO:teuthology.orchestra.run.smithi043.stderr:136700K .......... .......... .......... .......... .......... 81% 10.4M 4s 2024-03-30T13:12:59.755 INFO:teuthology.orchestra.run.smithi043.stderr:136750K .......... .......... .......... .......... .......... 81% 8.75M 4s 2024-03-30T13:12:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:136800K .......... .......... .......... .......... .......... 81% 9.89M 4s 2024-03-30T13:12:59.764 INFO:teuthology.orchestra.run.smithi043.stderr:136850K .......... .......... .......... .......... .......... 81% 11.3M 4s 2024-03-30T13:12:59.768 INFO:teuthology.orchestra.run.smithi043.stderr:136900K .......... .......... .......... .......... .......... 81% 11.8M 4s 2024-03-30T13:12:59.773 INFO:teuthology.orchestra.run.smithi043.stderr:136950K .......... .......... .......... .......... .......... 81% 10.1M 4s 2024-03-30T13:12:59.777 INFO:teuthology.orchestra.run.smithi043.stderr:137000K .......... .......... .......... .......... .......... 81% 11.5M 4s 2024-03-30T13:12:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:137050K .......... .......... .......... .......... .......... 81% 11.5M 4s 2024-03-30T13:12:59.785 INFO:teuthology.orchestra.run.smithi043.stderr:137100K .......... .......... .......... .......... .......... 81% 11.4M 4s 2024-03-30T13:12:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:137150K .......... .......... .......... .......... .......... 81% 8.85M 4s 2024-03-30T13:12:59.796 INFO:teuthology.orchestra.run.smithi043.stderr:137200K .......... .......... .......... .......... .......... 81% 9.32M 4s 2024-03-30T13:12:59.801 INFO:teuthology.orchestra.run.smithi043.stderr:137250K .......... .......... .......... .......... .......... 81% 9.80M 4s 2024-03-30T13:12:59.806 INFO:teuthology.orchestra.run.smithi043.stderr:137300K .......... .......... .......... .......... .......... 81% 9.74M 4s 2024-03-30T13:12:59.812 INFO:teuthology.orchestra.run.smithi043.stderr:137350K .......... .......... .......... .......... .......... 81% 8.80M 4s 2024-03-30T13:12:59.817 INFO:teuthology.orchestra.run.smithi043.stderr:137400K .......... .......... .......... .......... .......... 81% 9.89M 4s 2024-03-30T13:12:59.822 INFO:teuthology.orchestra.run.smithi043.stderr:137450K .......... .......... .......... .......... .......... 81% 9.81M 4s 2024-03-30T13:12:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:137500K .......... .......... .......... .......... .......... 81% 9.87M 4s 2024-03-30T13:12:59.833 INFO:teuthology.orchestra.run.smithi043.stderr:137550K .......... .......... .......... .......... .......... 81% 8.40M 4s 2024-03-30T13:12:59.838 INFO:teuthology.orchestra.run.smithi043.stderr:137600K .......... .......... .......... .......... .......... 81% 9.84M 4s 2024-03-30T13:12:59.843 INFO:teuthology.orchestra.run.smithi043.stderr:137650K .......... .......... .......... .......... .......... 81% 9.62M 4s 2024-03-30T13:12:59.848 INFO:teuthology.orchestra.run.smithi043.stderr:137700K .......... .......... .......... .......... .......... 81% 9.52M 3s 2024-03-30T13:12:59.854 INFO:teuthology.orchestra.run.smithi043.stderr:137750K .......... .......... .......... .......... .......... 81% 7.97M 3s 2024-03-30T13:12:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:137800K .......... .......... .......... .......... .......... 81% 9.55M 3s 2024-03-30T13:12:59.864 INFO:teuthology.orchestra.run.smithi043.stderr:137850K .......... .......... .......... .......... .......... 81% 9.59M 3s 2024-03-30T13:12:59.869 INFO:teuthology.orchestra.run.smithi043.stderr:137900K .......... .......... .......... .......... .......... 81% 9.56M 3s 2024-03-30T13:12:59.875 INFO:teuthology.orchestra.run.smithi043.stderr:137950K .......... .......... .......... .......... .......... 81% 8.35M 3s 2024-03-30T13:12:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:138000K .......... .......... .......... .......... .......... 81% 9.74M 3s 2024-03-30T13:12:59.885 INFO:teuthology.orchestra.run.smithi043.stderr:138050K .......... .......... .......... .......... .......... 81% 9.92M 3s 2024-03-30T13:12:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:138100K .......... .......... .......... .......... .......... 81% 8.90M 3s 2024-03-30T13:12:59.897 INFO:teuthology.orchestra.run.smithi043.stderr:138150K .......... .......... .......... .......... .......... 81% 7.61M 3s 2024-03-30T13:12:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:138200K .......... .......... .......... .......... .......... 82% 9.05M 3s 2024-03-30T13:12:59.907 INFO:teuthology.orchestra.run.smithi043.stderr:138250K .......... .......... .......... .......... .......... 82% 9.32M 3s 2024-03-30T13:12:59.913 INFO:teuthology.orchestra.run.smithi043.stderr:138300K .......... .......... .......... .......... .......... 82% 9.45M 3s 2024-03-30T13:12:59.919 INFO:teuthology.orchestra.run.smithi043.stderr:138350K .......... .......... .......... .......... .......... 82% 8.10M 3s 2024-03-30T13:12:59.924 INFO:teuthology.orchestra.run.smithi043.stderr:138400K .......... .......... .......... .......... .......... 82% 9.27M 3s 2024-03-30T13:12:59.929 INFO:teuthology.orchestra.run.smithi043.stderr:138450K .......... .......... .......... .......... .......... 82% 9.14M 3s 2024-03-30T13:12:59.935 INFO:teuthology.orchestra.run.smithi043.stderr:138500K .......... .......... .......... .......... .......... 82% 9.22M 3s 2024-03-30T13:12:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:138550K .......... .......... .......... .......... .......... 82% 7.87M 3s 2024-03-30T13:12:59.946 INFO:teuthology.orchestra.run.smithi043.stderr:138600K .......... .......... .......... .......... .......... 82% 8.97M 3s 2024-03-30T13:12:59.951 INFO:teuthology.orchestra.run.smithi043.stderr:138650K .......... .......... .......... .......... .......... 82% 9.02M 3s 2024-03-30T13:12:59.957 INFO:teuthology.orchestra.run.smithi043.stderr:138700K .......... .......... .......... .......... .......... 82% 8.68M 3s 2024-03-30T13:12:59.964 INFO:teuthology.orchestra.run.smithi043.stderr:138750K .......... .......... .......... .......... .......... 82% 7.17M 3s 2024-03-30T13:12:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:138800K .......... .......... .......... .......... .......... 82% 8.09M 3s 2024-03-30T13:12:59.976 INFO:teuthology.orchestra.run.smithi043.stderr:138850K .......... .......... .......... .......... .......... 82% 7.74M 3s 2024-03-30T13:12:59.983 INFO:teuthology.orchestra.run.smithi043.stderr:138900K .......... .......... .......... .......... .......... 82% 7.62M 3s 2024-03-30T13:12:59.990 INFO:teuthology.orchestra.run.smithi043.stderr:138950K .......... .......... .......... .......... .......... 82% 6.59M 3s 2024-03-30T13:12:59.997 INFO:teuthology.orchestra.run.smithi043.stderr:139000K .......... .......... .......... .......... .......... 82% 7.40M 3s 2024-03-30T13:13:00.004 INFO:teuthology.orchestra.run.smithi043.stderr:139050K .......... .......... .......... .......... .......... 82% 7.21M 3s 2024-03-30T13:13:00.010 INFO:teuthology.orchestra.run.smithi043.stderr:139100K .......... .......... .......... .......... .......... 82% 8.31M 3s 2024-03-30T13:13:00.022 INFO:teuthology.orchestra.run.smithi043.stderr:139150K .......... .......... .......... .......... .......... 82% 6.84M 3s 2024-03-30T13:13:00.023 INFO:teuthology.orchestra.run.smithi043.stderr:139200K .......... .......... .......... .......... .......... 82% 8.08M 3s 2024-03-30T13:13:00.029 INFO:teuthology.orchestra.run.smithi043.stderr:139250K .......... .......... .......... .......... .......... 82% 6.94M 3s 2024-03-30T13:13:00.034 INFO:teuthology.orchestra.run.smithi043.stderr:139300K .......... .......... .......... .......... .......... 82% 9.93M 3s 2024-03-30T13:13:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:139350K .......... .......... .......... .......... .......... 82% 9.67M 3s 2024-03-30T13:13:00.043 INFO:teuthology.orchestra.run.smithi043.stderr:139400K .......... .......... .......... .......... .......... 82% 12.2M 3s 2024-03-30T13:13:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:139450K .......... .......... .......... .......... .......... 82% 12.5M 3s 2024-03-30T13:13:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:139500K .......... .......... .......... .......... .......... 82% 13.4M 3s 2024-03-30T13:13:00.054 INFO:teuthology.orchestra.run.smithi043.stderr:139550K .......... .......... .......... .......... .......... 82% 13.1M 3s 2024-03-30T13:13:00.058 INFO:teuthology.orchestra.run.smithi043.stderr:139600K .......... .......... .......... .......... .......... 82% 15.4M 3s 2024-03-30T13:13:00.061 INFO:teuthology.orchestra.run.smithi043.stderr:139650K .......... .......... .......... .......... .......... 82% 15.4M 3s 2024-03-30T13:13:00.064 INFO:teuthology.orchestra.run.smithi043.stderr:139700K .......... .......... .......... .......... .......... 82% 15.2M 3s 2024-03-30T13:13:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:139750K .......... .......... .......... .......... .......... 82% 12.6M 3s 2024-03-30T13:13:00.073 INFO:teuthology.orchestra.run.smithi043.stderr:139800K .......... .......... .......... .......... .......... 82% 9.84M 3s 2024-03-30T13:13:00.078 INFO:teuthology.orchestra.run.smithi043.stderr:139850K .......... .......... .......... .......... .......... 82% 11.0M 3s 2024-03-30T13:13:00.082 INFO:teuthology.orchestra.run.smithi043.stderr:139900K .......... .......... .......... .......... .......... 83% 9.95M 3s 2024-03-30T13:13:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:139950K .......... .......... .......... .......... .......... 83% 9.73M 3s 2024-03-30T13:13:00.092 INFO:teuthology.orchestra.run.smithi043.stderr:140000K .......... .......... .......... .......... .......... 83% 10.5M 3s 2024-03-30T13:13:00.097 INFO:teuthology.orchestra.run.smithi043.stderr:140050K .......... .......... .......... .......... .......... 83% 9.95M 3s 2024-03-30T13:13:00.102 INFO:teuthology.orchestra.run.smithi043.stderr:140100K .......... .......... .......... .......... .......... 83% 10.3M 3s 2024-03-30T13:13:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:140150K .......... .......... .......... .......... .......... 83% 8.88M 3s 2024-03-30T13:13:00.111 INFO:teuthology.orchestra.run.smithi043.stderr:140200K .......... .......... .......... .......... .......... 83% 11.3M 3s 2024-03-30T13:13:00.116 INFO:teuthology.orchestra.run.smithi043.stderr:140250K .......... .......... .......... .......... .......... 83% 11.8M 3s 2024-03-30T13:13:00.120 INFO:teuthology.orchestra.run.smithi043.stderr:140300K .......... .......... .......... .......... .......... 83% 11.7M 3s 2024-03-30T13:13:00.125 INFO:teuthology.orchestra.run.smithi043.stderr:140350K .......... .......... .......... .......... .......... 83% 10.2M 3s 2024-03-30T13:13:00.129 INFO:teuthology.orchestra.run.smithi043.stderr:140400K .......... .......... .......... .......... .......... 83% 10.7M 3s 2024-03-30T13:13:00.134 INFO:teuthology.orchestra.run.smithi043.stderr:140450K .......... .......... .......... .......... .......... 83% 10.4M 3s 2024-03-30T13:13:00.140 INFO:teuthology.orchestra.run.smithi043.stderr:140500K .......... .......... .......... .......... .......... 83% 8.34M 3s 2024-03-30T13:13:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:140550K .......... .......... .......... .......... .......... 83% 7.95M 3s 2024-03-30T13:13:00.150 INFO:teuthology.orchestra.run.smithi043.stderr:140600K .......... .......... .......... .......... .......... 83% 11.1M 3s 2024-03-30T13:13:00.155 INFO:teuthology.orchestra.run.smithi043.stderr:140650K .......... .......... .......... .......... .......... 83% 10.8M 3s 2024-03-30T13:13:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:140700K .......... .......... .......... .......... .......... 83% 11.0M 3s 2024-03-30T13:13:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:140750K .......... .......... .......... .......... .......... 83% 8.66M 3s 2024-03-30T13:13:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:140800K .......... .......... .......... .......... .......... 83% 9.80M 3s 2024-03-30T13:13:00.175 INFO:teuthology.orchestra.run.smithi043.stderr:140850K .......... .......... .......... .......... .......... 83% 9.41M 3s 2024-03-30T13:13:00.180 INFO:teuthology.orchestra.run.smithi043.stderr:140900K .......... .......... .......... .......... .......... 83% 10.7M 3s 2024-03-30T13:13:00.185 INFO:teuthology.orchestra.run.smithi043.stderr:140950K .......... .......... .......... .......... .......... 83% 9.06M 3s 2024-03-30T13:13:00.190 INFO:teuthology.orchestra.run.smithi043.stderr:141000K .......... .......... .......... .......... .......... 83% 9.34M 3s 2024-03-30T13:13:00.195 INFO:teuthology.orchestra.run.smithi043.stderr:141050K .......... .......... .......... .......... .......... 83% 10.1M 3s 2024-03-30T13:13:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:141100K .......... .......... .......... .......... .......... 83% 9.90M 3s 2024-03-30T13:13:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:141150K .......... .......... .......... .......... .......... 83% 7.99M 3s 2024-03-30T13:13:00.211 INFO:teuthology.orchestra.run.smithi043.stderr:141200K .......... .......... .......... .......... .......... 83% 9.54M 3s 2024-03-30T13:13:00.217 INFO:teuthology.orchestra.run.smithi043.stderr:141250K .......... .......... .......... .......... .......... 83% 9.21M 3s 2024-03-30T13:13:00.221 INFO:teuthology.orchestra.run.smithi043.stderr:141300K .......... .......... .......... .......... .......... 83% 9.92M 3s 2024-03-30T13:13:00.227 INFO:teuthology.orchestra.run.smithi043.stderr:141350K .......... .......... .......... .......... .......... 83% 8.33M 3s 2024-03-30T13:13:00.233 INFO:teuthology.orchestra.run.smithi043.stderr:141400K .......... .......... .......... .......... .......... 83% 8.61M 3s 2024-03-30T13:13:00.238 INFO:teuthology.orchestra.run.smithi043.stderr:141450K .......... .......... .......... .......... .......... 83% 10.2M 3s 2024-03-30T13:13:00.243 INFO:teuthology.orchestra.run.smithi043.stderr:141500K .......... .......... .......... .......... .......... 83% 10.4M 3s 2024-03-30T13:13:00.248 INFO:teuthology.orchestra.run.smithi043.stderr:141550K .......... .......... .......... .......... .......... 84% 8.97M 3s 2024-03-30T13:13:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:141600K .......... .......... .......... .......... .......... 84% 10.1M 3s 2024-03-30T13:13:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:141650K .......... .......... .......... .......... .......... 84% 9.66M 3s 2024-03-30T13:13:00.263 INFO:teuthology.orchestra.run.smithi043.stderr:141700K .......... .......... .......... .......... .......... 84% 10.2M 3s 2024-03-30T13:13:00.268 INFO:teuthology.orchestra.run.smithi043.stderr:141750K .......... .......... .......... .......... .......... 84% 8.47M 3s 2024-03-30T13:13:00.273 INFO:teuthology.orchestra.run.smithi043.stderr:141800K .......... .......... .......... .......... .......... 84% 10.3M 3s 2024-03-30T13:13:00.278 INFO:teuthology.orchestra.run.smithi043.stderr:141850K .......... .......... .......... .......... .......... 84% 10.1M 3s 2024-03-30T13:13:00.283 INFO:teuthology.orchestra.run.smithi043.stderr:141900K .......... .......... .......... .......... .......... 84% 9.44M 3s 2024-03-30T13:13:00.290 INFO:teuthology.orchestra.run.smithi043.stderr:141950K .......... .......... .......... .......... .......... 84% 7.78M 3s 2024-03-30T13:13:00.295 INFO:teuthology.orchestra.run.smithi043.stderr:142000K .......... .......... .......... .......... .......... 84% 9.05M 3s 2024-03-30T13:13:00.302 INFO:teuthology.orchestra.run.smithi043.stderr:142050K .......... .......... .......... .......... .......... 84% 9.20M 3s 2024-03-30T13:13:00.306 INFO:teuthology.orchestra.run.smithi043.stderr:142100K .......... .......... .......... .......... .......... 84% 9.18M 3s 2024-03-30T13:13:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:142150K .......... .......... .......... .......... .......... 84% 8.37M 3s 2024-03-30T13:13:00.318 INFO:teuthology.orchestra.run.smithi043.stderr:142200K .......... .......... .......... .......... .......... 84% 9.28M 3s 2024-03-30T13:13:00.322 INFO:teuthology.orchestra.run.smithi043.stderr:142250K .......... .......... .......... .......... .......... 84% 9.20M 3s 2024-03-30T13:13:00.328 INFO:teuthology.orchestra.run.smithi043.stderr:142300K .......... .......... .......... .......... .......... 84% 8.94M 3s 2024-03-30T13:13:00.334 INFO:teuthology.orchestra.run.smithi043.stderr:142350K .......... .......... .......... .......... .......... 84% 7.24M 3s 2024-03-30T13:13:00.340 INFO:teuthology.orchestra.run.smithi043.stderr:142400K .......... .......... .......... .......... .......... 84% 8.10M 3s 2024-03-30T13:13:00.346 INFO:teuthology.orchestra.run.smithi043.stderr:142450K .......... .......... .......... .......... .......... 84% 8.53M 3s 2024-03-30T13:13:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:142500K .......... .......... .......... .......... .......... 84% 7.58M 3s 2024-03-30T13:13:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:142550K .......... .......... .......... .......... .......... 84% 7.51M 3s 2024-03-30T13:13:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:142600K .......... .......... .......... .......... .......... 84% 9.38M 3s 2024-03-30T13:13:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:142650K .......... .......... .......... .......... .......... 84% 9.00M 3s 2024-03-30T13:13:00.375 INFO:teuthology.orchestra.run.smithi043.stderr:142700K .......... .......... .......... .......... .......... 84% 9.40M 3s 2024-03-30T13:13:00.381 INFO:teuthology.orchestra.run.smithi043.stderr:142750K .......... .......... .......... .......... .......... 84% 7.84M 3s 2024-03-30T13:13:00.387 INFO:teuthology.orchestra.run.smithi043.stderr:142800K .......... .......... .......... .......... .......... 84% 8.58M 3s 2024-03-30T13:13:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:142850K .......... .......... .......... .......... .......... 84% 8.69M 3s 2024-03-30T13:13:00.398 INFO:teuthology.orchestra.run.smithi043.stderr:142900K .......... .......... .......... .......... .......... 84% 9.17M 3s 2024-03-30T13:13:00.404 INFO:teuthology.orchestra.run.smithi043.stderr:142950K .......... .......... .......... .......... .......... 84% 7.82M 3s 2024-03-30T13:13:00.410 INFO:teuthology.orchestra.run.smithi043.stderr:143000K .......... .......... .......... .......... .......... 84% 8.03M 3s 2024-03-30T13:13:00.417 INFO:teuthology.orchestra.run.smithi043.stderr:143050K .......... .......... .......... .......... .......... 84% 7.26M 3s 2024-03-30T13:13:00.425 INFO:teuthology.orchestra.run.smithi043.stderr:143100K .......... .......... .......... .......... .......... 84% 6.52M 3s 2024-03-30T13:13:00.433 INFO:teuthology.orchestra.run.smithi043.stderr:143150K .......... .......... .......... .......... .......... 84% 5.55M 3s 2024-03-30T13:13:00.441 INFO:teuthology.orchestra.run.smithi043.stderr:143200K .......... .......... .......... .......... .......... 84% 6.31M 3s 2024-03-30T13:13:00.449 INFO:teuthology.orchestra.run.smithi043.stderr:143250K .......... .......... .......... .......... .......... 85% 6.29M 3s 2024-03-30T13:13:00.457 INFO:teuthology.orchestra.run.smithi043.stderr:143300K .......... .......... .......... .......... .......... 85% 6.29M 3s 2024-03-30T13:13:00.464 INFO:teuthology.orchestra.run.smithi043.stderr:143350K .......... .......... .......... .......... .......... 85% 6.36M 3s 2024-03-30T13:13:00.469 INFO:teuthology.orchestra.run.smithi043.stderr:143400K .......... .......... .......... .......... .......... 85% 9.74M 3s 2024-03-30T13:13:00.474 INFO:teuthology.orchestra.run.smithi043.stderr:143450K .......... .......... .......... .......... .......... 85% 9.13M 3s 2024-03-30T13:13:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:143500K .......... .......... .......... .......... .......... 85% 5.99M 3s 2024-03-30T13:13:00.487 INFO:teuthology.orchestra.run.smithi043.stderr:143550K .......... .......... .......... .......... .......... 85% 10.5M 3s 2024-03-30T13:13:00.490 INFO:teuthology.orchestra.run.smithi043.stderr:143600K .......... .......... .......... .......... .......... 85% 13.7M 3s 2024-03-30T13:13:00.495 INFO:teuthology.orchestra.run.smithi043.stderr:143650K .......... .......... .......... .......... .......... 85% 11.1M 3s 2024-03-30T13:13:00.499 INFO:teuthology.orchestra.run.smithi043.stderr:143700K .......... .......... .......... .......... .......... 85% 12.5M 3s 2024-03-30T13:13:00.506 INFO:teuthology.orchestra.run.smithi043.stderr:143750K .......... .......... .......... .......... .......... 85% 10.0M 3s 2024-03-30T13:13:00.508 INFO:teuthology.orchestra.run.smithi043.stderr:143800K .......... .......... .......... .......... .......... 85% 10.8M 3s 2024-03-30T13:13:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:143850K .......... .......... .......... .......... .......... 85% 11.5M 3s 2024-03-30T13:13:00.516 INFO:teuthology.orchestra.run.smithi043.stderr:143900K .......... .......... .......... .......... .......... 85% 12.1M 3s 2024-03-30T13:13:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:143950K .......... .......... .......... .......... .......... 85% 10.6M 3s 2024-03-30T13:13:00.525 INFO:teuthology.orchestra.run.smithi043.stderr:144000K .......... .......... .......... .......... .......... 85% 11.1M 3s 2024-03-30T13:13:00.530 INFO:teuthology.orchestra.run.smithi043.stderr:144050K .......... .......... .......... .......... .......... 85% 9.93M 3s 2024-03-30T13:13:00.535 INFO:teuthology.orchestra.run.smithi043.stderr:144100K .......... .......... .......... .......... .......... 85% 9.67M 3s 2024-03-30T13:13:00.541 INFO:teuthology.orchestra.run.smithi043.stderr:144150K .......... .......... .......... .......... .......... 85% 8.07M 3s 2024-03-30T13:13:00.546 INFO:teuthology.orchestra.run.smithi043.stderr:144200K .......... .......... .......... .......... .......... 85% 9.89M 3s 2024-03-30T13:13:00.551 INFO:teuthology.orchestra.run.smithi043.stderr:144250K .......... .......... .......... .......... .......... 85% 9.90M 3s 2024-03-30T13:13:00.556 INFO:teuthology.orchestra.run.smithi043.stderr:144300K .......... .......... .......... .......... .......... 85% 10.4M 3s 2024-03-30T13:13:00.561 INFO:teuthology.orchestra.run.smithi043.stderr:144350K .......... .......... .......... .......... .......... 85% 8.97M 3s 2024-03-30T13:13:00.566 INFO:teuthology.orchestra.run.smithi043.stderr:144400K .......... .......... .......... .......... .......... 85% 11.1M 3s 2024-03-30T13:13:00.571 INFO:teuthology.orchestra.run.smithi043.stderr:144450K .......... .......... .......... .......... .......... 85% 9.37M 3s 2024-03-30T13:13:00.576 INFO:teuthology.orchestra.run.smithi043.stderr:144500K .......... .......... .......... .......... .......... 85% 9.69M 3s 2024-03-30T13:13:00.581 INFO:teuthology.orchestra.run.smithi043.stderr:144550K .......... .......... .......... .......... .......... 85% 9.19M 3s 2024-03-30T13:13:00.586 INFO:teuthology.orchestra.run.smithi043.stderr:144600K .......... .......... .......... .......... .......... 85% 9.93M 3s 2024-03-30T13:13:00.591 INFO:teuthology.orchestra.run.smithi043.stderr:144650K .......... .......... .......... .......... .......... 85% 10.2M 3s 2024-03-30T13:13:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:144700K .......... .......... .......... .......... .......... 85% 10.5M 3s 2024-03-30T13:13:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:144750K .......... .......... .......... .......... .......... 85% 8.73M 3s 2024-03-30T13:13:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:144800K .......... .......... .......... .......... .......... 85% 8.83M 3s 2024-03-30T13:13:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:144850K .......... .......... .......... .......... .......... 85% 8.47M 3s 2024-03-30T13:13:00.619 INFO:teuthology.orchestra.run.smithi043.stderr:144900K .......... .......... .......... .......... .......... 85% 8.35M 3s 2024-03-30T13:13:00.626 INFO:teuthology.orchestra.run.smithi043.stderr:144950K .......... .......... .......... .......... .......... 86% 6.97M 3s 2024-03-30T13:13:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:145000K .......... .......... .......... .......... .......... 86% 7.86M 3s 2024-03-30T13:13:00.638 INFO:teuthology.orchestra.run.smithi043.stderr:145050K .......... .......... .......... .......... .......... 86% 7.99M 3s 2024-03-30T13:13:00.644 INFO:teuthology.orchestra.run.smithi043.stderr:145100K .......... .......... .......... .......... .......... 86% 7.92M 3s 2024-03-30T13:13:00.652 INFO:teuthology.orchestra.run.smithi043.stderr:145150K .......... .......... .......... .......... .......... 86% 6.82M 3s 2024-03-30T13:13:00.657 INFO:teuthology.orchestra.run.smithi043.stderr:145200K .......... .......... .......... .......... .......... 86% 8.30M 3s 2024-03-30T13:13:00.663 INFO:teuthology.orchestra.run.smithi043.stderr:145250K .......... .......... .......... .......... .......... 86% 8.46M 3s 2024-03-30T13:13:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:145300K .......... .......... .......... .......... .......... 86% 8.28M 3s 2024-03-30T13:13:00.676 INFO:teuthology.orchestra.run.smithi043.stderr:145350K .......... .......... .......... .......... .......... 86% 7.11M 3s 2024-03-30T13:13:00.682 INFO:teuthology.orchestra.run.smithi043.stderr:145400K .......... .......... .......... .......... .......... 86% 8.24M 3s 2024-03-30T13:13:00.688 INFO:teuthology.orchestra.run.smithi043.stderr:145450K .......... .......... .......... .......... .......... 86% 8.00M 3s 2024-03-30T13:13:00.695 INFO:teuthology.orchestra.run.smithi043.stderr:145500K .......... .......... .......... .......... .......... 86% 7.38M 3s 2024-03-30T13:13:00.702 INFO:teuthology.orchestra.run.smithi043.stderr:145550K .......... .......... .......... .......... .......... 86% 6.72M 3s 2024-03-30T13:13:00.708 INFO:teuthology.orchestra.run.smithi043.stderr:145600K .......... .......... .......... .......... .......... 86% 7.71M 3s 2024-03-30T13:13:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:145650K .......... .......... .......... .......... .......... 86% 7.76M 3s 2024-03-30T13:13:00.720 INFO:teuthology.orchestra.run.smithi043.stderr:145700K .......... .......... .......... .......... .......... 86% 7.96M 3s 2024-03-30T13:13:00.728 INFO:teuthology.orchestra.run.smithi043.stderr:145750K .......... .......... .......... .......... .......... 86% 6.89M 3s 2024-03-30T13:13:00.733 INFO:teuthology.orchestra.run.smithi043.stderr:145800K .......... .......... .......... .......... .......... 86% 8.19M 3s 2024-03-30T13:13:00.739 INFO:teuthology.orchestra.run.smithi043.stderr:145850K .......... .......... .......... .......... .......... 86% 8.26M 3s 2024-03-30T13:13:00.745 INFO:teuthology.orchestra.run.smithi043.stderr:145900K .......... .......... .......... .......... .......... 86% 8.69M 3s 2024-03-30T13:13:00.752 INFO:teuthology.orchestra.run.smithi043.stderr:145950K .......... .......... .......... .......... .......... 86% 6.68M 3s 2024-03-30T13:13:00.759 INFO:teuthology.orchestra.run.smithi043.stderr:146000K .......... .......... .......... .......... .......... 86% 7.72M 3s 2024-03-30T13:13:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:146050K .......... .......... .......... .......... .......... 86% 7.31M 3s 2024-03-30T13:13:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:146100K .......... .......... .......... .......... .......... 86% 7.00M 3s 2024-03-30T13:13:00.780 INFO:teuthology.orchestra.run.smithi043.stderr:146150K .......... .......... .......... .......... .......... 86% 6.22M 3s 2024-03-30T13:13:00.787 INFO:teuthology.orchestra.run.smithi043.stderr:146200K .......... .......... .......... .......... .......... 86% 7.16M 3s 2024-03-30T13:13:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:146250K .......... .......... .......... .......... .......... 86% 7.25M 3s 2024-03-30T13:13:00.801 INFO:teuthology.orchestra.run.smithi043.stderr:146300K .......... .......... .......... .......... .......... 86% 6.93M 3s 2024-03-30T13:13:00.808 INFO:teuthology.orchestra.run.smithi043.stderr:146350K .......... .......... .......... .......... .......... 86% 6.63M 3s 2024-03-30T13:13:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:146400K .......... .......... .......... .......... .......... 86% 7.65M 3s 2024-03-30T13:13:00.821 INFO:teuthology.orchestra.run.smithi043.stderr:146450K .......... .......... .......... .......... .......... 86% 7.63M 3s 2024-03-30T13:13:00.828 INFO:teuthology.orchestra.run.smithi043.stderr:146500K .......... .......... .......... .......... .......... 86% 7.35M 2s 2024-03-30T13:13:00.836 INFO:teuthology.orchestra.run.smithi043.stderr:146550K .......... .......... .......... .......... .......... 86% 6.14M 2s 2024-03-30T13:13:00.843 INFO:teuthology.orchestra.run.smithi043.stderr:146600K .......... .......... .......... .......... .......... 86% 7.06M 2s 2024-03-30T13:13:00.850 INFO:teuthology.orchestra.run.smithi043.stderr:146650K .......... .......... .......... .......... .......... 87% 6.95M 2s 2024-03-30T13:13:00.856 INFO:teuthology.orchestra.run.smithi043.stderr:146700K .......... .......... .......... .......... .......... 87% 7.02M 2s 2024-03-30T13:13:00.864 INFO:teuthology.orchestra.run.smithi043.stderr:146750K .......... .......... .......... .......... .......... 87% 6.30M 2s 2024-03-30T13:13:00.871 INFO:teuthology.orchestra.run.smithi043.stderr:146800K .......... .......... .......... .......... .......... 87% 7.34M 2s 2024-03-30T13:13:00.878 INFO:teuthology.orchestra.run.smithi043.stderr:146850K .......... .......... .......... .......... .......... 87% 7.13M 2s 2024-03-30T13:13:00.884 INFO:teuthology.orchestra.run.smithi043.stderr:146900K .......... .......... .......... .......... .......... 87% 7.52M 2s 2024-03-30T13:13:00.893 INFO:teuthology.orchestra.run.smithi043.stderr:146950K .......... .......... .......... .......... .......... 87% 5.80M 2s 2024-03-30T13:13:00.900 INFO:teuthology.orchestra.run.smithi043.stderr:147000K .......... .......... .......... .......... .......... 87% 6.94M 2s 2024-03-30T13:13:00.907 INFO:teuthology.orchestra.run.smithi043.stderr:147050K .......... .......... .......... .......... .......... 87% 6.78M 2s 2024-03-30T13:13:00.914 INFO:teuthology.orchestra.run.smithi043.stderr:147100K .......... .......... .......... .......... .......... 87% 6.75M 2s 2024-03-30T13:13:00.922 INFO:teuthology.orchestra.run.smithi043.stderr:147150K .......... .......... .......... .......... .......... 87% 6.08M 2s 2024-03-30T13:13:00.930 INFO:teuthology.orchestra.run.smithi043.stderr:147200K .......... .......... .......... .......... .......... 87% 6.58M 2s 2024-03-30T13:13:00.937 INFO:teuthology.orchestra.run.smithi043.stderr:147250K .......... .......... .......... .......... .......... 87% 6.37M 2s 2024-03-30T13:13:00.945 INFO:teuthology.orchestra.run.smithi043.stderr:147300K .......... .......... .......... .......... .......... 87% 6.54M 2s 2024-03-30T13:13:00.954 INFO:teuthology.orchestra.run.smithi043.stderr:147350K .......... .......... .......... .......... .......... 87% 5.41M 2s 2024-03-30T13:13:00.962 INFO:teuthology.orchestra.run.smithi043.stderr:147400K .......... .......... .......... .......... .......... 87% 6.33M 2s 2024-03-30T13:13:00.968 INFO:teuthology.orchestra.run.smithi043.stderr:147450K .......... .......... .......... .......... .......... 87% 7.47M 2s 2024-03-30T13:13:00.974 INFO:teuthology.orchestra.run.smithi043.stderr:147500K .......... .......... .......... .......... .......... 87% 8.31M 2s 2024-03-30T13:13:00.979 INFO:teuthology.orchestra.run.smithi043.stderr:147550K .......... .......... .......... .......... .......... 87% 9.35M 2s 2024-03-30T13:13:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:147600K .......... .......... .......... .......... .......... 87% 11.2M 2s 2024-03-30T13:13:00.987 INFO:teuthology.orchestra.run.smithi043.stderr:147650K .......... .......... .......... .......... .......... 87% 11.2M 2s 2024-03-30T13:13:00.992 INFO:teuthology.orchestra.run.smithi043.stderr:147700K .......... .......... .......... .......... .......... 87% 11.2M 2s 2024-03-30T13:13:00.997 INFO:teuthology.orchestra.run.smithi043.stderr:147750K .......... .......... .......... .......... .......... 87% 9.55M 2s 2024-03-30T13:13:01.001 INFO:teuthology.orchestra.run.smithi043.stderr:147800K .......... .......... .......... .......... .......... 87% 11.2M 2s 2024-03-30T13:13:01.006 INFO:teuthology.orchestra.run.smithi043.stderr:147850K .......... .......... .......... .......... .......... 87% 10.7M 2s 2024-03-30T13:13:01.010 INFO:teuthology.orchestra.run.smithi043.stderr:147900K .......... .......... .......... .......... .......... 87% 10.5M 2s 2024-03-30T13:13:01.016 INFO:teuthology.orchestra.run.smithi043.stderr:147950K .......... .......... .......... .......... .......... 87% 8.25M 2s 2024-03-30T13:13:01.021 INFO:teuthology.orchestra.run.smithi043.stderr:148000K .......... .......... .......... .......... .......... 87% 9.66M 2s 2024-03-30T13:13:01.027 INFO:teuthology.orchestra.run.smithi043.stderr:148050K .......... .......... .......... .......... .......... 87% 9.43M 2s 2024-03-30T13:13:01.032 INFO:teuthology.orchestra.run.smithi043.stderr:148100K .......... .......... .......... .......... .......... 87% 8.77M 2s 2024-03-30T13:13:01.039 INFO:teuthology.orchestra.run.smithi043.stderr:148150K .......... .......... .......... .......... .......... 87% 7.84M 2s 2024-03-30T13:13:01.044 INFO:teuthology.orchestra.run.smithi043.stderr:148200K .......... .......... .......... .......... .......... 87% 8.74M 2s 2024-03-30T13:13:01.049 INFO:teuthology.orchestra.run.smithi043.stderr:148250K .......... .......... .......... .......... .......... 87% 9.49M 2s 2024-03-30T13:13:01.054 INFO:teuthology.orchestra.run.smithi043.stderr:148300K .......... .......... .......... .......... .......... 88% 9.25M 2s 2024-03-30T13:13:01.061 INFO:teuthology.orchestra.run.smithi043.stderr:148350K .......... .......... .......... .......... .......... 88% 7.73M 2s 2024-03-30T13:13:01.067 INFO:teuthology.orchestra.run.smithi043.stderr:148400K .......... .......... .......... .......... .......... 88% 8.71M 2s 2024-03-30T13:13:01.072 INFO:teuthology.orchestra.run.smithi043.stderr:148450K .......... .......... .......... .......... .......... 88% 8.55M 2s 2024-03-30T13:13:01.078 INFO:teuthology.orchestra.run.smithi043.stderr:148500K .......... .......... .......... .......... .......... 88% 8.51M 2s 2024-03-30T13:13:01.085 INFO:teuthology.orchestra.run.smithi043.stderr:148550K .......... .......... .......... .......... .......... 88% 6.77M 2s 2024-03-30T13:13:01.091 INFO:teuthology.orchestra.run.smithi043.stderr:148600K .......... .......... .......... .......... .......... 88% 8.56M 2s 2024-03-30T13:13:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:148650K .......... .......... .......... .......... .......... 88% 10.0M 2s 2024-03-30T13:13:01.101 INFO:teuthology.orchestra.run.smithi043.stderr:148700K .......... .......... .......... .......... .......... 88% 9.93M 2s 2024-03-30T13:13:01.107 INFO:teuthology.orchestra.run.smithi043.stderr:148750K .......... .......... .......... .......... .......... 88% 7.94M 2s 2024-03-30T13:13:01.112 INFO:teuthology.orchestra.run.smithi043.stderr:148800K .......... .......... .......... .......... .......... 88% 9.10M 2s 2024-03-30T13:13:01.117 INFO:teuthology.orchestra.run.smithi043.stderr:148850K .......... .......... .......... .......... .......... 88% 9.44M 2s 2024-03-30T13:13:01.122 INFO:teuthology.orchestra.run.smithi043.stderr:148900K .......... .......... .......... .......... .......... 88% 9.35M 2s 2024-03-30T13:13:01.129 INFO:teuthology.orchestra.run.smithi043.stderr:148950K .......... .......... .......... .......... .......... 88% 7.64M 2s 2024-03-30T13:13:01.134 INFO:teuthology.orchestra.run.smithi043.stderr:149000K .......... .......... .......... .......... .......... 88% 8.87M 2s 2024-03-30T13:13:01.140 INFO:teuthology.orchestra.run.smithi043.stderr:149050K .......... .......... .......... .......... .......... 88% 9.11M 2s 2024-03-30T13:13:01.145 INFO:teuthology.orchestra.run.smithi043.stderr:149100K .......... .......... .......... .......... .......... 88% 8.88M 2s 2024-03-30T13:13:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:149150K .......... .......... .......... .......... .......... 88% 8.49M 2s 2024-03-30T13:13:01.156 INFO:teuthology.orchestra.run.smithi043.stderr:149200K .......... .......... .......... .......... .......... 88% 9.56M 2s 2024-03-30T13:13:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:149250K .......... .......... .......... .......... .......... 88% 9.19M 2s 2024-03-30T13:13:01.167 INFO:teuthology.orchestra.run.smithi043.stderr:149300K .......... .......... .......... .......... .......... 88% 8.82M 2s 2024-03-30T13:13:01.173 INFO:teuthology.orchestra.run.smithi043.stderr:149350K .......... .......... .......... .......... .......... 88% 7.81M 2s 2024-03-30T13:13:01.179 INFO:teuthology.orchestra.run.smithi043.stderr:149400K .......... .......... .......... .......... .......... 88% 9.09M 2s 2024-03-30T13:13:01.184 INFO:teuthology.orchestra.run.smithi043.stderr:149450K .......... .......... .......... .......... .......... 88% 8.40M 2s 2024-03-30T13:13:01.190 INFO:teuthology.orchestra.run.smithi043.stderr:149500K .......... .......... .......... .......... .......... 88% 8.66M 2s 2024-03-30T13:13:01.196 INFO:teuthology.orchestra.run.smithi043.stderr:149550K .......... .......... .......... .......... .......... 88% 7.68M 2s 2024-03-30T13:13:01.202 INFO:teuthology.orchestra.run.smithi043.stderr:149600K .......... .......... .......... .......... .......... 88% 9.07M 2s 2024-03-30T13:13:01.207 INFO:teuthology.orchestra.run.smithi043.stderr:149650K .......... .......... .......... .......... .......... 88% 9.13M 2s 2024-03-30T13:13:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:149700K .......... .......... .......... .......... .......... 88% 8.55M 2s 2024-03-30T13:13:01.220 INFO:teuthology.orchestra.run.smithi043.stderr:149750K .......... .......... .......... .......... .......... 88% 7.21M 2s 2024-03-30T13:13:01.226 INFO:teuthology.orchestra.run.smithi043.stderr:149800K .......... .......... .......... .......... .......... 88% 8.11M 2s 2024-03-30T13:13:01.232 INFO:teuthology.orchestra.run.smithi043.stderr:149850K .......... .......... .......... .......... .......... 88% 7.94M 2s 2024-03-30T13:13:01.238 INFO:teuthology.orchestra.run.smithi043.stderr:149900K .......... .......... .......... .......... .......... 88% 8.52M 2s 2024-03-30T13:13:01.245 INFO:teuthology.orchestra.run.smithi043.stderr:149950K .......... .......... .......... .......... .......... 88% 6.81M 2s 2024-03-30T13:13:01.251 INFO:teuthology.orchestra.run.smithi043.stderr:150000K .......... .......... .......... .......... .......... 89% 8.26M 2s 2024-03-30T13:13:01.256 INFO:teuthology.orchestra.run.smithi043.stderr:150050K .......... .......... .......... .......... .......... 89% 8.76M 2s 2024-03-30T13:13:01.262 INFO:teuthology.orchestra.run.smithi043.stderr:150100K .......... .......... .......... .......... .......... 89% 8.58M 2s 2024-03-30T13:13:01.269 INFO:teuthology.orchestra.run.smithi043.stderr:150150K .......... .......... .......... .......... .......... 89% 7.45M 2s 2024-03-30T13:13:01.274 INFO:teuthology.orchestra.run.smithi043.stderr:150200K .......... .......... .......... .......... .......... 89% 8.62M 2s 2024-03-30T13:13:01.280 INFO:teuthology.orchestra.run.smithi043.stderr:150250K .......... .......... .......... .......... .......... 89% 8.00M 2s 2024-03-30T13:13:01.286 INFO:teuthology.orchestra.run.smithi043.stderr:150300K .......... .......... .......... .......... .......... 89% 8.40M 2s 2024-03-30T13:13:01.293 INFO:teuthology.orchestra.run.smithi043.stderr:150350K .......... .......... .......... .......... .......... 89% 6.96M 2s 2024-03-30T13:13:01.299 INFO:teuthology.orchestra.run.smithi043.stderr:150400K .......... .......... .......... .......... .......... 89% 8.10M 2s 2024-03-30T13:13:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:150450K .......... .......... .......... .......... .......... 89% 8.03M 2s 2024-03-30T13:13:01.311 INFO:teuthology.orchestra.run.smithi043.stderr:150500K .......... .......... .......... .......... .......... 89% 8.36M 2s 2024-03-30T13:13:01.318 INFO:teuthology.orchestra.run.smithi043.stderr:150550K .......... .......... .......... .......... .......... 89% 6.90M 2s 2024-03-30T13:13:01.325 INFO:teuthology.orchestra.run.smithi043.stderr:150600K .......... .......... .......... .......... .......... 89% 7.49M 2s 2024-03-30T13:13:01.331 INFO:teuthology.orchestra.run.smithi043.stderr:150650K .......... .......... .......... .......... .......... 89% 7.30M 2s 2024-03-30T13:13:01.338 INFO:teuthology.orchestra.run.smithi043.stderr:150700K .......... .......... .......... .......... .......... 89% 7.87M 2s 2024-03-30T13:13:01.346 INFO:teuthology.orchestra.run.smithi043.stderr:150750K .......... .......... .......... .......... .......... 89% 6.09M 2s 2024-03-30T13:13:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:150800K .......... .......... .......... .......... .......... 89% 7.51M 2s 2024-03-30T13:13:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:150850K .......... .......... .......... .......... .......... 89% 7.36M 2s 2024-03-30T13:13:01.365 INFO:teuthology.orchestra.run.smithi043.stderr:150900K .......... .......... .......... .......... .......... 89% 8.06M 2s 2024-03-30T13:13:01.372 INFO:teuthology.orchestra.run.smithi043.stderr:150950K .......... .......... .......... .......... .......... 89% 6.69M 2s 2024-03-30T13:13:01.378 INFO:teuthology.orchestra.run.smithi043.stderr:151000K .......... .......... .......... .......... .......... 89% 7.78M 2s 2024-03-30T13:13:01.385 INFO:teuthology.orchestra.run.smithi043.stderr:151050K .......... .......... .......... .......... .......... 89% 7.48M 2s 2024-03-30T13:13:01.392 INFO:teuthology.orchestra.run.smithi043.stderr:151100K .......... .......... .......... .......... .......... 89% 7.05M 2s 2024-03-30T13:13:01.400 INFO:teuthology.orchestra.run.smithi043.stderr:151150K .......... .......... .......... .......... .......... 89% 6.08M 2s 2024-03-30T13:13:01.407 INFO:teuthology.orchestra.run.smithi043.stderr:151200K .......... .......... .......... .......... .......... 89% 7.03M 2s 2024-03-30T13:13:01.413 INFO:teuthology.orchestra.run.smithi043.stderr:151250K .......... .......... .......... .......... .......... 89% 7.59M 2s 2024-03-30T13:13:01.420 INFO:teuthology.orchestra.run.smithi043.stderr:151300K .......... .......... .......... .......... .......... 89% 7.05M 2s 2024-03-30T13:13:01.428 INFO:teuthology.orchestra.run.smithi043.stderr:151350K .......... .......... .......... .......... .......... 89% 6.39M 2s 2024-03-30T13:13:01.434 INFO:teuthology.orchestra.run.smithi043.stderr:151400K .......... .......... .......... .......... .......... 89% 8.27M 2s 2024-03-30T13:13:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:151450K .......... .......... .......... .......... .......... 89% 7.98M 2s 2024-03-30T13:13:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:151500K .......... .......... .......... .......... .......... 89% 7.85M 2s 2024-03-30T13:13:01.452 INFO:teuthology.orchestra.run.smithi043.stderr:151550K .......... .......... .......... .......... .......... 89% 7.96M 2s 2024-03-30T13:13:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:151600K .......... .......... .......... .......... .......... 89% 11.3M 2s 2024-03-30T13:13:01.461 INFO:teuthology.orchestra.run.smithi043.stderr:151650K .......... .......... .......... .......... .......... 89% 10.1M 2s 2024-03-30T13:13:01.467 INFO:teuthology.orchestra.run.smithi043.stderr:151700K .......... .......... .......... .......... .......... 90% 9.06M 2s 2024-03-30T13:13:01.472 INFO:teuthology.orchestra.run.smithi043.stderr:151750K .......... .......... .......... .......... .......... 90% 8.34M 2s 2024-03-30T13:13:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:151800K .......... .......... .......... .......... .......... 90% 10.3M 2s 2024-03-30T13:13:01.483 INFO:teuthology.orchestra.run.smithi043.stderr:151850K .......... .......... .......... .......... .......... 90% 9.27M 2s 2024-03-30T13:13:01.488 INFO:teuthology.orchestra.run.smithi043.stderr:151900K .......... .......... .......... .......... .......... 90% 10.2M 2s 2024-03-30T13:13:01.493 INFO:teuthology.orchestra.run.smithi043.stderr:151950K .......... .......... .......... .......... .......... 90% 9.16M 2s 2024-03-30T13:13:01.498 INFO:teuthology.orchestra.run.smithi043.stderr:152000K .......... .......... .......... .......... .......... 90% 9.90M 2s 2024-03-30T13:13:01.502 INFO:teuthology.orchestra.run.smithi043.stderr:152050K .......... .......... .......... .......... .......... 90% 9.83M 2s 2024-03-30T13:13:01.508 INFO:teuthology.orchestra.run.smithi043.stderr:152100K .......... .......... .......... .......... .......... 90% 9.18M 2s 2024-03-30T13:13:01.514 INFO:teuthology.orchestra.run.smithi043.stderr:152150K .......... .......... .......... .......... .......... 90% 8.43M 2s 2024-03-30T13:13:01.519 INFO:teuthology.orchestra.run.smithi043.stderr:152200K .......... .......... .......... .......... .......... 90% 9.85M 2s 2024-03-30T13:13:01.524 INFO:teuthology.orchestra.run.smithi043.stderr:152250K .......... .......... .......... .......... .......... 90% 9.31M 2s 2024-03-30T13:13:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:152300K .......... .......... .......... .......... .......... 90% 9.53M 2s 2024-03-30T13:13:01.535 INFO:teuthology.orchestra.run.smithi043.stderr:152350K .......... .......... .......... .......... .......... 90% 7.78M 2s 2024-03-30T13:13:01.540 INFO:teuthology.orchestra.run.smithi043.stderr:152400K .......... .......... .......... .......... .......... 90% 9.61M 2s 2024-03-30T13:13:01.545 INFO:teuthology.orchestra.run.smithi043.stderr:152450K .......... .......... .......... .......... .......... 90% 10.4M 2s 2024-03-30T13:13:01.550 INFO:teuthology.orchestra.run.smithi043.stderr:152500K .......... .......... .......... .......... .......... 90% 10.1M 2s 2024-03-30T13:13:01.555 INFO:teuthology.orchestra.run.smithi043.stderr:152550K .......... .......... .......... .......... .......... 90% 8.67M 2s 2024-03-30T13:13:01.561 INFO:teuthology.orchestra.run.smithi043.stderr:152600K .......... .......... .......... .......... .......... 90% 9.70M 2s 2024-03-30T13:13:01.566 INFO:teuthology.orchestra.run.smithi043.stderr:152650K .......... .......... .......... .......... .......... 90% 9.49M 2s 2024-03-30T13:13:01.571 INFO:teuthology.orchestra.run.smithi043.stderr:152700K .......... .......... .......... .......... .......... 90% 9.44M 2s 2024-03-30T13:13:01.578 INFO:teuthology.orchestra.run.smithi043.stderr:152750K .......... .......... .......... .......... .......... 90% 7.33M 2s 2024-03-30T13:13:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:152800K .......... .......... .......... .......... .......... 90% 8.40M 2s 2024-03-30T13:13:01.589 INFO:teuthology.orchestra.run.smithi043.stderr:152850K .......... .......... .......... .......... .......... 90% 8.59M 2s 2024-03-30T13:13:01.595 INFO:teuthology.orchestra.run.smithi043.stderr:152900K .......... .......... .......... .......... .......... 90% 8.09M 2s 2024-03-30T13:13:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:152950K .......... .......... .......... .......... .......... 90% 6.93M 2s 2024-03-30T13:13:01.607 INFO:teuthology.orchestra.run.smithi043.stderr:153000K .......... .......... .......... .......... .......... 90% 9.32M 2s 2024-03-30T13:13:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:153050K .......... .......... .......... .......... .......... 90% 9.55M 2s 2024-03-30T13:13:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:153100K .......... .......... .......... .......... .......... 90% 9.29M 2s 2024-03-30T13:13:01.625 INFO:teuthology.orchestra.run.smithi043.stderr:153150K .......... .......... .......... .......... .......... 90% 7.08M 2s 2024-03-30T13:13:01.630 INFO:teuthology.orchestra.run.smithi043.stderr:153200K .......... .......... .......... .......... .......... 90% 8.33M 2s 2024-03-30T13:13:01.637 INFO:teuthology.orchestra.run.smithi043.stderr:153250K .......... .......... .......... .......... .......... 90% 8.35M 2s 2024-03-30T13:13:01.642 INFO:teuthology.orchestra.run.smithi043.stderr:153300K .......... .......... .......... .......... .......... 90% 8.32M 2s 2024-03-30T13:13:01.649 INFO:teuthology.orchestra.run.smithi043.stderr:153350K .......... .......... .......... .......... .......... 91% 7.31M 2s 2024-03-30T13:13:01.655 INFO:teuthology.orchestra.run.smithi043.stderr:153400K .......... .......... .......... .......... .......... 91% 8.64M 2s 2024-03-30T13:13:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:153450K .......... .......... .......... .......... .......... 91% 8.71M 2s 2024-03-30T13:13:01.666 INFO:teuthology.orchestra.run.smithi043.stderr:153500K .......... .......... .......... .......... .......... 91% 8.23M 2s 2024-03-30T13:13:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:153550K .......... .......... .......... .......... .......... 91% 6.64M 2s 2024-03-30T13:13:01.679 INFO:teuthology.orchestra.run.smithi043.stderr:153600K .......... .......... .......... .......... .......... 91% 8.33M 2s 2024-03-30T13:13:01.685 INFO:teuthology.orchestra.run.smithi043.stderr:153650K .......... .......... .......... .......... .......... 91% 8.19M 2s 2024-03-30T13:13:01.692 INFO:teuthology.orchestra.run.smithi043.stderr:153700K .......... .......... .......... .......... .......... 91% 7.71M 2s 2024-03-30T13:13:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:153750K .......... .......... .......... .......... .......... 91% 6.60M 2s 2024-03-30T13:13:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:153800K .......... .......... .......... .......... .......... 91% 7.88M 2s 2024-03-30T13:13:01.711 INFO:teuthology.orchestra.run.smithi043.stderr:153850K .......... .......... .......... .......... .......... 91% 7.76M 2s 2024-03-30T13:13:01.717 INFO:teuthology.orchestra.run.smithi043.stderr:153900K .......... .......... .......... .......... .......... 91% 8.01M 2s 2024-03-30T13:13:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:153950K .......... .......... .......... .......... .......... 91% 7.06M 2s 2024-03-30T13:13:01.731 INFO:teuthology.orchestra.run.smithi043.stderr:154000K .......... .......... .......... .......... .......... 91% 7.85M 2s 2024-03-30T13:13:01.738 INFO:teuthology.orchestra.run.smithi043.stderr:154050K .......... .......... .......... .......... .......... 91% 8.20M 2s 2024-03-30T13:13:01.842 INFO:teuthology.orchestra.run.smithi043.stderr:154100K .......... .......... .......... .......... .......... 91% 8.21M 2s 2024-03-30T13:13:01.842 INFO:teuthology.orchestra.run.smithi043.stderr:154150K .......... .......... .......... .......... .......... 91% 7.25M 2s 2024-03-30T13:13:01.842 INFO:teuthology.orchestra.run.smithi043.stderr:154200K .......... .......... .......... .......... .......... 91% 7.37M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154250K .......... .......... .......... .......... .......... 91% 7.72M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154300K .......... .......... .......... .......... .......... 91% 8.10M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154350K .......... .......... .......... .......... .......... 91% 6.66M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154400K .......... .......... .......... .......... .......... 91% 7.11M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154450K .......... .......... .......... .......... .......... 91% 7.25M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154500K .......... .......... .......... .......... .......... 91% 7.32M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154550K .......... .......... .......... .......... .......... 91% 6.64M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154600K .......... .......... .......... .......... .......... 91% 7.60M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154650K .......... .......... .......... .......... .......... 91% 7.71M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154700K .......... .......... .......... .......... .......... 91% 7.59M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154750K .......... .......... .......... .......... .......... 91% 6.06M 2s 2024-03-30T13:13:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:154800K .......... .......... .......... .......... .......... 91% 7.26M 2s 2024-03-30T13:13:01.844 INFO:teuthology.orchestra.run.smithi043.stderr:154850K .......... .......... .......... .......... .......... 91% 7.28M 2s 2024-03-30T13:13:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:154900K .......... .......... .......... .......... .......... 91% 7.55M 2s 2024-03-30T13:13:01.858 INFO:teuthology.orchestra.run.smithi043.stderr:154950K .......... .......... .......... .......... .......... 91% 6.64M 2s 2024-03-30T13:13:01.864 INFO:teuthology.orchestra.run.smithi043.stderr:155000K .......... .......... .......... .......... .......... 91% 7.60M 2s 2024-03-30T13:13:01.870 INFO:teuthology.orchestra.run.smithi043.stderr:155050K .......... .......... .......... .......... .......... 92% 8.12M 2s 2024-03-30T13:13:01.876 INFO:teuthology.orchestra.run.smithi043.stderr:155100K .......... .......... .......... .......... .......... 92% 8.63M 2s 2024-03-30T13:13:01.882 INFO:teuthology.orchestra.run.smithi043.stderr:155150K .......... .......... .......... .......... .......... 92% 7.25M 2s 2024-03-30T13:13:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:155200K .......... .......... .......... .......... .......... 92% 8.73M 2s 2024-03-30T13:13:01.894 INFO:teuthology.orchestra.run.smithi043.stderr:155250K .......... .......... .......... .......... .......... 92% 8.21M 2s 2024-03-30T13:13:01.901 INFO:teuthology.orchestra.run.smithi043.stderr:155300K .......... .......... .......... .......... .......... 92% 7.64M 2s 2024-03-30T13:13:01.909 INFO:teuthology.orchestra.run.smithi043.stderr:155350K .......... .......... .......... .......... .......... 92% 6.16M 2s 2024-03-30T13:13:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:155400K .......... .......... .......... .......... .......... 92% 7.06M 1s 2024-03-30T13:13:01.923 INFO:teuthology.orchestra.run.smithi043.stderr:155450K .......... .......... .......... .......... .......... 92% 7.01M 1s 2024-03-30T13:13:01.930 INFO:teuthology.orchestra.run.smithi043.stderr:155500K .......... .......... .......... .......... .......... 92% 6.80M 1s 2024-03-30T13:13:01.938 INFO:teuthology.orchestra.run.smithi043.stderr:155550K .......... .......... .......... .......... .......... 92% 5.70M 1s 2024-03-30T13:13:01.945 INFO:teuthology.orchestra.run.smithi043.stderr:155600K .......... .......... .......... .......... .......... 92% 6.77M 1s 2024-03-30T13:13:01.951 INFO:teuthology.orchestra.run.smithi043.stderr:155650K .......... .......... .......... .......... .......... 92% 8.30M 1s 2024-03-30T13:13:01.956 INFO:teuthology.orchestra.run.smithi043.stderr:155700K .......... .......... .......... .......... .......... 92% 9.58M 1s 2024-03-30T13:13:01.962 INFO:teuthology.orchestra.run.smithi043.stderr:155750K .......... .......... .......... .......... .......... 92% 8.58M 1s 2024-03-30T13:13:01.966 INFO:teuthology.orchestra.run.smithi043.stderr:155800K .......... .......... .......... .......... .......... 92% 11.0M 1s 2024-03-30T13:13:01.971 INFO:teuthology.orchestra.run.smithi043.stderr:155850K .......... .......... .......... .......... .......... 92% 10.4M 1s 2024-03-30T13:13:01.976 INFO:teuthology.orchestra.run.smithi043.stderr:155900K .......... .......... .......... .......... .......... 92% 10.7M 1s 2024-03-30T13:13:01.980 INFO:teuthology.orchestra.run.smithi043.stderr:155950K .......... .......... .......... .......... .......... 92% 9.85M 1s 2024-03-30T13:13:01.984 INFO:teuthology.orchestra.run.smithi043.stderr:156000K .......... .......... .......... .......... .......... 92% 12.1M 1s 2024-03-30T13:13:01.988 INFO:teuthology.orchestra.run.smithi043.stderr:156050K .......... .......... .......... .......... .......... 92% 12.2M 1s 2024-03-30T13:13:01.993 INFO:teuthology.orchestra.run.smithi043.stderr:156100K .......... .......... .......... .......... .......... 92% 11.6M 1s 2024-03-30T13:13:01.998 INFO:teuthology.orchestra.run.smithi043.stderr:156150K .......... .......... .......... .......... .......... 92% 9.10M 1s 2024-03-30T13:13:02.002 INFO:teuthology.orchestra.run.smithi043.stderr:156200K .......... .......... .......... .......... .......... 92% 11.5M 1s 2024-03-30T13:13:02.007 INFO:teuthology.orchestra.run.smithi043.stderr:156250K .......... .......... .......... .......... .......... 92% 11.1M 1s 2024-03-30T13:13:02.011 INFO:teuthology.orchestra.run.smithi043.stderr:156300K .......... .......... .......... .......... .......... 92% 11.7M 1s 2024-03-30T13:13:02.016 INFO:teuthology.orchestra.run.smithi043.stderr:156350K .......... .......... .......... .......... .......... 92% 10.1M 1s 2024-03-30T13:13:02.019 INFO:teuthology.orchestra.run.smithi043.stderr:156400K .......... .......... .......... .......... .......... 92% 12.3M 1s 2024-03-30T13:13:02.024 INFO:teuthology.orchestra.run.smithi043.stderr:156450K .......... .......... .......... .......... .......... 92% 12.1M 1s 2024-03-30T13:13:02.028 INFO:teuthology.orchestra.run.smithi043.stderr:156500K .......... .......... .......... .......... .......... 92% 12.3M 1s 2024-03-30T13:13:02.032 INFO:teuthology.orchestra.run.smithi043.stderr:156550K .......... .......... .......... .......... .......... 92% 10.6M 1s 2024-03-30T13:13:02.036 INFO:teuthology.orchestra.run.smithi043.stderr:156600K .......... .......... .......... .......... .......... 92% 12.2M 1s 2024-03-30T13:13:02.040 INFO:teuthology.orchestra.run.smithi043.stderr:156650K .......... .......... .......... .......... .......... 92% 11.7M 1s 2024-03-30T13:13:02.044 INFO:teuthology.orchestra.run.smithi043.stderr:156700K .......... .......... .......... .......... .......... 92% 11.9M 1s 2024-03-30T13:13:02.049 INFO:teuthology.orchestra.run.smithi043.stderr:156750K .......... .......... .......... .......... .......... 93% 10.1M 1s 2024-03-30T13:13:02.054 INFO:teuthology.orchestra.run.smithi043.stderr:156800K .......... .......... .......... .......... .......... 93% 11.3M 1s 2024-03-30T13:13:02.058 INFO:teuthology.orchestra.run.smithi043.stderr:156850K .......... .......... .......... .......... .......... 93% 11.6M 1s 2024-03-30T13:13:02.063 INFO:teuthology.orchestra.run.smithi043.stderr:156900K .......... .......... .......... .......... .......... 93% 10.3M 1s 2024-03-30T13:13:02.069 INFO:teuthology.orchestra.run.smithi043.stderr:156950K .......... .......... .......... .......... .......... 93% 8.45M 1s 2024-03-30T13:13:02.073 INFO:teuthology.orchestra.run.smithi043.stderr:157000K .......... .......... .......... .......... .......... 93% 9.84M 1s 2024-03-30T13:13:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:157050K .......... .......... .......... .......... .......... 93% 9.79M 1s 2024-03-30T13:13:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:157100K .......... .......... .......... .......... .......... 93% 9.02M 1s 2024-03-30T13:13:02.089 INFO:teuthology.orchestra.run.smithi043.stderr:157150K .......... .......... .......... .......... .......... 93% 9.02M 1s 2024-03-30T13:13:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:157200K .......... .......... .......... .......... .......... 93% 11.0M 1s 2024-03-30T13:13:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:157250K .......... .......... .......... .......... .......... 93% 10.5M 1s 2024-03-30T13:13:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:157300K .......... .......... .......... .......... .......... 93% 9.93M 1s 2024-03-30T13:13:02.111 INFO:teuthology.orchestra.run.smithi043.stderr:157350K .......... .......... .......... .......... .......... 93% 6.82M 1s 2024-03-30T13:13:02.117 INFO:teuthology.orchestra.run.smithi043.stderr:157400K .......... .......... .......... .......... .......... 93% 7.76M 1s 2024-03-30T13:13:02.123 INFO:teuthology.orchestra.run.smithi043.stderr:157450K .......... .......... .......... .......... .......... 93% 7.71M 1s 2024-03-30T13:13:02.129 INFO:teuthology.orchestra.run.smithi043.stderr:157500K .......... .......... .......... .......... .......... 93% 8.11M 1s 2024-03-30T13:13:02.136 INFO:teuthology.orchestra.run.smithi043.stderr:157550K .......... .......... .......... .......... .......... 93% 7.28M 1s 2024-03-30T13:13:02.142 INFO:teuthology.orchestra.run.smithi043.stderr:157600K .......... .......... .......... .......... .......... 93% 8.44M 1s 2024-03-30T13:13:02.147 INFO:teuthology.orchestra.run.smithi043.stderr:157650K .......... .......... .......... .......... .......... 93% 8.66M 1s 2024-03-30T13:13:02.152 INFO:teuthology.orchestra.run.smithi043.stderr:157700K .......... .......... .......... .......... .......... 93% 9.28M 1s 2024-03-30T13:13:02.159 INFO:teuthology.orchestra.run.smithi043.stderr:157750K .......... .......... .......... .......... .......... 93% 7.77M 1s 2024-03-30T13:13:02.165 INFO:teuthology.orchestra.run.smithi043.stderr:157800K .......... .......... .......... .......... .......... 93% 7.92M 1s 2024-03-30T13:13:02.172 INFO:teuthology.orchestra.run.smithi043.stderr:157850K .......... .......... .......... .......... .......... 93% 7.10M 1s 2024-03-30T13:13:02.179 INFO:teuthology.orchestra.run.smithi043.stderr:157900K .......... .......... .......... .......... .......... 93% 7.60M 1s 2024-03-30T13:13:02.186 INFO:teuthology.orchestra.run.smithi043.stderr:157950K .......... .......... .......... .......... .......... 93% 6.30M 1s 2024-03-30T13:13:02.193 INFO:teuthology.orchestra.run.smithi043.stderr:158000K .......... .......... .......... .......... .......... 93% 7.38M 1s 2024-03-30T13:13:02.199 INFO:teuthology.orchestra.run.smithi043.stderr:158050K .......... .......... .......... .......... .......... 93% 7.40M 1s 2024-03-30T13:13:02.206 INFO:teuthology.orchestra.run.smithi043.stderr:158100K .......... .......... .......... .......... .......... 93% 7.36M 1s 2024-03-30T13:13:02.215 INFO:teuthology.orchestra.run.smithi043.stderr:158150K .......... .......... .......... .......... .......... 93% 6.18M 1s 2024-03-30T13:13:02.221 INFO:teuthology.orchestra.run.smithi043.stderr:158200K .......... .......... .......... .......... .......... 93% 7.05M 1s 2024-03-30T13:13:02.228 INFO:teuthology.orchestra.run.smithi043.stderr:158250K .......... .......... .......... .......... .......... 93% 7.10M 1s 2024-03-30T13:13:02.234 INFO:teuthology.orchestra.run.smithi043.stderr:158300K .......... .......... .......... .......... .......... 93% 7.89M 1s 2024-03-30T13:13:02.240 INFO:teuthology.orchestra.run.smithi043.stderr:158350K .......... .......... .......... .......... .......... 93% 8.29M 1s 2024-03-30T13:13:02.245 INFO:teuthology.orchestra.run.smithi043.stderr:158400K .......... .......... .......... .......... .......... 93% 8.67M 1s 2024-03-30T13:13:02.251 INFO:teuthology.orchestra.run.smithi043.stderr:158450K .......... .......... .......... .......... .......... 94% 8.72M 1s 2024-03-30T13:13:02.256 INFO:teuthology.orchestra.run.smithi043.stderr:158500K .......... .......... .......... .......... .......... 94% 8.91M 1s 2024-03-30T13:13:02.263 INFO:teuthology.orchestra.run.smithi043.stderr:158550K .......... .......... .......... .......... .......... 94% 7.43M 1s 2024-03-30T13:13:02.269 INFO:teuthology.orchestra.run.smithi043.stderr:158600K .......... .......... .......... .......... .......... 94% 8.87M 1s 2024-03-30T13:13:02.274 INFO:teuthology.orchestra.run.smithi043.stderr:158650K .......... .......... .......... .......... .......... 94% 8.63M 1s 2024-03-30T13:13:02.280 INFO:teuthology.orchestra.run.smithi043.stderr:158700K .......... .......... .......... .......... .......... 94% 8.65M 1s 2024-03-30T13:13:02.287 INFO:teuthology.orchestra.run.smithi043.stderr:158750K .......... .......... .......... .......... .......... 94% 7.17M 1s 2024-03-30T13:13:02.293 INFO:teuthology.orchestra.run.smithi043.stderr:158800K .......... .......... .......... .......... .......... 94% 7.62M 1s 2024-03-30T13:13:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:158850K .......... .......... .......... .......... .......... 94% 7.78M 1s 2024-03-30T13:13:02.306 INFO:teuthology.orchestra.run.smithi043.stderr:158900K .......... .......... .......... .......... .......... 94% 8.08M 1s 2024-03-30T13:13:02.313 INFO:teuthology.orchestra.run.smithi043.stderr:158950K .......... .......... .......... .......... .......... 94% 6.47M 1s 2024-03-30T13:13:02.319 INFO:teuthology.orchestra.run.smithi043.stderr:159000K .......... .......... .......... .......... .......... 94% 7.72M 1s 2024-03-30T13:13:02.326 INFO:teuthology.orchestra.run.smithi043.stderr:159050K .......... .......... .......... .......... .......... 94% 7.91M 1s 2024-03-30T13:13:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:159100K .......... .......... .......... .......... .......... 94% 7.91M 1s 2024-03-30T13:13:02.340 INFO:teuthology.orchestra.run.smithi043.stderr:159150K .......... .......... .......... .......... .......... 94% 6.31M 1s 2024-03-30T13:13:02.345 INFO:teuthology.orchestra.run.smithi043.stderr:159200K .......... .......... .......... .......... .......... 94% 8.13M 1s 2024-03-30T13:13:02.352 INFO:teuthology.orchestra.run.smithi043.stderr:159250K .......... .......... .......... .......... .......... 94% 7.64M 1s 2024-03-30T13:13:02.358 INFO:teuthology.orchestra.run.smithi043.stderr:159300K .......... .......... .......... .......... .......... 94% 7.62M 1s 2024-03-30T13:13:02.365 INFO:teuthology.orchestra.run.smithi043.stderr:159350K .......... .......... .......... .......... .......... 94% 7.40M 1s 2024-03-30T13:13:02.370 INFO:teuthology.orchestra.run.smithi043.stderr:159400K .......... .......... .......... .......... .......... 94% 8.85M 1s 2024-03-30T13:13:02.376 INFO:teuthology.orchestra.run.smithi043.stderr:159450K .......... .......... .......... .......... .......... 94% 8.65M 1s 2024-03-30T13:13:02.381 INFO:teuthology.orchestra.run.smithi043.stderr:159500K .......... .......... .......... .......... .......... 94% 8.96M 1s 2024-03-30T13:13:02.388 INFO:teuthology.orchestra.run.smithi043.stderr:159550K .......... .......... .......... .......... .......... 94% 7.51M 1s 2024-03-30T13:13:02.393 INFO:teuthology.orchestra.run.smithi043.stderr:159600K .......... .......... .......... .......... .......... 94% 8.89M 1s 2024-03-30T13:13:02.399 INFO:teuthology.orchestra.run.smithi043.stderr:159650K .......... .......... .......... .......... .......... 94% 8.61M 1s 2024-03-30T13:13:02.404 INFO:teuthology.orchestra.run.smithi043.stderr:159700K .......... .......... .......... .......... .......... 94% 8.73M 1s 2024-03-30T13:13:02.409 INFO:teuthology.orchestra.run.smithi043.stderr:159750K .......... .......... .......... .......... .......... 94% 9.66M 1s 2024-03-30T13:13:02.414 INFO:teuthology.orchestra.run.smithi043.stderr:159800K .......... .......... .......... .......... .......... 94% 10.5M 1s 2024-03-30T13:13:02.418 INFO:teuthology.orchestra.run.smithi043.stderr:159850K .......... .......... .......... .......... .......... 94% 12.7M 1s 2024-03-30T13:13:02.424 INFO:teuthology.orchestra.run.smithi043.stderr:159900K .......... .......... .......... .......... .......... 94% 9.04M 1s 2024-03-30T13:13:02.428 INFO:teuthology.orchestra.run.smithi043.stderr:159950K .......... .......... .......... .......... .......... 94% 10.1M 1s 2024-03-30T13:13:02.431 INFO:teuthology.orchestra.run.smithi043.stderr:160000K .......... .......... .......... .......... .......... 94% 16.7M 1s 2024-03-30T13:13:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:160050K .......... .......... .......... .......... .......... 94% 13.0M 1s 2024-03-30T13:13:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:160100K .......... .......... .......... .......... .......... 95% 15.0M 1s 2024-03-30T13:13:02.442 INFO:teuthology.orchestra.run.smithi043.stderr:160150K .......... .......... .......... .......... .......... 95% 11.4M 1s 2024-03-30T13:13:02.446 INFO:teuthology.orchestra.run.smithi043.stderr:160200K .......... .......... .......... .......... .......... 95% 11.8M 1s 2024-03-30T13:13:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:160250K .......... .......... .......... .......... .......... 95% 11.3M 1s 2024-03-30T13:13:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:160300K .......... .......... .......... .......... .......... 95% 11.1M 1s 2024-03-30T13:13:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:160350K .......... .......... .......... .......... .......... 95% 9.49M 1s 2024-03-30T13:13:02.465 INFO:teuthology.orchestra.run.smithi043.stderr:160400K .......... .......... .......... .......... .......... 95% 11.3M 1s 2024-03-30T13:13:02.469 INFO:teuthology.orchestra.run.smithi043.stderr:160450K .......... .......... .......... .......... .......... 95% 10.2M 1s 2024-03-30T13:13:02.473 INFO:teuthology.orchestra.run.smithi043.stderr:160500K .......... .......... .......... .......... .......... 95% 14.2M 1s 2024-03-30T13:13:02.477 INFO:teuthology.orchestra.run.smithi043.stderr:160550K .......... .......... .......... .......... .......... 95% 12.3M 1s 2024-03-30T13:13:02.480 INFO:teuthology.orchestra.run.smithi043.stderr:160600K .......... .......... .......... .......... .......... 95% 14.4M 1s 2024-03-30T13:13:02.483 INFO:teuthology.orchestra.run.smithi043.stderr:160650K .......... .......... .......... .......... .......... 95% 14.2M 1s 2024-03-30T13:13:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:160700K .......... .......... .......... .......... .......... 95% 13.0M 1s 2024-03-30T13:13:02.493 INFO:teuthology.orchestra.run.smithi043.stderr:160750K .......... .......... .......... .......... .......... 95% 9.24M 1s 2024-03-30T13:13:02.497 INFO:teuthology.orchestra.run.smithi043.stderr:160800K .......... .......... .......... .......... .......... 95% 10.7M 1s 2024-03-30T13:13:02.502 INFO:teuthology.orchestra.run.smithi043.stderr:160850K .......... .......... .......... .......... .......... 95% 12.0M 1s 2024-03-30T13:13:02.506 INFO:teuthology.orchestra.run.smithi043.stderr:160900K .......... .......... .......... .......... .......... 95% 11.4M 1s 2024-03-30T13:13:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:160950K .......... .......... .......... .......... .......... 95% 9.12M 1s 2024-03-30T13:13:02.516 INFO:teuthology.orchestra.run.smithi043.stderr:161000K .......... .......... .......... .......... .......... 95% 10.4M 1s 2024-03-30T13:13:02.520 INFO:teuthology.orchestra.run.smithi043.stderr:161050K .......... .......... .......... .......... .......... 95% 12.2M 1s 2024-03-30T13:13:02.524 INFO:teuthology.orchestra.run.smithi043.stderr:161100K .......... .......... .......... .......... .......... 95% 11.6M 1s 2024-03-30T13:13:02.529 INFO:teuthology.orchestra.run.smithi043.stderr:161150K .......... .......... .......... .......... .......... 95% 9.67M 1s 2024-03-30T13:13:02.533 INFO:teuthology.orchestra.run.smithi043.stderr:161200K .......... .......... .......... .......... .......... 95% 10.9M 1s 2024-03-30T13:13:02.538 INFO:teuthology.orchestra.run.smithi043.stderr:161250K .......... .......... .......... .......... .......... 95% 10.8M 1s 2024-03-30T13:13:02.543 INFO:teuthology.orchestra.run.smithi043.stderr:161300K .......... .......... .......... .......... .......... 95% 10.4M 1s 2024-03-30T13:13:02.549 INFO:teuthology.orchestra.run.smithi043.stderr:161350K .......... .......... .......... .......... .......... 95% 7.87M 1s 2024-03-30T13:13:02.554 INFO:teuthology.orchestra.run.smithi043.stderr:161400K .......... .......... .......... .......... .......... 95% 10.6M 1s 2024-03-30T13:13:02.559 INFO:teuthology.orchestra.run.smithi043.stderr:161450K .......... .......... .......... .......... .......... 95% 9.19M 1s 2024-03-30T13:13:02.564 INFO:teuthology.orchestra.run.smithi043.stderr:161500K .......... .......... .......... .......... .......... 95% 9.83M 1s 2024-03-30T13:13:02.569 INFO:teuthology.orchestra.run.smithi043.stderr:161550K .......... .......... .......... .......... .......... 95% 8.75M 1s 2024-03-30T13:13:02.574 INFO:teuthology.orchestra.run.smithi043.stderr:161600K .......... .......... .......... .......... .......... 95% 10.3M 1s 2024-03-30T13:13:02.579 INFO:teuthology.orchestra.run.smithi043.stderr:161650K .......... .......... .......... .......... .......... 95% 10.4M 1s 2024-03-30T13:13:02.583 INFO:teuthology.orchestra.run.smithi043.stderr:161700K .......... .......... .......... .......... .......... 95% 10.5M 1s 2024-03-30T13:13:02.589 INFO:teuthology.orchestra.run.smithi043.stderr:161750K .......... .......... .......... .......... .......... 95% 9.57M 1s 2024-03-30T13:13:02.593 INFO:teuthology.orchestra.run.smithi043.stderr:161800K .......... .......... .......... .......... .......... 96% 10.6M 1s 2024-03-30T13:13:02.598 INFO:teuthology.orchestra.run.smithi043.stderr:161850K .......... .......... .......... .......... .......... 96% 10.9M 1s 2024-03-30T13:13:02.602 INFO:teuthology.orchestra.run.smithi043.stderr:161900K .......... .......... .......... .......... .......... 96% 10.5M 1s 2024-03-30T13:13:02.608 INFO:teuthology.orchestra.run.smithi043.stderr:161950K .......... .......... .......... .......... .......... 96% 8.16M 1s 2024-03-30T13:13:02.614 INFO:teuthology.orchestra.run.smithi043.stderr:162000K .......... .......... .......... .......... .......... 96% 9.52M 1s 2024-03-30T13:13:02.619 INFO:teuthology.orchestra.run.smithi043.stderr:162050K .......... .......... .......... .......... .......... 96% 8.74M 1s 2024-03-30T13:13:02.624 INFO:teuthology.orchestra.run.smithi043.stderr:162100K .......... .......... .......... .......... .......... 96% 9.22M 1s 2024-03-30T13:13:02.631 INFO:teuthology.orchestra.run.smithi043.stderr:162150K .......... .......... .......... .......... .......... 96% 7.55M 1s 2024-03-30T13:13:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:162200K .......... .......... .......... .......... .......... 96% 8.32M 1s 2024-03-30T13:13:02.643 INFO:teuthology.orchestra.run.smithi043.stderr:162250K .......... .......... .......... .......... .......... 96% 8.60M 1s 2024-03-30T13:13:02.648 INFO:teuthology.orchestra.run.smithi043.stderr:162300K .......... .......... .......... .......... .......... 96% 8.18M 1s 2024-03-30T13:13:02.656 INFO:teuthology.orchestra.run.smithi043.stderr:162350K .......... .......... .......... .......... .......... 96% 6.95M 1s 2024-03-30T13:13:02.661 INFO:teuthology.orchestra.run.smithi043.stderr:162400K .......... .......... .......... .......... .......... 96% 8.39M 1s 2024-03-30T13:13:02.667 INFO:teuthology.orchestra.run.smithi043.stderr:162450K .......... .......... .......... .......... .......... 96% 8.35M 1s 2024-03-30T13:13:02.673 INFO:teuthology.orchestra.run.smithi043.stderr:162500K .......... .......... .......... .......... .......... 96% 8.70M 1s 2024-03-30T13:13:02.680 INFO:teuthology.orchestra.run.smithi043.stderr:162550K .......... .......... .......... .......... .......... 96% 7.33M 1s 2024-03-30T13:13:02.685 INFO:teuthology.orchestra.run.smithi043.stderr:162600K .......... .......... .......... .......... .......... 96% 8.72M 1s 2024-03-30T13:13:02.691 INFO:teuthology.orchestra.run.smithi043.stderr:162650K .......... .......... .......... .......... .......... 96% 8.69M 1s 2024-03-30T13:13:02.697 INFO:teuthology.orchestra.run.smithi043.stderr:162700K .......... .......... .......... .......... .......... 96% 7.96M 1s 2024-03-30T13:13:02.704 INFO:teuthology.orchestra.run.smithi043.stderr:162750K .......... .......... .......... .......... .......... 96% 6.81M 1s 2024-03-30T13:13:02.710 INFO:teuthology.orchestra.run.smithi043.stderr:162800K .......... .......... .......... .......... .......... 96% 8.14M 1s 2024-03-30T13:13:02.717 INFO:teuthology.orchestra.run.smithi043.stderr:162850K .......... .......... .......... .......... .......... 96% 7.22M 1s 2024-03-30T13:13:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:162900K .......... .......... .......... .......... .......... 96% 6.95M 1s 2024-03-30T13:13:02.732 INFO:teuthology.orchestra.run.smithi043.stderr:162950K .......... .......... .......... .......... .......... 96% 6.23M 1s 2024-03-30T13:13:02.739 INFO:teuthology.orchestra.run.smithi043.stderr:163000K .......... .......... .......... .......... .......... 96% 6.92M 1s 2024-03-30T13:13:02.745 INFO:teuthology.orchestra.run.smithi043.stderr:163050K .......... .......... .......... .......... .......... 96% 7.85M 1s 2024-03-30T13:13:02.751 INFO:teuthology.orchestra.run.smithi043.stderr:163100K .......... .......... .......... .......... .......... 96% 7.64M 1s 2024-03-30T13:13:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:163150K .......... .......... .......... .......... .......... 96% 6.18M 1s 2024-03-30T13:13:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:163200K .......... .......... .......... .......... .......... 96% 7.60M 1s 2024-03-30T13:13:02.773 INFO:teuthology.orchestra.run.smithi043.stderr:163250K .......... .......... .......... .......... .......... 96% 6.98M 1s 2024-03-30T13:13:02.779 INFO:teuthology.orchestra.run.smithi043.stderr:163300K .......... .......... .......... .......... .......... 96% 7.36M 1s 2024-03-30T13:13:02.787 INFO:teuthology.orchestra.run.smithi043.stderr:163350K .......... .......... .......... .......... .......... 96% 6.51M 1s 2024-03-30T13:13:02.794 INFO:teuthology.orchestra.run.smithi043.stderr:163400K .......... .......... .......... .......... .......... 96% 7.18M 1s 2024-03-30T13:13:02.800 INFO:teuthology.orchestra.run.smithi043.stderr:163450K .......... .......... .......... .......... .......... 96% 7.40M 1s 2024-03-30T13:13:02.807 INFO:teuthology.orchestra.run.smithi043.stderr:163500K .......... .......... .......... .......... .......... 97% 7.36M 1s 2024-03-30T13:13:02.815 INFO:teuthology.orchestra.run.smithi043.stderr:163550K .......... .......... .......... .......... .......... 97% 6.20M 1s 2024-03-30T13:13:02.822 INFO:teuthology.orchestra.run.smithi043.stderr:163600K .......... .......... .......... .......... .......... 97% 7.16M 1s 2024-03-30T13:13:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:163650K .......... .......... .......... .......... .......... 97% 7.02M 1s 2024-03-30T13:13:02.835 INFO:teuthology.orchestra.run.smithi043.stderr:163700K .......... .......... .......... .......... .......... 97% 7.16M 1s 2024-03-30T13:13:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:163750K .......... .......... .......... .......... .......... 97% 6.12M 1s 2024-03-30T13:13:02.851 INFO:teuthology.orchestra.run.smithi043.stderr:163800K .......... .......... .......... .......... .......... 97% 6.54M 1s 2024-03-30T13:13:02.857 INFO:teuthology.orchestra.run.smithi043.stderr:163850K .......... .......... .......... .......... .......... 97% 8.22M 1s 2024-03-30T13:13:02.862 INFO:teuthology.orchestra.run.smithi043.stderr:163900K .......... .......... .......... .......... .......... 97% 9.56M 1s 2024-03-30T13:13:02.868 INFO:teuthology.orchestra.run.smithi043.stderr:163950K .......... .......... .......... .......... .......... 97% 9.32M 1s 2024-03-30T13:13:02.871 INFO:teuthology.orchestra.run.smithi043.stderr:164000K .......... .......... .......... .......... .......... 97% 11.6M 1s 2024-03-30T13:13:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:164050K .......... .......... .......... .......... .......... 97% 11.4M 1s 2024-03-30T13:13:02.881 INFO:teuthology.orchestra.run.smithi043.stderr:164100K .......... .......... .......... .......... .......... 97% 9.33M 1s 2024-03-30T13:13:02.886 INFO:teuthology.orchestra.run.smithi043.stderr:164150K .......... .......... .......... .......... .......... 97% 8.69M 0s 2024-03-30T13:13:02.890 INFO:teuthology.orchestra.run.smithi043.stderr:164200K .......... .......... .......... .......... .......... 97% 11.8M 0s 2024-03-30T13:13:02.895 INFO:teuthology.orchestra.run.smithi043.stderr:164250K .......... .......... .......... .......... .......... 97% 10.8M 0s 2024-03-30T13:13:02.900 INFO:teuthology.orchestra.run.smithi043.stderr:164300K .......... .......... .......... .......... .......... 97% 10.3M 0s 2024-03-30T13:13:02.906 INFO:teuthology.orchestra.run.smithi043.stderr:164350K .......... .......... .......... .......... .......... 97% 8.32M 0s 2024-03-30T13:13:02.911 INFO:teuthology.orchestra.run.smithi043.stderr:164400K .......... .......... .......... .......... .......... 97% 9.39M 0s 2024-03-30T13:13:02.916 INFO:teuthology.orchestra.run.smithi043.stderr:164450K .......... .......... .......... .......... .......... 97% 9.62M 0s 2024-03-30T13:13:02.921 INFO:teuthology.orchestra.run.smithi043.stderr:164500K .......... .......... .......... .......... .......... 97% 9.47M 0s 2024-03-30T13:13:02.927 INFO:teuthology.orchestra.run.smithi043.stderr:164550K .......... .......... .......... .......... .......... 97% 8.12M 0s 2024-03-30T13:13:02.932 INFO:teuthology.orchestra.run.smithi043.stderr:164600K .......... .......... .......... .......... .......... 97% 9.18M 0s 2024-03-30T13:13:02.938 INFO:teuthology.orchestra.run.smithi043.stderr:164650K .......... .......... .......... .......... .......... 97% 9.11M 0s 2024-03-30T13:13:02.943 INFO:teuthology.orchestra.run.smithi043.stderr:164700K .......... .......... .......... .......... .......... 97% 9.42M 0s 2024-03-30T13:13:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:164750K .......... .......... .......... .......... .......... 97% 7.99M 0s 2024-03-30T13:13:02.955 INFO:teuthology.orchestra.run.smithi043.stderr:164800K .......... .......... .......... .......... .......... 97% 8.71M 0s 2024-03-30T13:13:02.960 INFO:teuthology.orchestra.run.smithi043.stderr:164850K .......... .......... .......... .......... .......... 97% 8.99M 0s 2024-03-30T13:13:02.967 INFO:teuthology.orchestra.run.smithi043.stderr:164900K .......... .......... .......... .......... .......... 97% 9.67M 0s 2024-03-30T13:13:02.971 INFO:teuthology.orchestra.run.smithi043.stderr:164950K .......... .......... .......... .......... .......... 97% 7.90M 0s 2024-03-30T13:13:02.977 INFO:teuthology.orchestra.run.smithi043.stderr:165000K .......... .......... .......... .......... .......... 97% 9.74M 0s 2024-03-30T13:13:02.982 INFO:teuthology.orchestra.run.smithi043.stderr:165050K .......... .......... .......... .......... .......... 97% 9.20M 0s 2024-03-30T13:13:02.987 INFO:teuthology.orchestra.run.smithi043.stderr:165100K .......... .......... .......... .......... .......... 97% 8.90M 0s 2024-03-30T13:13:02.993 INFO:teuthology.orchestra.run.smithi043.stderr:165150K .......... .......... .......... .......... .......... 98% 7.79M 0s 2024-03-30T13:13:03.000 INFO:teuthology.orchestra.run.smithi043.stderr:165200K .......... .......... .......... .......... .......... 98% 7.74M 0s 2024-03-30T13:13:03.006 INFO:teuthology.orchestra.run.smithi043.stderr:165250K .......... .......... .......... .......... .......... 98% 7.52M 0s 2024-03-30T13:13:03.012 INFO:teuthology.orchestra.run.smithi043.stderr:165300K .......... .......... .......... .......... .......... 98% 7.93M 0s 2024-03-30T13:13:03.019 INFO:teuthology.orchestra.run.smithi043.stderr:165350K .......... .......... .......... .......... .......... 98% 7.39M 0s 2024-03-30T13:13:03.025 INFO:teuthology.orchestra.run.smithi043.stderr:165400K .......... .......... .......... .......... .......... 98% 8.55M 0s 2024-03-30T13:13:03.031 INFO:teuthology.orchestra.run.smithi043.stderr:165450K .......... .......... .......... .......... .......... 98% 8.37M 0s 2024-03-30T13:13:03.036 INFO:teuthology.orchestra.run.smithi043.stderr:165500K .......... .......... .......... .......... .......... 98% 8.41M 0s 2024-03-30T13:13:03.043 INFO:teuthology.orchestra.run.smithi043.stderr:165550K .......... .......... .......... .......... .......... 98% 7.31M 0s 2024-03-30T13:13:03.049 INFO:teuthology.orchestra.run.smithi043.stderr:165600K .......... .......... .......... .......... .......... 98% 8.58M 0s 2024-03-30T13:13:03.054 INFO:teuthology.orchestra.run.smithi043.stderr:165650K .......... .......... .......... .......... .......... 98% 8.94M 0s 2024-03-30T13:13:03.060 INFO:teuthology.orchestra.run.smithi043.stderr:165700K .......... .......... .......... .......... .......... 98% 8.48M 0s 2024-03-30T13:13:03.067 INFO:teuthology.orchestra.run.smithi043.stderr:165750K .......... .......... .......... .......... .......... 98% 7.18M 0s 2024-03-30T13:13:03.073 INFO:teuthology.orchestra.run.smithi043.stderr:165800K .......... .......... .......... .......... .......... 98% 7.74M 0s 2024-03-30T13:13:03.079 INFO:teuthology.orchestra.run.smithi043.stderr:165850K .......... .......... .......... .......... .......... 98% 8.44M 0s 2024-03-30T13:13:03.085 INFO:teuthology.orchestra.run.smithi043.stderr:165900K .......... .......... .......... .......... .......... 98% 8.28M 0s 2024-03-30T13:13:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:165950K .......... .......... .......... .......... .......... 98% 7.25M 0s 2024-03-30T13:13:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:166000K .......... .......... .......... .......... .......... 98% 7.98M 0s 2024-03-30T13:13:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:166050K .......... .......... .......... .......... .......... 98% 7.27M 0s 2024-03-30T13:13:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:166100K .......... .......... .......... .......... .......... 98% 7.31M 0s 2024-03-30T13:13:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:166150K .......... .......... .......... .......... .......... 98% 6.16M 0s 2024-03-30T13:13:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:166200K .......... .......... .......... .......... .......... 98% 7.27M 0s 2024-03-30T13:13:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:166250K .......... .......... .......... .......... .......... 98% 7.65M 0s 2024-03-30T13:13:03.139 INFO:teuthology.orchestra.run.smithi043.stderr:166300K .......... .......... .......... .......... .......... 98% 7.33M 0s 2024-03-30T13:13:03.146 INFO:teuthology.orchestra.run.smithi043.stderr:166350K .......... .......... .......... .......... .......... 98% 6.38M 0s 2024-03-30T13:13:03.152 INFO:teuthology.orchestra.run.smithi043.stderr:166400K .......... .......... .......... .......... .......... 98% 8.32M 0s 2024-03-30T13:13:03.158 INFO:teuthology.orchestra.run.smithi043.stderr:166450K .......... .......... .......... .......... .......... 98% 9.31M 0s 2024-03-30T13:13:03.163 INFO:teuthology.orchestra.run.smithi043.stderr:166500K .......... .......... .......... .......... .......... 98% 8.44M 0s 2024-03-30T13:13:03.170 INFO:teuthology.orchestra.run.smithi043.stderr:166550K .......... .......... .......... .......... .......... 98% 7.27M 0s 2024-03-30T13:13:03.176 INFO:teuthology.orchestra.run.smithi043.stderr:166600K .......... .......... .......... .......... .......... 98% 7.71M 0s 2024-03-30T13:13:03.182 INFO:teuthology.orchestra.run.smithi043.stderr:166650K .......... .......... .......... .......... .......... 98% 8.30M 0s 2024-03-30T13:13:03.188 INFO:teuthology.orchestra.run.smithi043.stderr:166700K .......... .......... .......... .......... .......... 98% 8.06M 0s 2024-03-30T13:13:03.196 INFO:teuthology.orchestra.run.smithi043.stderr:166750K .......... .......... .......... .......... .......... 98% 6.82M 0s 2024-03-30T13:13:03.202 INFO:teuthology.orchestra.run.smithi043.stderr:166800K .......... .......... .......... .......... .......... 98% 8.06M 0s 2024-03-30T13:13:03.208 INFO:teuthology.orchestra.run.smithi043.stderr:166850K .......... .......... .......... .......... .......... 99% 7.76M 0s 2024-03-30T13:13:03.214 INFO:teuthology.orchestra.run.smithi043.stderr:166900K .......... .......... .......... .......... .......... 99% 7.82M 0s 2024-03-30T13:13:03.222 INFO:teuthology.orchestra.run.smithi043.stderr:166950K .......... .......... .......... .......... .......... 99% 6.06M 0s 2024-03-30T13:13:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:167000K .......... .......... .......... .......... .......... 99% 6.95M 0s 2024-03-30T13:13:03.236 INFO:teuthology.orchestra.run.smithi043.stderr:167050K .......... .......... .......... .......... .......... 99% 6.91M 0s 2024-03-30T13:13:03.243 INFO:teuthology.orchestra.run.smithi043.stderr:167100K .......... .......... .......... .......... .......... 99% 6.91M 0s 2024-03-30T13:13:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:167150K .......... .......... .......... .......... .......... 99% 5.88M 0s 2024-03-30T13:13:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:167200K .......... .......... .......... .......... .......... 99% 6.86M 0s 2024-03-30T13:13:03.266 INFO:teuthology.orchestra.run.smithi043.stderr:167250K .......... .......... .......... .......... .......... 99% 6.97M 0s 2024-03-30T13:13:03.273 INFO:teuthology.orchestra.run.smithi043.stderr:167300K .......... .......... .......... .......... .......... 99% 6.61M 0s 2024-03-30T13:13:03.282 INFO:teuthology.orchestra.run.smithi043.stderr:167350K .......... .......... .......... .......... .......... 99% 5.60M 0s 2024-03-30T13:13:03.290 INFO:teuthology.orchestra.run.smithi043.stderr:167400K .......... .......... .......... .......... .......... 99% 6.47M 0s 2024-03-30T13:13:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:167450K .......... .......... .......... .......... .......... 99% 6.57M 0s 2024-03-30T13:13:03.305 INFO:teuthology.orchestra.run.smithi043.stderr:167500K .......... .......... .......... .......... .......... 99% 6.48M 0s 2024-03-30T13:13:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:167550K .......... .......... .......... .......... .......... 99% 5.96M 0s 2024-03-30T13:13:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:167600K .......... .......... .......... .......... .......... 99% 7.22M 0s 2024-03-30T13:13:03.326 INFO:teuthology.orchestra.run.smithi043.stderr:167650K .......... .......... .......... .......... .......... 99% 7.22M 0s 2024-03-30T13:13:03.333 INFO:teuthology.orchestra.run.smithi043.stderr:167700K .......... .......... .......... .......... .......... 99% 6.63M 0s 2024-03-30T13:13:03.342 INFO:teuthology.orchestra.run.smithi043.stderr:167750K .......... .......... .......... .......... .......... 99% 5.83M 0s 2024-03-30T13:13:03.349 INFO:teuthology.orchestra.run.smithi043.stderr:167800K .......... .......... .......... .......... .......... 99% 6.75M 0s 2024-03-30T13:13:03.356 INFO:teuthology.orchestra.run.smithi043.stderr:167850K .......... .......... .......... .......... .......... 99% 6.78M 0s 2024-03-30T13:13:03.363 INFO:teuthology.orchestra.run.smithi043.stderr:167900K .......... .......... .......... .......... .......... 99% 7.17M 0s 2024-03-30T13:13:03.370 INFO:teuthology.orchestra.run.smithi043.stderr:167950K .......... .......... .......... .......... .......... 99% 7.17M 0s 2024-03-30T13:13:03.376 INFO:teuthology.orchestra.run.smithi043.stderr:168000K .......... .......... .......... .......... .......... 99% 8.05M 0s 2024-03-30T13:13:03.381 INFO:teuthology.orchestra.run.smithi043.stderr:168050K .......... .......... .......... .......... .......... 99% 9.64M 0s 2024-03-30T13:13:03.385 INFO:teuthology.orchestra.run.smithi043.stderr:168100K .......... .......... .......... .......... .......... 99% 10.2M 0s 2024-03-30T13:13:03.390 INFO:teuthology.orchestra.run.smithi043.stderr:168150K .......... .......... .......... .......... .......... 99% 9.98M 0s 2024-03-30T13:13:03.395 INFO:teuthology.orchestra.run.smithi043.stderr:168200K .......... .......... .......... .......... .......... 99% 11.3M 0s 2024-03-30T13:13:03.399 INFO:teuthology.orchestra.run.smithi043.stderr:168250K .......... .......... .......... .......... .......... 99% 11.4M 0s 2024-03-30T13:13:03.403 INFO:teuthology.orchestra.run.smithi043.stderr:168300K .......... .......... .......... .......... .......... 99% 11.9M 0s 2024-03-30T13:13:03.408 INFO:teuthology.orchestra.run.smithi043.stderr:168350K .......... .......... .......... .......... .......... 99% 9.81M 0s 2024-03-30T13:13:03.412 INFO:teuthology.orchestra.run.smithi043.stderr:168400K .......... .......... .......... .......... .......... 99% 11.5M 0s 2024-03-30T13:13:03.417 INFO:teuthology.orchestra.run.smithi043.stderr:168450K .......... .......... .......... .......... .......... 99% 9.54M 0s 2024-03-30T13:13:03.423 INFO:teuthology.orchestra.run.smithi043.stderr:168500K .......... .......... .......... .......... .......... 99% 9.11M 0s 2024-03-30T13:13:03.426 INFO:teuthology.orchestra.run.smithi043.stderr:168550K .......... .......... . 100% 7.57M=19s 2024-03-30T13:13:03.426 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:13:03.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30 13:13:03 (8.57 MB/s) - 'linux-5.4.tar.gz' saved [172616875/172616875] 2024-03-30T13:13:03.429 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:13:03.429 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-30T13:13:03.429 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-30T13:13:03.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-30T13:13:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.606+0000 7f3204f4d640 1 Processor -- start 2024-03-30T13:13:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.606+0000 7f3204f4d640 1 -- start start 2024-03-30T13:13:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3200070270 0x7f3200161f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3200162de0 con 0x7f32000ce2c0 2024-03-30T13:13:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3200162540 con 0x7f32000c9a00 2024-03-30T13:13:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3200162820 con 0x7f3200070270 2024-03-30T13:13:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fdd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3200070270 0x7f3200161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f32000ce2c0 0x7f320015efa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55076/0 (socket says 172.21.15.43:55076) 2024-03-30T13:13:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fed9d640 1 -- 172.21.15.43:0/3850549960 learned_addr learned my addr 172.21.15.43:0/3850549960 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fdd9b640 1 -- 172.21.15.43:0/3850549960 >> v1:172.21.15.113:6789/0 conn(0x7f32000c9a00 legacy=0x7f32000c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fdd9b640 1 -- 172.21.15.43:0/3850549960 >> v1:172.21.15.43:6789/0 conn(0x7f32000ce2c0 legacy=0x7f320015efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fdd9b640 1 -- 172.21.15.43:0/3850549960 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3200162b00 con 0x7f3200070270 2024-03-30T13:13:03.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fd59a640 1 -- 172.21.15.43:0/3850549960 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346755611 0 0) 0x7f3200162de0 con 0x7f32000ce2c0 2024-03-30T13:13:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fdd9b640 1 --2- 172.21.15.43:0/3850549960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f3200161f60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f31f40097c0 tx=0x7f31f401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a19db6da6c4cf5a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fd59a640 1 -- 172.21.15.43:0/3850549960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f40aa5f0 con 0x7f3200070270 2024-03-30T13:13:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fd59a640 1 -- 172.21.15.43:0/3850549960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f40af070 con 0x7f3200070270 2024-03-30T13:13:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f31fd59a640 1 -- 172.21.15.43:0/3850549960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f40aa9b0 con 0x7f3200070270 2024-03-30T13:13:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 -- 172.21.15.43:0/3850549960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 msgr2=0x7f3200161f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 --2- 172.21.15.43:0/3850549960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f3200161f60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f31f40097c0 tx=0x7f31f401c470 comp rx=0 tx=0).stop 2024-03-30T13:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 -- 172.21.15.43:0/3850549960 shutdown_connections 2024-03-30T13:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 --2- 172.21.15.43:0/3850549960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f3200161f60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 -- 172.21.15.43:0/3850549960 >> 172.21.15.43:0/3850549960 conn(0x7f32000cd170 msgr2=0x7f32000cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.610+0000 7f3204f4d640 1 -- 172.21.15.43:0/3850549960 shutdown_connections 2024-03-30T13:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- 172.21.15.43:0/3850549960 wait complete. 2024-03-30T13:13:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 Processor -- start 2024-03-30T13:13:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- start start 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f320014d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000c9a00 0x7f32001613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32000ce2c0 0x7f3200161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f320016cb20 con 0x7f3200070270 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f320016cca0 con 0x7f32000c9a00 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f320016ce20 con 0x7f32000ce2c0 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f320014d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f320014d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51088/0 (socket says 172.21.15.43:51088) 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 -- 172.21.15.43:0/282288910 learned_addr learned my addr 172.21.15.43:0/282288910 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fdd9b640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000c9a00 0x7f32001613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 -- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32000ce2c0 msgr2=0x7f3200161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32000ce2c0 0x7f3200161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 -- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000c9a00 msgr2=0x7f32001613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000c9a00 0x7f32001613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 -- 172.21.15.43:0/282288910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320014d8a0 con 0x7f3200070270 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fdd9b640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000c9a00 0x7f32001613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31fe59c640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f320014d360 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f31f000c530 tx=0x7f31f000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31eeffd640 1 -- 172.21.15.43:0/282288910 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0016020 con 0x7f3200070270 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31eeffd640 1 -- 172.21.15.43:0/282288910 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f0005220 con 0x7f3200070270 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320015fac0 con 0x7f3200070270 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f31eeffd640 1 -- 172.21.15.43:0/282288910 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0007030 con 0x7f3200070270 2024-03-30T13:13:03.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3200160080 con 0x7f3200070270 2024-03-30T13:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.614+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3200068900 con 0x7f3200070270 2024-03-30T13:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.618+0000 7f31eeffd640 1 -- 172.21.15.43:0/282288910 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f31f00071d0 con 0x7f3200070270 2024-03-30T13:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.618+0000 7f31eeffd640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f31e0067fc0 0x7f31e006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.618+0000 7f31eeffd640 1 -- 172.21.15.43:0/282288910 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f31f0086010 con 0x7f3200070270 2024-03-30T13:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.618+0000 7f31fdd9b640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f31e0067fc0 0x7f31e006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.618+0000 7f31fdd9b640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f31e0067fc0 0x7f31e006a480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31f40ae000 tx=0x7f31f4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:03.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.622+0000 7f31eeffd640 1 -- 172.21.15.43:0/282288910 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f31f0020020 con 0x7f3200070270 2024-03-30T13:13:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:03.818+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f320015b3e0 con 0x7f3200070270 2024-03-30T13:13:04.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.142+0000 7f31eeffd640 1 -- 172.21.15.43:0/282288910 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v20) ==== 105+0+0 (secure 0 0 0) 0x7f31f00074c0 con 0x7f3200070270 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f31e0067fc0 msgr2=0x7f31e006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f31e0067fc0 0x7f31e006a480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f31f40ae000 tx=0x7f31f4008040 comp rx=0 tx=0).stop 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 msgr2=0x7f320014d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f320014d360 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f31f000c530 tx=0x7f31f000c9f0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 shutdown_connections 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f31e0067fc0 0x7f31e006a480 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32000ce2c0 0x7f3200161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f32000c9a00 0x7f32001613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 --2- 172.21.15.43:0/282288910 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3200070270 0x7f320014d360 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 >> 172.21.15.43:0/282288910 conn(0x7f32000cd170 msgr2=0x7f32000b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 shutdown_connections 2024-03-30T13:13:04.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:04.146+0000 7f3204f4d640 1 -- 172.21.15.43:0/282288910 wait complete. 2024-03-30T13:13:04.174 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-30T13:13:04.174 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-30T13:13:04.174 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-30T13:13:04.174 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-30T13:13:15.272 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-30T13:13:15.273 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-30T13:13:15.334 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph 2024-03-30T13:13:15.334 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2 2024-03-30T13:13:15.334 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs 2024-03-30T13:13:15.334 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-30T13:13:15.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:15.335 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-30T13:13:15.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:15.346 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-30T13:13:15.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:15.399 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-30T13:13:15.450 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-30T13:13:15.450 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-30T13:13:15.588 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:13:15.588 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/Makefile 2024-03-30T13:13:15.588 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:13:15.588 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/io.h 2024-03-30T13:13:15.588 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/Makefile 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/acl.h 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:13:15.589 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:13:15.590 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:13:15.590 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:13:15.590 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:13:15.590 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:13:15.590 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:13:15.590 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-30T13:13:15.590 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-30T13:13:15.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:15.590 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-30T13:13:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7face4fb6640 1 Processor -- start 2024-03-30T13:13:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7face4fb6640 1 -- start start 2024-03-30T13:13:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7face4fb6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7face006cb60 0x7face015f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7face4fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face00ae2f0 0x7face00ae740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7face4fb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face015f890 con 0x7face00b5940 2024-03-30T13:13:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7face4fb6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7face00aec80 con 0x7face006cb60 2024-03-30T13:13:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7face4fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7face0162320 con 0x7face00ae2f0 2024-03-30T13:13:15.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facde59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7face006cb60 0x7face015f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facde59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7face006cb60 0x7face015f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40770/0 (socket says 172.21.15.43:40770) 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facde59c640 1 -- 172.21.15.43:0/1335457718 learned_addr learned my addr 172.21.15.43:0/1335457718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facded9d640 1 --2- 172.21.15.43:0/1335457718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face00ae2f0 0x7face00ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facde59c640 1 -- 172.21.15.43:0/1335457718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face00ae2f0 msgr2=0x7face00ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facde59c640 1 --2- 172.21.15.43:0/1335457718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face00ae2f0 0x7face00ae740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facde59c640 1 -- 172.21.15.43:0/1335457718 >> v1:172.21.15.113:6789/0 conn(0x7face00b5940 legacy=0x7face01627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facde59c640 1 -- 172.21.15.43:0/1335457718 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face00ada10 con 0x7face006cb60 2024-03-30T13:13:15.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.718+0000 7facded9d640 1 --2- 172.21.15.43:0/1335457718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face00ae2f0 0x7face00ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7facde59c640 1 --2- 172.21.15.43:0/1335457718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face006cb60 0x7face015f350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7facd0009870 tx=0x7facd001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56515379913aacc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7facdd59a640 1 -- 172.21.15.43:0/1335457718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd00b0070 con 0x7face006cb60 2024-03-30T13:13:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7facdd59a640 1 -- 172.21.15.43:0/1335457718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd00ab600 con 0x7face006cb60 2024-03-30T13:13:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7facdd59a640 1 -- 172.21.15.43:0/1335457718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd00ab9d0 con 0x7face006cb60 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 -- 172.21.15.43:0/1335457718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face006cb60 msgr2=0x7face015f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 --2- 172.21.15.43:0/1335457718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face006cb60 0x7face015f350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7facd0009870 tx=0x7facd001c440 comp rx=0 tx=0).stop 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 -- 172.21.15.43:0/1335457718 shutdown_connections 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 --2- 172.21.15.43:0/1335457718 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face00ae2f0 0x7face00ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 --2- 172.21.15.43:0/1335457718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face006cb60 0x7face015f350 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 -- 172.21.15.43:0/1335457718 >> 172.21.15.43:0/1335457718 conn(0x7face00d08e0 msgr2=0x7face00d0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 -- 172.21.15.43:0/1335457718 shutdown_connections 2024-03-30T13:13:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.722+0000 7face4fb6640 1 -- 172.21.15.43:0/1335457718 wait complete. 2024-03-30T13:13:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 Processor -- start 2024-03-30T13:13:15.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 -- start start 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face006cb60 0x7face0163980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face00ae2f0 0x7face014e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face00b5940 0x7face014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7face006be70 con 0x7face00ae2f0 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7face006cf40 con 0x7face00b5940 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7face006bff0 con 0x7face006cb60 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face00ae2f0 0x7face014e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face00ae2f0 0x7face014e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40772/0 (socket says 172.21.15.43:40772) 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 -- 172.21.15.43:0/4290664967 learned_addr learned my addr 172.21.15.43:0/4290664967 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facde59c640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face006cb60 0x7face0163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facded9d640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face00b5940 0x7face014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 -- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face006cb60 msgr2=0x7face0163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face006cb60 0x7face0163980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 -- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face00b5940 msgr2=0x7face014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face00b5940 0x7face014e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 -- 172.21.15.43:0/4290664967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face006f240 con 0x7face00ae2f0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facddd9b640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face00ae2f0 0x7face014e4b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7facd400c340 tx=0x7facd400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facceffd640 1 -- 172.21.15.43:0/4290664967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd40052d0 con 0x7face00ae2f0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facceffd640 1 -- 172.21.15.43:0/4290664967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4005470 con 0x7face00ae2f0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face0156540 con 0x7face00ae2f0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7facceffd640 1 -- 172.21.15.43:0/4290664967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4007260 con 0x7face00ae2f0 2024-03-30T13:13:15.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.726+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face0156b00 con 0x7face00ae2f0 2024-03-30T13:13:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facceffd640 1 -- 172.21.15.43:0/4290664967 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7facd4007400 con 0x7face00ae2f0 2024-03-30T13:13:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facceffd640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7facc0067fc0 0x7facc006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facceffd640 1 -- 172.21.15.43:0/4290664967 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7facd4085b30 con 0x7face00ae2f0 2024-03-30T13:13:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facde59c640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7facc0067fc0 0x7facc006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 Processor -- start 2024-03-30T13:13:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- start start 2024-03-30T13:13:15.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facde59c640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7facc0067fc0 0x7facc006a480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7facd00ad000 tx=0x7facd0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa000 0x55e73cdfa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdfa920 0x55e73cdb0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdb0850 0x55e73cdb0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e73cbfec60 con 0x55e73cdb0850 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e73cb9bb80 con 0x55e73cdfa920 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e73cbfac10 con 0x55e73cdfa000 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdb0850 0x55e73cdb0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdb0850 0x55e73cdb0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40774/0 (socket says 172.21.15.43:40774) 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 -- 172.21.15.43:0/3462221108 learned_addr learned my addr 172.21.15.43:0/3462221108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facde59c640 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa000 0x55e73cdfa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facddd9b640 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdfa920 0x55e73cdb0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 -- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa000 msgr2=0x55e73cdfa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa000 0x55e73cdfa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 -- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdfa920 msgr2=0x55e73cdb0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdfa920 0x55e73cdb0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 -- 172.21.15.43:0/3462221108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73cca5f10 con 0x55e73cdb0850 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facddd9b640 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdfa920 0x55e73cdb0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facded9d640 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdb0850 0x55e73cdb0c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7facd800f470 tx=0x7facd800f930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b388941acbfd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facbffff640 1 -- 172.21.15.43:0/3462221108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd801c070 con 0x55e73cdb0850 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facbffff640 1 -- 172.21.15.43:0/3462221108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd800d700 con 0x55e73cdb0850 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7facbffff640 1 -- 172.21.15.43:0/3462221108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd800dab0 con 0x55e73cdb0850 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- 172.21.15.43:0/3462221108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdb0850 msgr2=0x55e73cdb0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdb0850 0x55e73cdb0c30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7facd800f470 tx=0x7facd800f930 comp rx=0 tx=0).stop 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- 172.21.15.43:0/3462221108 shutdown_connections 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdb0850 0x55e73cdb0c30 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdfa920 0x55e73cdb0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 --2- 172.21.15.43:0/3462221108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa000 0x55e73cdfa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- 172.21.15.43:0/3462221108 >> 172.21.15.43:0/3462221108 conn(0x55e73cbabbd0 msgr2=0x55e73cdaebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- 172.21.15.43:0/3462221108 shutdown_connections 2024-03-30T13:13:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.730+0000 7face8138000 1 -- 172.21.15.43:0/3462221108 wait complete. 2024-03-30T13:13:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 -- start start 2024-03-30T13:13:15.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 Processor -- start 2024-03-30T13:13:15.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdb0850 0x55e73cdb4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73cdde400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa920 0x55e73cdde940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e73cbbc2a0 con 0x55e73cdfa000 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e73cbbd760 con 0x55e73cdb0850 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e73cbbfee0 con 0x55e73cdfa920 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 10 client.0 _ll_register_callbacks cb 0x55e73cd9baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73cdde400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73cdde400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40788/0 (socket says 172.21.15.43:40788) 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 -- 172.21.15.43:0/616656379 learned_addr learned my addr 172.21.15.43:0/616656379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facded9d640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa920 0x55e73cdde940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facde59c640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdb0850 0x55e73cdb4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 -- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa920 msgr2=0x55e73cdde940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa920 0x55e73cdde940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 -- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdb0850 msgr2=0x55e73cdb4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdb0850 0x55e73cdb4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 -- 172.21.15.43:0/616656379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73cbd88d0 con 0x55e73cdfa000 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facded9d640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa920 0x55e73cdde940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facddd9b640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73cdde400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7facd400a040 tx=0x7facd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a363d8c756a68 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facbd7fa640 1 -- 172.21.15.43:0/616656379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd405c0b0 con 0x55e73cdfa000 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facbd7fa640 1 -- 172.21.15.43:0/616656379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7facd405c250 con 0x55e73cdfa000 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 -- 172.21.15.43:0/616656379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd800d700 con 0x55e73cdfa000 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facbd7fa640 1 -- 172.21.15.43:0/616656379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7facd4054ea0 con 0x55e73cdfa000 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 10 client.5793 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 -- 172.21.15.43:0/616656379 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55e73cc440d0 con 0x55e73cdfa000 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facbd7fa640 1 -- 172.21.15.43:0/616656379 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd800d700 con 0x55e73cdfa000 2024-03-30T13:13:15.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facbd7fa640 1 -- 172.21.15.43:0/616656379 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7facd4055040 con 0x55e73cdfa000 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 10 client.5793 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 10 client.5793 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 20 client.5793 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73ce00640 0x55e73ce00a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 4 client.5793 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7face8138000 1 -- 172.21.15.43:0/616656379 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e73cb99f70 con 0x55e73ce00640 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facde59c640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73ce00640 0x55e73ce00a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facbd7fa640 10 client.5793 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.734+0000 7facde59c640 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73ce00640 0x55e73ce00a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f9aef9eab99a3 server_cookie=eed0e3216c9f9349 in_seq=0 out_seq=0 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7facbd7fa640 1 -- 172.21.15.43:0/616656379 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55e73cb99f70 con 0x55e73ce00640 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7facbd7fa640 10 client.5793 handle_command_reply: tid=0 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 client.5793 shutdown 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 -- 172.21.15.43:0/616656379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 msgr2=0x55e73cdde400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73cdde400 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7facd400a040 tx=0x7facd400cf90 comp rx=0 tx=0).stop 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 -- 172.21.15.43:0/616656379 shutdown_connections 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73ce00640 0x55e73ce00a20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdfa920 0x55e73cdde940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73cdde400 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7facbd7fa640 0 client.5793 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 --2- 172.21.15.43:0/616656379 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cdb0850 0x55e73cdb4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 -- 172.21.15.43:0/616656379 >> 172.21.15.43:0/616656379 conn(0x55e73cbabbd0 msgr2=0x55e73cdaebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 -- 172.21.15.43:0/616656379 shutdown_connections 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 1 -- 172.21.15.43:0/616656379 wait complete. 2024-03-30T13:13:15.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.738+0000 7face8138000 20 client.5793 trim_cache size 0 max 16384 2024-03-30T13:13:15.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.754+0000 7face8138000 1 Processor -- start 2024-03-30T13:13:15.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.754+0000 7face8138000 1 -- start start 2024-03-30T13:13:15.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73ce06e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce10770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce10cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7face0202f70 con 0x55e73cdfa000 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7face006ead0 con 0x55e73cbabbd0 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7face0202a00 con 0x55e73cdb0850 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce10cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce10cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40804/0 (socket says 172.21.15.43:40804) 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/820047345 learned_addr learned my addr 172.21.15.43:0/820047345 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facddd9b640 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce10770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facde59c640 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73ce06e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 msgr2=0x55e73ce10770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce10770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 msgr2=0x55e73ce06e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73ce06e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/820047345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e73cbcc410 con 0x55e73cdfa000 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facde59c640 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73ce06e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce10cb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7facd8011010 tx=0x7facd8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809b519a27a9b3fa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facbf7fe640 1 -- 172.21.15.43:0/820047345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd801c040 con 0x55e73cdfa000 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facbf7fe640 1 -- 172.21.15.43:0/820047345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd8027070 con 0x55e73cdfa000 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facbf7fe640 1 -- 172.21.15.43:0/820047345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd800db40 con 0x55e73cdfa000 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- 172.21.15.43:0/820047345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 msgr2=0x55e73ce10cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce10cb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7facd8011010 tx=0x7facd8012000 comp rx=0 tx=0).stop 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- 172.21.15.43:0/820047345 shutdown_connections 2024-03-30T13:13:15.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce10cb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce10770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- 172.21.15.43:0/820047345 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73ce06e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- 172.21.15.43:0/820047345 >> 172.21.15.43:0/820047345 conn(0x55e73cddd690 msgr2=0x55e73cdfed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- 172.21.15.43:0/820047345 shutdown_connections 2024-03-30T13:13:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- 172.21.15.43:0/820047345 wait complete. 2024-03-30T13:13:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- start start 2024-03-30T13:13:15.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 Processor -- start 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73cdaf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce02a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce02f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e73cc03290 con 0x55e73cdfa000 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e73cbc4370 con 0x55e73cbabbd0 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e73cb862c0 con 0x55e73cdb0850 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 10 client.0 _ll_register_callbacks cb 0x7facd00ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce02f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce02f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40808/0 (socket says 172.21.15.43:40808) 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/1718576711 learned_addr learned my addr 172.21.15.43:0/1718576711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facde59c640 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73cdaf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facddd9b640 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce02a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 msgr2=0x55e73ce02a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce02a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 msgr2=0x55e73cdaf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73cdaf7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 -- 172.21.15.43:0/1718576711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e73cc801e0 con 0x55e73cdfa000 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facded9d640 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce02f70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7facd8011010 tx=0x7facd8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb1befc801e32a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facbd7fa640 1 -- 172.21.15.43:0/1718576711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd801c030 con 0x55e73cdfa000 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7facbd7fa640 1 -- 172.21.15.43:0/1718576711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7facd8022480 con 0x55e73cdfa000 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.758+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd8027070 con 0x55e73cdfa000 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facbd7fa640 1 -- 172.21.15.43:0/1718576711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7facd803e050 con 0x55e73cdfa000 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 10 client.5799 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55e73c8ba3b0 con 0x55e73cdfa000 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facbd7fa640 1 -- 172.21.15.43:0/1718576711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd8027070 con 0x55e73cdfa000 2024-03-30T13:13:15.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facbd7fa640 1 -- 172.21.15.43:0/1718576711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7facd8041020 con 0x55e73cdfa000 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 10 client.5799 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 10 client.5799 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 20 client.5799 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73cdfa920 0x55e73ce034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 4 client.5799 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55e73cc0c1a0 con 0x55e73cdfa920 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facde59c640 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73cdfa920 0x55e73ce034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facde59c640 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73cdfa920 0x55e73ce034b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c3a4e213f11b81 server_cookie=7d031b8c3a00a497 in_seq=0 out_seq=0 2024-03-30T13:13:15.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facbd7fa640 10 client.5799 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facbd7fa640 1 -- 172.21.15.43:0/1718576711 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e73cc0c1a0 con 0x55e73cdfa920 2024-03-30T13:13:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facbd7fa640 10 client.5799 handle_command_reply: tid=0 2024-03-30T13:13:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 client.5799 shutdown 2024-03-30T13:13:15.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 msgr2=0x55e73ce02f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce02f70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7facd8011010 tx=0x7facd8004ac0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 shutdown_connections 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e73cdfa920 0x55e73ce034b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e73cdfa000 0x55e73ce02f70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e73cdb0850 0x55e73ce02a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 --2- 172.21.15.43:0/1718576711 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e73cbabbd0 0x55e73cdaf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7facbd7fa640 0 client.5799 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.762+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 >> 172.21.15.43:0/1718576711 conn(0x55e73cddd690 msgr2=0x55e73cdff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 shutdown_connections 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face8138000 1 -- 172.21.15.43:0/1718576711 wait complete. 2024-03-30T13:13:15.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face8138000 20 client.5799 trim_cache size 0 max 16384 2024-03-30T13:13:15.773 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:15.773 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-03-30T13:13:15.773 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "9ec31cb9-756a-4ece-b87a-bdc758385b7c", 2024-03-30T13:13:15.773 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-03-30T13:13:15.773 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7facc0067fc0 msgr2=0x7facc006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7facc0067fc0 0x7facc006a480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7facd00ad000 tx=0x7facd0008040 comp rx=0 tx=0).stop 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face00ae2f0 msgr2=0x7face014e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face00ae2f0 0x7face014e4b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7facd400c340 tx=0x7facd400c800 comp rx=0 tx=0).stop 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 shutdown_connections 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7facc0067fc0 0x7facc006a480 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7face00b5940 0x7face014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7face00ae2f0 0x7face014e4b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 --2- 172.21.15.43:0/4290664967 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7face006cb60 0x7face0163980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 >> 172.21.15.43:0/4290664967 conn(0x7face00d08e0 msgr2=0x7face01550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 shutdown_connections 2024-03-30T13:13:15.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.766+0000 7face4fb6640 1 -- 172.21.15.43:0/4290664967 wait complete. 2024-03-30T13:13:15.787 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': '9ec31cb9-756a-4ece-b87a-bdc758385b7c', 'mode': 'asynchronous'} 2024-03-30T13:13:15.787 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-30T13:13:15.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:15.787 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:13:15.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b7ae37640 1 Processor -- start 2024-03-30T13:13:15.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b7ae37640 1 -- start start 2024-03-30T13:13:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b7ae37640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b740b9e30 0x7f2b740cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b7ae37640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7406c850 con 0x7f2b740cf4f0 2024-03-30T13:13:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b7ae37640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b7406cb30 con 0x7f2b7415e4c0 2024-03-30T13:13:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b7ae37640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2b740ba290 con 0x7f2b740b9e30 2024-03-30T13:13:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b793d4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2b740cf4f0 0x7f2b740d2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41446/0 (socket says 172.21.15.43:41446) 2024-03-30T13:13:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b793d4640 1 -- 172.21.15.43:0/1029758761 learned_addr learned my addr 172.21.15.43:0/1029758761 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b73fff640 1 --2- 172.21.15.43:0/1029758761 >> v2:172.21.15.113:3300/0 conn(0x7f2b740b9e30 0x7f2b740cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 638167895 0 0) 0x7f2b7406c850 con 0x7f2b740cf4f0 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b64003530 con 0x7f2b740cf4f0 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1990979720 0 0) 0x7f2b7406cb30 con 0x7f2b7415e4c0 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b7406c850 con 0x7f2b7415e4c0 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4081459197 0 0) 0x7f2b64003530 con 0x7f2b740cf4f0 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b73fff640 1 -- 172.21.15.43:0/1029758761 >> v1:172.21.15.113:6789/0 conn(0x7f2b7415e4c0 legacy=0x7f2b7406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b73fff640 1 -- 172.21.15.43:0/1029758761 >> v1:172.21.15.43:6789/0 conn(0x7f2b740cf4f0 legacy=0x7f2b740d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.910+0000 7f2b73fff640 1 -- 172.21.15.43:0/1029758761 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7406ce10 con 0x7f2b740b9e30 2024-03-30T13:13:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b73fff640 1 --2- 172.21.15.43:0/1029758761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b9e30 0x7f2b740cefb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2b600097c0 tx=0x7f2b6001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7e3eea62b1e4c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600b0070 con 0x7f2b740b9e30 2024-03-30T13:13:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b600ab400 con 0x7f2b740b9e30 2024-03-30T13:13:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b737fe640 1 -- 172.21.15.43:0/1029758761 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c001a00 con 0x7f2b740b9e30 2024-03-30T13:13:15.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- 172.21.15.43:0/1029758761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b9e30 msgr2=0x7f2b740cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/1029758761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b9e30 0x7f2b740cefb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2b600097c0 tx=0x7f2b6001c3c0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- 172.21.15.43:0/1029758761 shutdown_connections 2024-03-30T13:13:15.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/1029758761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740b9e30 0x7f2b740cefb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- 172.21.15.43:0/1029758761 >> 172.21.15.43:0/1029758761 conn(0x7f2b740b6410 msgr2=0x7f2b740d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- 172.21.15.43:0/1029758761 shutdown_connections 2024-03-30T13:13:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- 172.21.15.43:0/1029758761 wait complete. 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 Processor -- start 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- start start 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b740b9e30 0x7f2b741596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740cf4f0 0x7f2b74150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b7415e4c0 0x7f2b740cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7415e8a0 con 0x7f2b7415e4c0 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b740d3f20 con 0x7f2b740cf4f0 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.914+0000 7f2b7ae37640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740cf940 con 0x7f2b740b9e30 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b7415e4c0 0x7f2b740cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b7415e4c0 0x7f2b740cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40810/0 (socket says 172.21.15.43:40810) 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 -- 172.21.15.43:0/3887135902 learned_addr learned my addr 172.21.15.43:0/3887135902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b73fff640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740cf4f0 0x7f2b74150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b740b9e30 0x7f2b741596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 -- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b740b9e30 msgr2=0x7f2b741596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b740b9e30 0x7f2b741596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 -- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740cf4f0 msgr2=0x7f2b74150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740cf4f0 0x7f2b74150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 -- 172.21.15.43:0/3887135902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7406c5e0 con 0x7f2b7415e4c0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b793d4640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b7415e4c0 0x7f2b740cd6f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c001f40 tx=0x7f2b6c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b717fa640 1 -- 172.21.15.43:0/3887135902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c017070 con 0x7f2b7415e4c0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b717fa640 1 -- 172.21.15.43:0/3887135902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c012400 con 0x7f2b7415e4c0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b740c9770 con 0x7f2b7415e4c0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b717fa640 1 -- 172.21.15.43:0/3887135902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c012830 con 0x7f2b7415e4c0 2024-03-30T13:13:15.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.918+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b740c9d30 con 0x7f2b7415e4c0 2024-03-30T13:13:15.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 Processor -- start 2024-03-30T13:13:15.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- start start 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495dd59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 0x562495d887a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495d890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562495c71e20 con 0x562495d88ce0 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562495bdab00 con 0x562495dd5f20 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x562495b77940 con 0x562495dd5600 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495d890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495d890c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40824/0 (socket says 172.21.15.43:40824) 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 -- 172.21.15.43:0/996445364 learned_addr learned my addr 172.21.15.43:0/996445364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b717fa640 1 -- 172.21.15.43:0/3887135902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2b6c01a070 con 0x7f2b7415e4c0 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495dd59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 -- 172.21.15.43:0/996445364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 msgr2=0x562495dd59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495dd59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 -- 172.21.15.43:0/996445364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 msgr2=0x562495d887a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 0x562495d887a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 -- 172.21.15.43:0/996445364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562495b60380 con 0x562495d88ce0 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b717fa640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b58067fc0 0x7f2b5806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b717fa640 1 -- 172.21.15.43:0/3887135902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2b6c0866a0 con 0x7f2b7415e4c0 2024-03-30T13:13:15.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b73fff640 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495d890c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b6001c400 tx=0x7f2b60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec6e59ced008743 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b58067fc0 0x7f2b5806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b5e7fc640 1 -- 172.21.15.43:0/996445364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600b0040 con 0x562495d88ce0 2024-03-30T13:13:15.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b5e7fc640 1 -- 172.21.15.43:0/996445364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b60003820 con 0x562495d88ce0 2024-03-30T13:13:15.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b5e7fc640 1 -- 172.21.15.43:0/996445364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60003bd0 con 0x562495d88ce0 2024-03-30T13:13:15.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/996445364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 msgr2=0x562495d890c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495d890c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2b6001c400 tx=0x7f2b60008040 comp rx=0 tx=0).stop 2024-03-30T13:13:15.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b58067fc0 0x7f2b5806a480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b680011f0 tx=0x7f2b68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/996445364 shutdown_connections 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495d890c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 0x562495d887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/996445364 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495dd59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/996445364 >> 172.21.15.43:0/996445364 conn(0x562495db63a0 msgr2=0x562495db8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/996445364 shutdown_connections 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.922+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/996445364 wait complete. 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 -- start start 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 Processor -- start 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495ddc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495ddcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 0x562495d8e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562495b981e0 con 0x562495d88ce0 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x562495bd6ab0 con 0x562495dd5f20 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x562495ba6150 con 0x562495dd5600 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495ddc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495ddc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40834/0 (socket says 172.21.15.43:40834) 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 -- 172.21.15.43:0/1036742845 learned_addr learned my addr 172.21.15.43:0/1036742845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b73fff640 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495ddcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b793d4640 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 0x562495d8e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 10 client.0 _ll_register_callbacks cb 0x562495d766a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 -- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 msgr2=0x562495ddcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495ddcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 -- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 msgr2=0x562495d8e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 0x562495d8e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 -- 172.21.15.43:0/1036742845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562495b96b70 con 0x562495d88ce0 2024-03-30T13:13:15.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495ddc8a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2b68003880 tx=0x7f2b68001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d83e32f05611883 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b537fe640 1 -- 172.21.15.43:0/1036742845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68003e00 con 0x562495d88ce0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b537fe640 1 -- 172.21.15.43:0/1036742845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2b68023c90 con 0x562495d88ce0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b60003820 con 0x562495d88ce0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b537fe640 1 -- 172.21.15.43:0/1036742845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2b6800ebc0 con 0x562495d88ce0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 10 client.5811 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x562495b77b90 con 0x562495d88ce0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b537fe640 1 -- 172.21.15.43:0/1036742845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b60003820 con 0x562495d88ce0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b537fe640 1 -- 172.21.15.43:0/1036742845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f2b6802a050 con 0x562495d88ce0 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 10 client.5811 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 10 client.5811 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 20 client.5811 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495d8f190 0x562495d8f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 4 client.5811 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:15.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562495b75dd0 con 0x562495d8f190 2024-03-30T13:13:15.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.926+0000 7f2b73fff640 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495d8f190 0x562495d8f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b537fe640 10 client.5811 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b73fff640 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495d8f190 0x562495d8f5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27a396462b66cf0 server_cookie=21c06704b655bab8 in_seq=0 out_seq=0 2024-03-30T13:13:15.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b537fe640 1 -- 172.21.15.43:0/1036742845 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x562495b75dd0 con 0x562495d8f190 2024-03-30T13:13:15.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b537fe640 10 client.5811 handle_command_reply: tid=0 2024-03-30T13:13:15.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 client.5811 shutdown 2024-03-30T13:13:15.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 msgr2=0x562495ddc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495ddc8a0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2b68003880 tx=0x7f2b68001160 comp rx=0 tx=0).stop 2024-03-30T13:13:15.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 shutdown_connections 2024-03-30T13:13:15.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495d8f190 0x562495d8f5a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b537fe640 0 client.5811 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5f20 0x562495d8e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495dd5600 0x562495ddcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/1036742845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d88ce0 0x562495ddc8a0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 >> 172.21.15.43:0/1036742845 conn(0x562495db63a0 msgr2=0x562495d87ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 shutdown_connections 2024-03-30T13:13:15.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/1036742845 wait complete. 2024-03-30T13:13:15.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.930+0000 7f2b7dfb9000 20 client.5811 trim_cache size 0 max 16384 2024-03-30T13:13:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.950+0000 7f2b7dfb9000 1 Processor -- start 2024-03-30T13:13:15.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.950+0000 7f2b7dfb9000 1 -- start start 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495d88ce0 0x562495d8e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d8f190 0x562495d8cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495dd5600 0x562495d8d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74203880 con 0x562495dd5600 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74202860 con 0x562495d88ce0 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b740cc010 con 0x562495d8f190 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495dd5600 0x562495d8d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495dd5600 0x562495d8d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40850/0 (socket says 172.21.15.43:40850) 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 -- 172.21.15.43:0/2400804798 learned_addr learned my addr 172.21.15.43:0/2400804798 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b73fff640 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d8f190 0x562495d8cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495d88ce0 0x562495d8e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 -- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d8f190 msgr2=0x562495d8cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d8f190 0x562495d8cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 -- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495d88ce0 msgr2=0x562495d8e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495d88ce0 0x562495d8e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 -- 172.21.15.43:0/2400804798 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562495dd5d50 con 0x562495dd5600 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b73fff640 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d8f190 0x562495d8cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b793d4640 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495dd5600 0x562495d8d4a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0562c0 tx=0x7f2b6c001ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3da3d5c0aa5807 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b5dffb640 1 -- 172.21.15.43:0/2400804798 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c052c30 con 0x562495dd5600 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b5dffb640 1 -- 172.21.15.43:0/2400804798 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c05ec60 con 0x562495dd5600 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b5dffb640 1 -- 172.21.15.43:0/2400804798 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c052dd0 con 0x562495dd5600 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/2400804798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495dd5600 msgr2=0x562495d8d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495dd5600 0x562495d8d4a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0562c0 tx=0x7f2b6c001ed0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/2400804798 shutdown_connections 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495dd5600 0x562495d8d4a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d8f190 0x562495d8cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/2400804798 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495d88ce0 0x562495d8e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/2400804798 >> 172.21.15.43:0/2400804798 conn(0x562495db63a0 msgr2=0x562495d8f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/2400804798 shutdown_connections 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/2400804798 wait complete. 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 -- start start 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.954+0000 7f2b7dfb9000 1 Processor -- start 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d88ce0 0x562495d8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d8f190 0x562495e433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5600 0x562495e438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x562495b91750 con 0x562495d8f190 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74202210 con 0x562495dd5600 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x562495ca9750 con 0x562495d88ce0 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 10 client.0 _ll_register_callbacks cb 0x7f2b680012c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d8f190 0x562495e433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d8f190 0x562495e433a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40866/0 (socket says 172.21.15.43:40866) 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 -- 172.21.15.43:0/781938412 learned_addr learned my addr 172.21.15.43:0/781938412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d88ce0 0x562495d8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b793d4640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5600 0x562495e438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 -- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d88ce0 msgr2=0x562495d8e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d88ce0 0x562495d8e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 -- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5600 msgr2=0x562495e438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:15.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5600 0x562495e438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 -- 172.21.15.43:0/781938412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562495c5b470 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b793d4640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5600 0x562495e438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b73fff640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d8f190 0x562495e433a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2b60018fd0 tx=0x7f2b6001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8be9d8191c237d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b537fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b537fe640 1 -- 172.21.15.43:0/781938412 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600b0050 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b6c05ec60 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b537fe640 1 -- 172.21.15.43:0/781938412 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2b600c6ad0 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b537fe640 1 -- 172.21.15.43:0/781938412 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2b60005d40 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 10 client.5817 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x562495b691e0 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b537fe640 1 -- 172.21.15.43:0/781938412 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b6c05ec60 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b537fe640 1 -- 172.21.15.43:0/781938412 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f2b600b6070 con 0x562495d8f190 2024-03-30T13:13:15.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 10 client.5817 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 10 client.5817 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 20 client.5817 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495dd5f20 0x562495e43e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 4 client.5817 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562495be8040 con 0x562495dd5f20 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495dd5f20 0x562495e43e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b537fe640 10 client.5817 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.958+0000 7f2b78bd3640 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495dd5f20 0x562495e43e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5d5dd2d8038d99 server_cookie=6fbe4191a88e236c in_seq=0 out_seq=0 2024-03-30T13:13:15.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b537fe640 1 -- 172.21.15.43:0/781938412 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x562495be8040 con 0x562495dd5f20 2024-03-30T13:13:15.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b537fe640 10 client.5817 handle_command_reply: tid=0 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 client.5817 shutdown 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d8f190 msgr2=0x562495e433a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d8f190 0x562495e433a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2b60018fd0 tx=0x7f2b6001ac00 comp rx=0 tx=0).stop 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 shutdown_connections 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x562495dd5f20 0x562495e43e20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x562495dd5600 0x562495e438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562495d8f190 0x562495e433a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 --2- 172.21.15.43:0/781938412 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x562495d88ce0 0x562495d8e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b537fe640 0 client.5817 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 >> 172.21.15.43:0/781938412 conn(0x562495db63a0 msgr2=0x562495deccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.962+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 shutdown_connections 2024-03-30T13:13:15.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7dfb9000 1 -- 172.21.15.43:0/781938412 wait complete. 2024-03-30T13:13:15.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7dfb9000 20 client.5817 trim_cache size 0 max 16384 2024-03-30T13:13:15.972 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:15.972 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "scrub active (18 inodes in the stack)", 2024-03-30T13:13:15.972 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-03-30T13:13:15.972 INFO:teuthology.orchestra.run.smithi043.stdout: "9ec31cb9-756a-4ece-b87a-bdc758385b7c": { 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "9ec31cb9-756a-4ece-b87a-bdc758385b7c", 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive" 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_stats": { 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "9ec31cb9-756a-4ece-b87a-bdc758385b7c": { 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-03-30T13:13:15.761997+0000", 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 0, 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 0, 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-03-30T13:13:15.973 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:15.974 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:15.974 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:15.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b58067fc0 msgr2=0x7f2b5806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b58067fc0 0x7f2b5806a480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2b680011f0 tx=0x7f2b68009040 comp rx=0 tx=0).stop 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b7415e4c0 msgr2=0x7f2b740cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b7415e4c0 0x7f2b740cd6f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c001f40 tx=0x7f2b6c00a430 comp rx=0 tx=0).stop 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 shutdown_connections 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b58067fc0 0x7f2b5806a480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b7415e4c0 0x7f2b740cd6f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b740cf4f0 0x7f2b74150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 --2- 172.21.15.43:0/3887135902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b740b9e30 0x7f2b741596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 >> 172.21.15.43:0/3887135902 conn(0x7f2b740b6410 msgr2=0x7f2b740d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:15.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 shutdown_connections 2024-03-30T13:13:15.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:15.966+0000 7f2b7ae37640 1 -- 172.21.15.43:0/3887135902 wait complete. 2024-03-30T13:13:15.990 INFO:tasks.cephfs.filesystem:scrub status for tag:9ec31cb9-756a-4ece-b87a-bdc758385b7c - {'path': '/', 'tag': '9ec31cb9-756a-4ece-b87a-bdc758385b7c', 'options': 'recursive'} 2024-03-30T13:13:26.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:26.860 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:45.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:45.992 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:13:46.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.126+0000 7f9c98eaa640 1 Processor -- start 2024-03-30T13:13:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.126+0000 7f9c98eaa640 1 -- start start 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.126+0000 7f9c98eaa640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9414c2a0 0x7f9c94164040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c98eaa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940b47c0 0x7f9c9406cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c98eaa640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9414c680 con 0x7f9c9406c670 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c98eaa640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9c940b3310 con 0x7f9c9414c2a0 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c98eaa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9c940b35d0 con 0x7f9c940b47c0 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940b47c0 0x7f9c9406cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9259c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9c9414c2a0 0x7f9c94164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940b47c0 0x7f9c9406cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:43372/0 (socket says 172.21.15.43:43372) 2024-03-30T13:13:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c92d9d640 1 -- 172.21.15.43:0/3842879347 learned_addr learned my addr 172.21.15.43:0/3842879347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:46.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9259c640 1 -- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940b47c0 msgr2=0x7f9c9406cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940b47c0 0x7f9c9406cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9259c640 1 -- 172.21.15.43:0/3842879347 >> v1:172.21.15.113:6789/0 conn(0x7f9c9406c670 legacy=0x7f9c940b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9259c640 1 -- 172.21.15.43:0/3842879347 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c940b38b0 con 0x7f9c9414c2a0 2024-03-30T13:13:46.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940b47c0 0x7f9c9406cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:13:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9414c2a0 0x7f9c94164040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097c0 tx=0x7f9c8401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2504f841be79f0b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9159a640 1 -- 172.21.15.43:0/3842879347 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b1070 con 0x7f9c9414c2a0 2024-03-30T13:13:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9159a640 1 -- 172.21.15.43:0/3842879347 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840ac410 con 0x7f9c9414c2a0 2024-03-30T13:13:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.130+0000 7f9c9159a640 1 -- 172.21.15.43:0/3842879347 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840ac7e0 con 0x7f9c9414c2a0 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9414c2a0 msgr2=0x7f9c94164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9414c2a0 0x7f9c94164040 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097c0 tx=0x7f9c8401c4a0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- 172.21.15.43:0/3842879347 shutdown_connections 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c940b47c0 0x7f9c9406cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/3842879347 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c9414c2a0 0x7f9c94164040 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- 172.21.15.43:0/3842879347 >> 172.21.15.43:0/3842879347 conn(0x7f9c940d00a0 msgr2=0x7f9c940d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- 172.21.15.43:0/3842879347 shutdown_connections 2024-03-30T13:13:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- 172.21.15.43:0/3842879347 wait complete. 2024-03-30T13:13:46.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 Processor -- start 2024-03-30T13:13:46.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- start start 2024-03-30T13:13:46.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9406c670 0x7f9c94162440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940b47c0 0x7f9c9415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c9414c2a0 0x7f9c94142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94164790 con 0x7f9c9406c670 2024-03-30T13:13:46.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94163bb0 con 0x7f9c940b47c0 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9416f4c0 con 0x7f9c9414c2a0 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9406c670 0x7f9c94162440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9406c670 0x7f9c94162440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59088/0 (socket says 172.21.15.43:59088) 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 -- 172.21.15.43:0/2365949990 learned_addr learned my addr 172.21.15.43:0/2365949990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c9414c2a0 0x7f9c94142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940b47c0 0x7f9c9415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 -- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c9414c2a0 msgr2=0x7f9c94142ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c9414c2a0 0x7f9c94142ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 -- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940b47c0 msgr2=0x7f9c9415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940b47c0 0x7f9c9415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 -- 172.21.15.43:0/2365949990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9415ffe0 con 0x7f9c9406c670 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c9259c640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9406c670 0x7f9c94162440 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401a0d0 tx=0x7f9c84019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:46.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2365949990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b9070 con 0x7f9c9406c670 2024-03-30T13:13:46.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2365949990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840acd70 con 0x7f9c9406c670 2024-03-30T13:13:46.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c941602c0 con 0x7f9c9406c670 2024-03-30T13:13:46.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2365949990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840b1040 con 0x7f9c9406c670 2024-03-30T13:13:46.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.134+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c941fb7b0 con 0x7f9c9406c670 2024-03-30T13:13:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2365949990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9c840bc070 con 0x7f9c9406c670 2024-03-30T13:13:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c82ffd640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c74067ff0 0x7f9c7406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2365949990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f9c8411ffe0 con 0x7f9c9406c670 2024-03-30T13:13:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c74067ff0 0x7f9c7406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 Processor -- start 2024-03-30T13:13:46.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 -- start start 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c74067ff0 0x7f9c7406a4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001e00 tx=0x7f9c88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab938a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab933d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab933e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ab9226e20 con 0x560ab933dce0 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560ab918fb00 con 0x560ab938a600 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560ab912c940 con 0x560ab938af20 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab933e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab933e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59102/0 (socket says 172.21.15.43:59102) 2024-03-30T13:13:46.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 -- 172.21.15.43:0/1322029107 learned_addr learned my addr 172.21.15.43:0/1322029107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab933d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9259c640 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab938a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 -- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 msgr2=0x560ab933d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab933d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 -- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 msgr2=0x560ab938a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab938a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 -- 172.21.15.43:0/1322029107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab9115380 con 0x560ab933dce0 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab933d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c9259c640 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab938a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab933e0c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009b30 tx=0x7f9c8c0112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ea82653083946be server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.138+0000 7f9c73fff640 1 -- 172.21.15.43:0/1322029107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c011d00 con 0x560ab933dce0 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c73fff640 1 -- 172.21.15.43:0/1322029107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c004020 con 0x560ab933dce0 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c73fff640 1 -- 172.21.15.43:0/1322029107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8c011ea0 con 0x560ab933dce0 2024-03-30T13:13:46.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1322029107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 msgr2=0x560ab933e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab933e0c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009b30 tx=0x7f9c8c0112c0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1322029107 shutdown_connections 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab933e0c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab933d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1322029107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab938a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1322029107 >> 172.21.15.43:0/1322029107 conn(0x560ab936b3a0 msgr2=0x560ab936d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1322029107 shutdown_connections 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1322029107 wait complete. 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- start start 2024-03-30T13:13:46.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 Processor -- start 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab9391960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938a600 0x560ab9391ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938af20 0x560ab9342920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ab914d1e0 con 0x560ab933dce0 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560ab918bab0 con 0x560ab938af20 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560ab9194130 con 0x560ab938a600 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 10 client.0 _ll_register_callbacks cb 0x560ab932b6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:46.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab9391960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab9391960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59114/0 (socket says 172.21.15.43:59114) 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 -- 172.21.15.43:0/3795618392 learned_addr learned my addr 172.21.15.43:0/3795618392 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938af20 0x560ab9342920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938a600 0x560ab9391ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 -- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938a600 msgr2=0x560ab9391ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938a600 0x560ab9391ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 -- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938af20 msgr2=0x560ab9342920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938af20 0x560ab9342920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 -- 172.21.15.43:0/3795618392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab914bb70 con 0x560ab933dce0 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938af20 0x560ab9342920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab9391960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401c980 tx=0x7f9c840ec380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1265ac7558e1daa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c717fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c717fa640 1 -- 172.21.15.43:0/3795618392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840c3040 con 0x560ab933dce0 2024-03-30T13:13:46.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c8c004020 con 0x560ab933dce0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c717fa640 1 -- 172.21.15.43:0/3795618392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f9c84108270 con 0x560ab933dce0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c717fa640 1 -- 172.21.15.43:0/3795618392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9c8411e0d0 con 0x560ab933dce0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 10 client.5826 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560ab912cb90 con 0x560ab933dce0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c717fa640 1 -- 172.21.15.43:0/3795618392 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c8c004020 con 0x560ab933dce0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c717fa640 1 -- 172.21.15.43:0/3795618392 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f9c840f3070 con 0x560ab933dce0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 10 client.5826 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 10 client.5826 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 20 client.5826 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab9343690 0x560ab9343a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 4 client.5826 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.142+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ab912add0 con 0x560ab9343690 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab9343690 0x560ab9343a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab9343690 0x560ab9343a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a7fa428ba1284c server_cookie=c19010842c1ba931 in_seq=0 out_seq=0 2024-03-30T13:13:46.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c717fa640 10 client.5826 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c717fa640 1 -- 172.21.15.43:0/3795618392 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x560ab912add0 con 0x560ab9343690 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c717fa640 10 client.5826 handle_command_reply: tid=0 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 client.5826 shutdown 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 msgr2=0x560ab9391960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab9391960 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401c980 tx=0x7f9c840ec380 comp rx=0 tx=0).stop 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 shutdown_connections 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab9343690 0x560ab9343a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938af20 0x560ab9342920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938a600 0x560ab9391ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3795618392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab9391960 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c717fa640 0 client.5826 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 >> 172.21.15.43:0/3795618392 conn(0x560ab936b3a0 msgr2=0x560ab936d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 shutdown_connections 2024-03-30T13:13:46.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3795618392 wait complete. 2024-03-30T13:13:46.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.146+0000 7f9c9c02c000 20 client.5826 trim_cache size 0 max 16384 2024-03-30T13:13:46.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 Processor -- start 2024-03-30T13:13:46.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- start start 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab93886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab9388bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab9389130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c941fd0f0 con 0x560ab933dce0 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c94206e40 con 0x560ab938a600 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560ab918ee10 con 0x560ab938af20 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab93886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab93886b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59126/0 (socket says 172.21.15.43:59126) 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 -- 172.21.15.43:0/1085216448 learned_addr learned my addr 172.21.15.43:0/1085216448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:46.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab9388bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab9389130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 -- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 msgr2=0x560ab9389130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab9389130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 -- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 msgr2=0x560ab9388bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab9388bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 -- 172.21.15.43:0/1085216448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ab9210470 con 0x560ab933dce0 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab9389130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab9388bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9259c640 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab93886b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401ca10 tx=0x7f9c840f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985592d5d2e25ba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c737fe640 1 -- 172.21.15.43:0/1085216448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c84108270 con 0x560ab933dce0 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c737fe640 1 -- 172.21.15.43:0/1085216448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840ef420 con 0x560ab933dce0 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c737fe640 1 -- 172.21.15.43:0/1085216448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c840ebd20 con 0x560ab933dce0 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1085216448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 msgr2=0x560ab93886b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab93886b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401ca10 tx=0x7f9c840f2000 comp rx=0 tx=0).stop 2024-03-30T13:13:46.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1085216448 shutdown_connections 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab9389130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab938a600 0x560ab9388bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/1085216448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab933dce0 0x560ab93886b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1085216448 >> 172.21.15.43:0/1085216448 conn(0x560ab936b3a0 msgr2=0x560ab93a0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1085216448 shutdown_connections 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- 172.21.15.43:0/1085216448 wait complete. 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 -- start start 2024-03-30T13:13:46.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.166+0000 7f9c9c02c000 1 Processor -- start 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab933dce0 0x560ab93f9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab938a600 0x560ab938e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab938e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560ab9146750 con 0x560ab938a600 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560ab915b150 con 0x560ab933dce0 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560ab925e750 con 0x560ab938af20 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 10 client.0 _ll_register_callbacks cb 0x7f9c8c006be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab938a600 0x560ab938e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab938a600 0x560ab938e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59132/0 (socket says 172.21.15.43:59132) 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 -- 172.21.15.43:0/3581003337 learned_addr learned my addr 172.21.15.43:0/3581003337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c92d9d640 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab938e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab933dce0 0x560ab93f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 -- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 msgr2=0x560ab938e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab938e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 -- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab933dce0 msgr2=0x560ab93f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab933dce0 0x560ab93f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 -- 172.21.15.43:0/3581003337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ab911e1e0 con 0x560ab938a600 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c91d9b640 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab938a600 0x560ab938e360 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x560ab914e6a0 tx=0x7f9c88000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823924986240ad0e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 1 -- 172.21.15.43:0/3581003337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88026650 con 0x560ab938a600 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 1 -- 172.21.15.43:0/3581003337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f9c88010040 con 0x560ab938a600 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c840ef420 con 0x560ab938a600 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 1 -- 172.21.15.43:0/3581003337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9c88036050 con 0x560ab938a600 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 10 client.5832 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560ab91237b0 con 0x560ab938a600 2024-03-30T13:13:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 1 -- 172.21.15.43:0/3581003337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c840ef420 con 0x560ab938a600 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 1 -- 172.21.15.43:0/3581003337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f9c88007560 con 0x560ab938a600 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 10 client.5832 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 10 client.5832 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 20 client.5832 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab938ede0 0x560ab938f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 4 client.5832 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560ab919d040 con 0x560ab938ede0 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab938ede0 0x560ab938f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9259c640 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab938ede0 0x560ab938f1c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba382e1913d6ea6e server_cookie=626adddf226eeca8 in_seq=0 out_seq=0 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 10 client.5832 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 1 -- 172.21.15.43:0/3581003337 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+376 (crc 0 0 0) 0x560ab919d040 con 0x560ab938ede0 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 10 client.5832 handle_command_reply: tid=0 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 client.5832 shutdown 2024-03-30T13:13:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab938a600 msgr2=0x560ab938e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab938a600 0x560ab938e360 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x560ab914e6a0 tx=0x7f9c88000f50 comp rx=0 tx=0).stop 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 shutdown_connections 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560ab938ede0 0x560ab938f1c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560ab938af20 0x560ab938e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560ab938a600 0x560ab938e360 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c717fa640 0 client.5832 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 --2- 172.21.15.43:0/3581003337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560ab933dce0 0x560ab93f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 >> 172.21.15.43:0/3581003337 conn(0x560ab936b3a0 msgr2=0x560ab9342450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 shutdown_connections 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 1 -- 172.21.15.43:0/3581003337 wait complete. 2024-03-30T13:13:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c9c02c000 20 client.5832 trim_cache size 0 max 16384 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {}, 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_stats": { 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "9ec31cb9-756a-4ece-b87a-bdc758385b7c": { 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-03-30T13:13:15.761997+0000", 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 23, 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 23, 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:46.182 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:46.183 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:46.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.170+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c74067ff0 msgr2=0x7f9c7406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c74067ff0 0x7f9c7406a4b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c88001e00 tx=0x7f9c88009040 comp rx=0 tx=0).stop 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9406c670 msgr2=0x7f9c94162440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9406c670 0x7f9c94162440 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9c8401a0d0 tx=0x7f9c84019240 comp rx=0 tx=0).stop 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 shutdown_connections 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c74067ff0 0x7f9c7406a4b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c9414c2a0 0x7f9c94142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c940b47c0 0x7f9c9415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 --2- 172.21.15.43:0/2365949990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c9406c670 0x7f9c94162440 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 >> 172.21.15.43:0/2365949990 conn(0x7f9c940d00a0 msgr2=0x7f9c94071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 shutdown_connections 2024-03-30T13:13:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:46.174+0000 7f9c98eaa640 1 -- 172.21.15.43:0/2365949990 wait complete. 2024-03-30T13:13:46.194 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T13:13:46.194 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-30T13:13:46.195 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:13:46.195 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-30T13:13:46.203 INFO:teuthology.orchestra.run.smithi043.stdout:1099511627780 linux-5.4/fs/ceph/Kconfig 2024-03-30T13:13:46.203 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-30T13:13:46.204 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-30T13:13:46.251 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628281 linux-5.4/fs/ceph/Makefile 2024-03-30T13:13:46.251 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:13:46.251 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:13:46.299 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628287 linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:13:46.300 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-30T13:13:46.300 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-30T13:13:46.349 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628294 linux-5.4/fs/ceph/io.h 2024-03-30T13:13:46.349 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:13:46.349 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:13:46.401 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628296 linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:13:46.401 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:13:46.402 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-30T13:13:46.453 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628308 linux-5.4/fs/ext2/Kconfig 2024-03-30T13:13:46.454 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:13:46.454 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:13:46.505 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628326 linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:13:46.505 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:13:46.506 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:13:46.557 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628324 linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:13:46.557 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-30T13:13:46.557 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-30T13:13:46.610 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628309 linux-5.4/fs/ext2/Makefile 2024-03-30T13:13:46.610 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:13:46.610 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-30T13:13:46.662 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628321 linux-5.4/fs/ext2/symlink.c 2024-03-30T13:13:46.662 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-30T13:13:46.662 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-30T13:13:46.713 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628311 linux-5.4/fs/ext2/acl.h 2024-03-30T13:13:46.713 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:13:46.713 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:13:46.765 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628325 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:13:46.765 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:13:46.766 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-30T13:13:46.818 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628323 linux-5.4/fs/ext2/xattr.h 2024-03-30T13:13:46.818 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:13:46.819 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:13:46.870 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628328 linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:13:46.870 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:13:46.870 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:13:46.921 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628341 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:13:46.922 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:13:46.922 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:13:46.973 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628348 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:13:46.973 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:13:46.974 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:13:47.025 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628331 linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:13:47.025 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:13:47.025 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:13:47.077 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628339 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:13:47.077 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:13:47.078 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:13:47.130 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628343 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:13:47.130 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:13:47.130 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:13:47.182 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628344 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:13:47.182 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:13:47.182 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-30T13:13:47.233 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628329 linux-5.4/fs/orangefs/Makefile 2024-03-30T13:13:47.233 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:13:47.233 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:13:47.285 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628352 linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:13:47.286 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:13:47.286 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:13:47.337 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628334 linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:13:47.337 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-30T13:13:47.337 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511627780 2024-03-30T13:13:47.337 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511627780 2024-03-30T13:13:47.338 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511627780 2024-03-30T13:13:47.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d28365640 1 Processor -- start 2024-03-30T13:13:47.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d28365640 1 -- start start 2024-03-30T13:13:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d28365640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d20144930 0x7f0d200b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d28365640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d2006cd80 0x7f0d200ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d28365640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d200b33c0 con 0x7f0d200b4870 2024-03-30T13:13:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d28365640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f0d200cdb90 con 0x7f0d2006cd80 2024-03-30T13:13:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d28365640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d200b3660 con 0x7f0d20144930 2024-03-30T13:13:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d25900640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0d200b4870 0x7f0d2006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47176/0 (socket says 172.21.15.43:47176) 2024-03-30T13:13:47.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d25900640 1 -- 172.21.15.43:0/2583919173 learned_addr learned my addr 172.21.15.43:0/2583919173 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d26902640 1 --2- 172.21.15.43:0/2583919173 >> v2:172.21.15.43:3300/0 conn(0x7f0d2006cd80 0x7f0d200ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d26101640 1 --2- 172.21.15.43:0/2583919173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d20144930 0x7f0d200b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d250ff640 1 -- 172.21.15.43:0/2583919173 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1772960418 0 0) 0x7f0d200b33c0 con 0x7f0d200b4870 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d250ff640 1 -- 172.21.15.43:0/2583919173 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d0c003530 con 0x7f0d200b4870 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d26902640 1 -- 172.21.15.43:0/2583919173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d20144930 msgr2=0x7f0d200b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d26902640 1 --2- 172.21.15.43:0/2583919173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d20144930 0x7f0d200b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d26902640 1 -- 172.21.15.43:0/2583919173 >> v1:172.21.15.43:6789/0 conn(0x7f0d200b4870 legacy=0x7f0d2006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.514+0000 7f0d26902640 1 -- 172.21.15.43:0/2583919173 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d200b3940 con 0x7f0d2006cd80 2024-03-30T13:13:47.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d26902640 1 --2- 172.21.15.43:0/2583919173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d2006cd80 0x7f0d200ce020 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009d90 tx=0x7f0d1801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f945d49ccca66a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d250ff640 1 -- 172.21.15.43:0/2583919173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180b2070 con 0x7f0d2006cd80 2024-03-30T13:13:47.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d250ff640 1 -- 172.21.15.43:0/2583919173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180a5930 con 0x7f0d2006cd80 2024-03-30T13:13:47.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d250ff640 1 -- 172.21.15.43:0/2583919173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180a5ce0 con 0x7f0d2006cd80 2024-03-30T13:13:47.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- 172.21.15.43:0/2583919173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d2006cd80 msgr2=0x7f0d200ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 --2- 172.21.15.43:0/2583919173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d2006cd80 0x7f0d200ce020 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009d90 tx=0x7f0d1801c8b0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- 172.21.15.43:0/2583919173 shutdown_connections 2024-03-30T13:13:47.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 --2- 172.21.15.43:0/2583919173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d2006cd80 0x7f0d200ce020 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 --2- 172.21.15.43:0/2583919173 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d20144930 0x7f0d200b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- 172.21.15.43:0/2583919173 >> 172.21.15.43:0/2583919173 conn(0x7f0d200d00a0 msgr2=0x7f0d200d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- 172.21.15.43:0/2583919173 shutdown_connections 2024-03-30T13:13:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- 172.21.15.43:0/2583919173 wait complete. 2024-03-30T13:13:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 Processor -- start 2024-03-30T13:13:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- start start 2024-03-30T13:13:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d2006cd80 0x7f0d20153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d200b4870 0x7f0d2015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20144930 0x7f0d200b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200ceef0 con 0x7f0d20144930 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200ced70 con 0x7f0d200b4870 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d28365640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d200cf070 con 0x7f0d2006cd80 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20144930 0x7f0d200b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20144930 0x7f0d200b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59148/0 (socket says 172.21.15.43:59148) 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d26902640 1 -- 172.21.15.43:0/3617512617 learned_addr learned my addr 172.21.15.43:0/3617512617 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.518+0000 7f0d26101640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d2006cd80 0x7f0d20153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d25900640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d200b4870 0x7f0d2015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26902640 1 -- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d2006cd80 msgr2=0x7f0d20153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26902640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d2006cd80 0x7f0d20153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26902640 1 -- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d200b4870 msgr2=0x7f0d2015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26902640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d200b4870 0x7f0d2015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26902640 1 -- 172.21.15.43:0/3617512617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2015d800 con 0x7f0d20144930 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26902640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20144930 0x7f0d200b0ec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0d1801a4e0 tx=0x7f0d18019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d16ffd640 1 -- 172.21.15.43:0/3617512617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180b2050 con 0x7f0d20144930 2024-03-30T13:13:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d16ffd640 1 -- 172.21.15.43:0/3617512617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180be070 con 0x7f0d20144930 2024-03-30T13:13:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2015dae0 con 0x7f0d20144930 2024-03-30T13:13:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d16ffd640 1 -- 172.21.15.43:0/3617512617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18004040 con 0x7f0d20144930 2024-03-30T13:13:47.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2015d4d0 con 0x7f0d20144930 2024-03-30T13:13:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d16ffd640 1 -- 172.21.15.43:0/3617512617 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0d180041e0 con 0x7f0d20144930 2024-03-30T13:13:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d16ffd640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d04067fc0 0x7f0d0406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d16ffd640 1 -- 172.21.15.43:0/3617512617 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0d18121980 con 0x7f0d20144930 2024-03-30T13:13:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26101640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d04067fc0 0x7f0d0406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.522+0000 7f0d26101640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d04067fc0 0x7f0d0406a480 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d20154430 tx=0x7f0d1c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 Processor -- start 2024-03-30T13:13:47.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- start start 2024-03-30T13:13:47.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9138d70 0x5643c9139150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9139690 0x5643c90ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c90ef830 0x5643c90efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643c8f3daa0 con 0x5643c90ef830 2024-03-30T13:13:47.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5643c8ec4af0 con 0x5643c9138d70 2024-03-30T13:13:47.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5643c8f39a50 con 0x5643c9139690 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c90ef830 0x5643c90efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c90ef830 0x5643c90efc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59164/0 (socket says 172.21.15.43:59164) 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 -- 172.21.15.43:0/1137302069 learned_addr learned my addr 172.21.15.43:0/1137302069 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d25900640 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9139690 0x5643c90ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26101640 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9138d70 0x5643c9139150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 -- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9139690 msgr2=0x5643c90ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9139690 0x5643c90ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 -- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9138d70 msgr2=0x5643c9139150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9138d70 0x5643c9139150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 -- 172.21.15.43:0/1137302069 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c8ec2d00 con 0x5643c90ef830 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26101640 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9138d70 0x5643c9139150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d26902640 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c90ef830 0x5643c90efc10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d180f9f20 tx=0x7f0d180f1f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f18b73923227e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d03fff640 1 -- 172.21.15.43:0/1137302069 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1811e070 con 0x5643c90ef830 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d03fff640 1 -- 172.21.15.43:0/1137302069 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180b3040 con 0x5643c90ef830 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d03fff640 1 -- 172.21.15.43:0/1137302069 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180f5c20 con 0x5643c90ef830 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1137302069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c90ef830 msgr2=0x5643c90efc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c90ef830 0x5643c90efc10 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d180f9f20 tx=0x7f0d180f1f70 comp rx=0 tx=0).stop 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1137302069 shutdown_connections 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c90ef830 0x5643c90efc10 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9139690 0x5643c90ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1137302069 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9138d70 0x5643c9139150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1137302069 >> 172.21.15.43:0/1137302069 conn(0x5643c90eb7e0 msgr2=0x5643c90edbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1137302069 shutdown_connections 2024-03-30T13:13:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1137302069 wait complete. 2024-03-30T13:13:47.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 -- start start 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.526+0000 7f0d2b4e7000 1 Processor -- start 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c90ef830 0x5643c90f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9138d70 0x5643c911dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9139690 0x5643c911e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643c8f05270 con 0x5643c9138d70 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5643c8efb6a0 con 0x5643c9139690 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5643c8ed5880 con 0x5643c90ef830 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 10 client.0 _ll_register_callbacks cb 0x5643c90da490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9138d70 0x5643c911dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9138d70 0x5643c911dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59172/0 (socket says 172.21.15.43:59172) 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 -- 172.21.15.43:0/3653617879 learned_addr learned my addr 172.21.15.43:0/3653617879 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d26902640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9139690 0x5643c911e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d26101640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c90ef830 0x5643c90f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 -- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c90ef830 msgr2=0x5643c90f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c90ef830 0x5643c90f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 -- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9139690 msgr2=0x5643c911e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9139690 0x5643c911e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 -- 172.21.15.43:0/3653617879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c8f02730 con 0x5643c9138d70 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d26902640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9139690 0x5643c911e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d26101640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c90ef830 0x5643c90f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d25900640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9138d70 0x5643c911dda0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0d10007370 tx=0x7f0d10007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cada2db24a169e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d017fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d017fa640 1 -- 172.21.15.43:0/3653617879 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10001c50 con 0x5643c9138d70 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d017fa640 1 -- 172.21.15.43:0/3653617879 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0d10014400 con 0x5643c9138d70 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d180b3040 con 0x5643c9138d70 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d017fa640 1 -- 172.21.15.43:0/3653617879 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0d10021050 con 0x5643c9138d70 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 10 client.5847 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5643c8ef5ca0 con 0x5643c9138d70 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d017fa640 1 -- 172.21.15.43:0/3653617879 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180b3040 con 0x5643c9138d70 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d017fa640 1 -- 172.21.15.43:0/3653617879 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f0d10024020 con 0x5643c9138d70 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 10 client.5847 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 10 client.5847 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 20 client.5847 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c91405d0 0x5643c91409e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 4 client.5847 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c8ed8c90 con 0x5643c91405d0 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d26101640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c91405d0 0x5643c91409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d26101640 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c91405d0 0x5643c91409e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9873475c8ed95e28 server_cookie=31795ee368a48e4f in_seq=0 out_seq=0 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.530+0000 7f0d017fa640 10 client.5847 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d017fa640 1 -- 172.21.15.43:0/3653617879 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5643c8ed8c90 con 0x5643c91405d0 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d017fa640 10 client.5847 handle_command_reply: tid=0 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 client.5847 shutdown 2024-03-30T13:13:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9138d70 msgr2=0x5643c911dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9138d70 0x5643c911dda0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0d10007370 tx=0x7f0d10007830 comp rx=0 tx=0).stop 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 shutdown_connections 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c91405d0 0x5643c91409e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c9139690 0x5643c911e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9138d70 0x5643c911dda0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/3653617879 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c90ef830 0x5643c90f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d017fa640 0 client.5847 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 >> 172.21.15.43:0/3653617879 conn(0x5643c90eb7e0 msgr2=0x5643c90ee460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 shutdown_connections 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/3653617879 wait complete. 2024-03-30T13:13:47.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.534+0000 7f0d2b4e7000 20 client.5847 trim_cache size 0 max 16384 2024-03-30T13:13:47.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 Processor -- start 2024-03-30T13:13:47.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 -- start start 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c911e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20203b10 con 0x5643c9139690 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20202b90 con 0x5643c90ef830 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2013e8a0 con 0x5643c9138d70 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59186/0 (socket says 172.21.15.43:59186) 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.546+0000 7f0d26902640 1 -- 172.21.15.43:0/1228500358 learned_addr learned my addr 172.21.15.43:0/1228500358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26101640 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c911e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d25900640 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 -- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 msgr2=0x5643c913ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 -- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 msgr2=0x5643c911e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c911e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 -- 172.21.15.43:0/1228500358 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c8ec4d00 con 0x5643c9139690 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26101640 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c911e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913bfc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0d1801cd90 tx=0x7f0d180f4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df460c3ee2873cb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d037fe640 1 -- 172.21.15.43:0/1228500358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1810f070 con 0x5643c9139690 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d037fe640 1 -- 172.21.15.43:0/1228500358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180fa450 con 0x5643c9139690 2024-03-30T13:13:47.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d037fe640 1 -- 172.21.15.43:0/1228500358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18109e60 con 0x5643c9139690 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1228500358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 msgr2=0x5643c913bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913bfc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0d1801cd90 tx=0x7f0d180f4000 comp rx=0 tx=0).stop 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1228500358 shutdown_connections 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913bfc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1228500358 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c911e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1228500358 >> 172.21.15.43:0/1228500358 conn(0x5643c90eb7e0 msgr2=0x5643c914ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1228500358 shutdown_connections 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1228500358 wait complete. 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- start start 2024-03-30T13:13:47.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 Processor -- start 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c91aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5643c8f3cdb0 con 0x5643c9139690 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20202540 con 0x5643c90ef830 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5643c8ff9b00 con 0x5643c9138d70 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 10 client.0 _ll_register_callbacks cb 0x7f0d10007e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:47.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59196/0 (socket says 172.21.15.43:59196) 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 -- 172.21.15.43:0/1773792315 learned_addr learned my addr 172.21.15.43:0/1773792315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d25900640 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26101640 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c91aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 -- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 msgr2=0x5643c913cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 -- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 msgr2=0x5643c91aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c91aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 -- 172.21.15.43:0/1773792315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c8fbef70 con 0x5643c9139690 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d26902640 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913d010 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0d1801c8f0 tx=0x7f0d18009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa6c4b43cdf04dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d017fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d017fa640 1 -- 172.21.15.43:0/1773792315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1810f040 con 0x5643c9139690 2024-03-30T13:13:47.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d017fa640 1 -- 172.21.15.43:0/1773792315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0d180efb70 con 0x5643c9139690 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d180fa450 con 0x5643c9139690 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d017fa640 1 -- 172.21.15.43:0/1773792315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0d1811e040 con 0x5643c9139690 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 10 client.5853 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5643c8ee4760 con 0x5643c9139690 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.550+0000 7f0d017fa640 1 -- 172.21.15.43:0/1773792315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d180fa450 con 0x5643c9139690 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d017fa640 1 -- 172.21.15.43:0/1773792315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f0d18119260 con 0x5643c9139690 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 10 client.5853 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 10 client.5853 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 20 client.5853 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c913d9d0 0x5643c913ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 4 client.5853 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511627780}] 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511627780}) -- 0x5643c8f4afe0 con 0x5643c913d9d0 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d26101640 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c913d9d0 0x5643c913ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d26101640 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c913d9d0 0x5643c913ddb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a67d3891342744e1 server_cookie=188fb955a562efb5 in_seq=0 out_seq=0 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d017fa640 10 client.5853 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d017fa640 1 -- 172.21.15.43:0/1773792315 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3812 (crc 0 0 0) 0x5643c8f4afe0 con 0x5643c913d9d0 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d017fa640 10 client.5853 handle_command_reply: tid=0 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 client.5853 shutdown 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 msgr2=0x5643c913d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913d010 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0d1801c8f0 tx=0x7f0d18009750 comp rx=0 tx=0).stop 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 shutdown_connections 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5643c913d9d0 0x5643c913ddb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5643c9139690 0x5643c913d010 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5643c9138d70 0x5643c913cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d017fa640 0 client.5853 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 --2- 172.21.15.43:0/1773792315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5643c90ef830 0x5643c91aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 >> 172.21.15.43:0/1773792315 conn(0x5643c90eb7e0 msgr2=0x5643c9150620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 shutdown_connections 2024-03-30T13:13:47.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 1 -- 172.21.15.43:0/1773792315 wait complete. 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d2b4e7000 20 client.5853 trim_cache size 0 max 16384 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/Kconfig", 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511627780, 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.808461+0000", 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.181312+0000", 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:47.564 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:47.565 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1481, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.181312+0000", 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:47.566 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1481, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:47.567 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.808461+0000" 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1481, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.808461+0000" 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 80, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 2, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:47.568 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.569 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.570 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:47.571 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:47.572 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.573 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:47.575 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d04067fc0 msgr2=0x7f0d0406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d04067fc0 0x7f0d0406a480 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d20154430 tx=0x7f0d1c008040 comp rx=0 tx=0).stop 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20144930 msgr2=0x7f0d200b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20144930 0x7f0d200b0ec0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0d1801a4e0 tx=0x7f0d18019650 comp rx=0 tx=0).stop 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 shutdown_connections 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0d04067fc0 0x7f0d0406a480 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d20144930 0x7f0d200b0ec0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0d200b4870 0x7f0d2015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 --2- 172.21.15.43:0/3617512617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0d2006cd80 0x7f0d20153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 >> 172.21.15.43:0/3617512617 conn(0x7f0d200d00a0 msgr2=0x7f0d2006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 shutdown_connections 2024-03-30T13:13:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.554+0000 7f0d28365640 1 -- 172.21.15.43:0/3617512617 wait complete. 2024-03-30T13:13:47.578 DEBUG:tasks.cephfs.test_uninlining:getting inode info #2 of 23:1099511628281 2024-03-30T13:13:47.578 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628281 2024-03-30T13:13:47.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628281 2024-03-30T13:13:47.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2678780640 1 Processor -- start 2024-03-30T13:13:47.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2678780640 1 -- start start 2024-03-30T13:13:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2678780640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f267015ff40 0x7f267006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2678780640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267006d710 con 0x7f267013f140 2024-03-30T13:13:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2678780640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f267006d9f0 con 0x7f267006cbb0 2024-03-30T13:13:47.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2678780640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f267006e320 con 0x7f267015ff40 2024-03-30T13:13:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2675d1b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f267015ff40 0x7f267006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f267651c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f267013f140 0x7f26701606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47178/0 (socket says 172.21.15.43:47178) 2024-03-30T13:13:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f267651c640 1 -- 172.21.15.43:0/3812630711 learned_addr learned my addr 172.21.15.43:0/3812630711 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2675d1b640 1 -- 172.21.15.43:0/3812630711 >> v1:172.21.15.113:6789/0 conn(0x7f267006cbb0 legacy=0x7f26700ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2675d1b640 1 -- 172.21.15.43:0/3812630711 >> v1:172.21.15.43:6789/0 conn(0x7f267013f140 legacy=0x7f26701606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f2675d1b640 1 -- 172.21.15.43:0/3812630711 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267006dcd0 con 0x7f267015ff40 2024-03-30T13:13:47.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.694+0000 7f267551a640 1 -- 172.21.15.43:0/3812630711 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039151513 0 0) 0x7f267006d710 con 0x7f267013f140 2024-03-30T13:13:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2675d1b640 1 --2- 172.21.15.43:0/3812630711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267015ff40 0x7f267006c670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2664009d90 tx=0x7f266401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc284d37a32d1a72 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f267551a640 1 -- 172.21.15.43:0/3812630711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640b2070 con 0x7f267015ff40 2024-03-30T13:13:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f267551a640 1 -- 172.21.15.43:0/3812630711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640a57f0 con 0x7f267015ff40 2024-03-30T13:13:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f267551a640 1 -- 172.21.15.43:0/3812630711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640a5bc0 con 0x7f267015ff40 2024-03-30T13:13:47.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- 172.21.15.43:0/3812630711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267015ff40 msgr2=0x7f267006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 --2- 172.21.15.43:0/3812630711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267015ff40 0x7f267006c670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2664009d90 tx=0x7f266401c930 comp rx=0 tx=0).stop 2024-03-30T13:13:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- 172.21.15.43:0/3812630711 shutdown_connections 2024-03-30T13:13:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 --2- 172.21.15.43:0/3812630711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267015ff40 0x7f267006c670 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- 172.21.15.43:0/3812630711 >> 172.21.15.43:0/3812630711 conn(0x7f26700d00a0 msgr2=0x7f26700d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- 172.21.15.43:0/3812630711 shutdown_connections 2024-03-30T13:13:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- 172.21.15.43:0/3812630711 wait complete. 2024-03-30T13:13:47.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 Processor -- start 2024-03-30T13:13:47.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- start start 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006cbb0 0x7f2670139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267013f140 0x7f2670154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267015ff40 0x7f267015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26700cfc30 con 0x7f267006cbb0 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f26700cfab0 con 0x7f267015ff40 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f2678780640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f267016a000 con 0x7f267013f140 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f267651c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006cbb0 0x7f2670139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f267651c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006cbb0 0x7f2670139f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59216/0 (socket says 172.21.15.43:59216) 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.698+0000 7f267651c640 1 -- 172.21.15.43:0/3663925920 learned_addr learned my addr 172.21.15.43:0/3663925920 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f2676d1d640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267015ff40 0x7f267015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267651c640 1 -- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267013f140 msgr2=0x7f2670154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267651c640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267013f140 0x7f2670154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267651c640 1 -- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267015ff40 msgr2=0x7f267015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267651c640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267015ff40 0x7f267015b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267651c640 1 -- 172.21.15.43:0/3663925920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267013a4a0 con 0x7f267006cbb0 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f2676d1d640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267015ff40 0x7f267015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267651c640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006cbb0 0x7f2670139f60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f26600011f0 tx=0x7f266000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f266effd640 1 -- 172.21.15.43:0/3663925920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660018070 con 0x7f267006cbb0 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f266effd640 1 -- 172.21.15.43:0/3663925920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26600074e0 con 0x7f267006cbb0 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2670156800 con 0x7f267006cbb0 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f266effd640 1 -- 172.21.15.43:0/3663925920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660004a10 con 0x7f267006cbb0 2024-03-30T13:13:47.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26700b3560 con 0x7f267006cbb0 2024-03-30T13:13:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f266effd640 1 -- 172.21.15.43:0/3663925920 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f266001b070 con 0x7f267006cbb0 2024-03-30T13:13:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f266effd640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f265c067fc0 0x7f265c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f266effd640 1 -- 172.21.15.43:0/3663925920 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2660085970 con 0x7f267006cbb0 2024-03-30T13:13:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f2675d1b640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f265c067fc0 0x7f265c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267b902000 1 Processor -- start 2024-03-30T13:13:47.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.702+0000 7f267b902000 1 -- start start 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2675d1b640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f265c067fc0 0x7f265c06a480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26640af000 tx=0x7f2664004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a72150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a28c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x565242876aa0 con 0x565242a71d70 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5652427fdaf0 con 0x565242a72690 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x565242872a50 con 0x565242a28830 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a72150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a72150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59220/0 (socket says 172.21.15.43:59220) 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/2804540532 learned_addr learned my addr 172.21.15.43:0/2804540532 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2675d1b640 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2676d1d640 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a28c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 msgr2=0x565242a28c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a28c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 msgr2=0x565242a282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/2804540532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652427fbd00 con 0x565242a71d70 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a72150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f266005dfa0 tx=0x7f2660055770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e42a444996b677 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2657fff640 1 -- 172.21.15.43:0/2804540532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660053bf0 con 0x565242a71d70 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2657fff640 1 -- 172.21.15.43:0/2804540532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266005c070 con 0x565242a71d70 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2657fff640 1 -- 172.21.15.43:0/2804540532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660004bb0 con 0x565242a71d70 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- 172.21.15.43:0/2804540532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 msgr2=0x565242a72150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a72150 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f266005dfa0 tx=0x7f2660055770 comp rx=0 tx=0).stop 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- 172.21.15.43:0/2804540532 shutdown_connections 2024-03-30T13:13:47.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a28c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- 172.21.15.43:0/2804540532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a72150 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- 172.21.15.43:0/2804540532 >> 172.21.15.43:0/2804540532 conn(0x565242a247e0 msgr2=0x565242a26bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- 172.21.15.43:0/2804540532 shutdown_connections 2024-03-30T13:13:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- 172.21.15.43:0/2804540532 wait complete. 2024-03-30T13:13:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- start start 2024-03-30T13:13:47.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 Processor -- start 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a28830 0x565242a2dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a71d70 0x565242a56da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56524283e270 con 0x565242a28830 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5652428346a0 con 0x565242a72690 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56524280e880 con 0x565242a71d70 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267b902000 10 client.0 _ll_register_callbacks cb 0x565242a13490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a28830 0x565242a2dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a28830 0x565242a2dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59226/0 (socket says 172.21.15.43:59226) 2024-03-30T13:13:47.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/1534859631 learned_addr learned my addr 172.21.15.43:0/1534859631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2675d1b640 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a71d70 0x565242a56da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f2676d1d640 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a71d70 msgr2=0x565242a56da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a71d70 0x565242a56da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 msgr2=0x565242a572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.706+0000 7f267651c640 1 -- 172.21.15.43:0/1534859631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524283b730 con 0x565242a28830 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267651c640 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a28830 0x565242a2dc80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2660083dc0 tx=0x7f2660058070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a789946bd6d04a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 1 -- 172.21.15.43:0/1534859631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2660071070 con 0x565242a28830 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f266005c070 con 0x565242a28830 2024-03-30T13:13:47.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 1 -- 172.21.15.43:0/1534859631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f2660004bb0 con 0x565242a28830 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 1 -- 172.21.15.43:0/1534859631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f266007b050 con 0x565242a28830 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 10 client.5868 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56524282eca0 con 0x565242a28830 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 1 -- 172.21.15.43:0/1534859631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f266005c070 con 0x565242a28830 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 1 -- 172.21.15.43:0/1534859631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f266007d2f0 con 0x565242a28830 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 10 client.5868 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 10 client.5868 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 20 client.5868 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a795d0 0x565242a799b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 4 client.5868 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565242811c90 con 0x565242a795d0 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f2675d1b640 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a795d0 0x565242a799b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f2675d1b640 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a795d0 0x565242a799b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2290cd849706dc8e server_cookie=21bec1f85e669f99 in_seq=0 out_seq=0 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 10 client.5868 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 1 -- 172.21.15.43:0/1534859631 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x565242811c90 con 0x565242a795d0 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 10 client.5868 handle_command_reply: tid=0 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 client.5868 shutdown 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a28830 msgr2=0x565242a2dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a28830 0x565242a2dc80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2660083dc0 tx=0x7f2660058070 comp rx=0 tx=0).stop 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 shutdown_connections 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a795d0 0x565242a799b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a71d70 0x565242a56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 --2- 172.21.15.43:0/1534859631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a28830 0x565242a2dc80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f26557fa640 0 client.5868 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 >> 172.21.15.43:0/1534859631 conn(0x565242a247e0 msgr2=0x565242a27460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 shutdown_connections 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 1 -- 172.21.15.43:0/1534859631 wait complete. 2024-03-30T13:13:47.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.710+0000 7f267b902000 20 client.5868 trim_cache size 0 max 16384 2024-03-30T13:13:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 Processor -- start 2024-03-30T13:13:47.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 -- start start 2024-03-30T13:13:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a74a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a74fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a75500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2670203d20 con 0x565242a71d70 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2670202dc0 con 0x565242a72690 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f26700b4d00 con 0x565242a28830 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a74fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a74fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59228/0 (socket says 172.21.15.43:59228) 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 -- 172.21.15.43:0/966048932 learned_addr learned my addr 172.21.15.43:0/966048932 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2676d1d640 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a75500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267651c640 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a74a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 -- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 msgr2=0x565242a74a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a74a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 -- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 msgr2=0x565242a75500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a75500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 -- 172.21.15.43:0/966048932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652427fdd00 con 0x565242a71d70 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2676d1d640 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a75500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f267651c640 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a74a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f2675d1b640 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a74fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f26640b0040 tx=0x7f26640a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d6c61939776692 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f26577fe640 1 -- 172.21.15.43:0/966048932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640b2040 con 0x565242a71d70 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f26577fe640 1 -- 172.21.15.43:0/966048932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640d2070 con 0x565242a71d70 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.726+0000 7f26577fe640 1 -- 172.21.15.43:0/966048932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640cce20 con 0x565242a71d70 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/966048932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 msgr2=0x565242a74fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a74fc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f26640b0040 tx=0x7f26640a7000 comp rx=0 tx=0).stop 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/966048932 shutdown_connections 2024-03-30T13:13:47.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a72690 0x565242a75500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a71d70 0x565242a74fc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- 172.21.15.43:0/966048932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242a74a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/966048932 >> 172.21.15.43:0/966048932 conn(0x565242a247e0 msgr2=0x565242a88f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/966048932 shutdown_connections 2024-03-30T13:13:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/966048932 wait complete. 2024-03-30T13:13:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- start start 2024-03-30T13:13:47.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 Processor -- start 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242ae3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a72690 0x565242a75ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a76010 0x565242a763f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2670202770 con 0x565242a72690 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56524287d6b0 con 0x565242a76010 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x565242932b00 con 0x565242a28830 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 10 client.0 _ll_register_callbacks cb 0x7f26600558f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a72690 0x565242a75ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a72690 0x565242a75ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59244/0 (socket says 172.21.15.43:59244) 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 -- 172.21.15.43:0/1593435303 learned_addr learned my addr 172.21.15.43:0/1593435303 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2676d1d640 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a76010 0x565242a763f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267651c640 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242ae3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 -- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 msgr2=0x565242ae3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242ae3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 -- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a76010 msgr2=0x565242a763f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a76010 0x565242a763f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 -- 172.21.15.43:0/1593435303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652428f7f70 con 0x565242a72690 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f2675d1b640 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a72690 0x565242a75ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f26640cf040 tx=0x7f266401b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ddbddcf641b162 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f26557fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f26557fa640 1 -- 172.21.15.43:0/1593435303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26640db060 con 0x565242a72690 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f26557fa640 1 -- 172.21.15.43:0/1593435303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f26640cc710 con 0x565242a72690 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26640d2070 con 0x565242a72690 2024-03-30T13:13:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f26557fa640 1 -- 172.21.15.43:0/1593435303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f26640e1050 con 0x565242a72690 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 10 client.5874 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56524281d760 con 0x565242a72690 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f26557fa640 1 -- 172.21.15.43:0/1593435303 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26640d2070 con 0x565242a72690 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f26557fa640 1 -- 172.21.15.43:0/1593435303 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f26640e4020 con 0x565242a72690 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 10 client.5874 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 10 client.5874 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 20 client.5874 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a76930 0x565242a76d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 4 client.5874 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628281}] 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628281}) -- 0x565242883fe0 con 0x565242a76930 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.730+0000 7f267651c640 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a76930 0x565242a76d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267651c640 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a76930 0x565242a76d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaf82fda6c3d6139 server_cookie=562af1e2e232739a in_seq=0 out_seq=0 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f26557fa640 10 client.5874 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f26557fa640 1 -- 172.21.15.43:0/1593435303 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3811 (crc 0 0 0) 0x565242883fe0 con 0x565242a76930 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f26557fa640 10 client.5874 handle_command_reply: tid=0 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 client.5874 shutdown 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a72690 msgr2=0x565242a75ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a72690 0x565242a75ad0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f26640cf040 tx=0x7f266401b170 comp rx=0 tx=0).stop 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 shutdown_connections 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565242a76930 0x565242a76d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565242a76010 0x565242a763f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565242a72690 0x565242a75ad0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 --2- 172.21.15.43:0/1593435303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565242a28830 0x565242ae3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 >> 172.21.15.43:0/1593435303 conn(0x565242a247e0 msgr2=0x565242a89620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f26557fa640 0 client.5874 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 shutdown_connections 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 1 -- 172.21.15.43:0/1593435303 wait complete. 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f267b902000 20 client.5874 trim_cache size 0 max 16384 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/Makefile", 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628281, 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.877101+0000", 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.188081+0000", 2024-03-30T13:13:47.744 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:47.745 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 356, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.188081+0000", 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:47.746 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:47.747 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.877101+0000" 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.748 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.877101+0000" 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 110, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 4, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:47.749 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.750 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:47.751 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:47.752 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.753 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.754 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.755 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:47.756 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:47.757 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:47.757 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:47.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f265c067fc0 msgr2=0x7f265c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f265c067fc0 0x7f265c06a480 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f26640af000 tx=0x7f2664004ac0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006cbb0 msgr2=0x7f2670139f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006cbb0 0x7f2670139f60 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f26600011f0 tx=0x7f266000c9b0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 shutdown_connections 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f265c067fc0 0x7f265c06a480 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f267015ff40 0x7f267015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f267013f140 0x7f2670154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 --2- 172.21.15.43:0/3663925920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f267006cbb0 0x7f2670139f60 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 >> 172.21.15.43:0/3663925920 conn(0x7f26700d00a0 msgr2=0x7f267006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 shutdown_connections 2024-03-30T13:13:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.734+0000 7f2678780640 1 -- 172.21.15.43:0/3663925920 wait complete. 2024-03-30T13:13:47.760 DEBUG:tasks.cephfs.test_uninlining:getting inode info #3 of 23:1099511628287 2024-03-30T13:13:47.760 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628287 2024-03-30T13:13:47.760 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628287 2024-03-30T13:13:47.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.870+0000 7f8466fb0640 1 Processor -- start 2024-03-30T13:13:47.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.870+0000 7f8466fb0640 1 -- start start 2024-03-30T13:13:47.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8466fb0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f84601606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8466fb0640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f846015ff40 0x7f846006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8466fb0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f846006d710 con 0x7f846006cbb0 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8466fb0640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f846006d000 con 0x7f846015ff40 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8466fb0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f846006e320 con 0x7f846013f140 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f845ffff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f846015ff40 0x7f846006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8464d4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f84601606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f846554d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f846006cbb0 0x7f8460141a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47186/0 (socket says 172.21.15.43:47186) 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8464d4c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f84601606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:43508/0 (socket says 172.21.15.43:43508) 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f846554d640 1 -- 172.21.15.43:0/2297181342 learned_addr learned my addr 172.21.15.43:0/2297181342 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f845f7fe640 1 -- 172.21.15.43:0/2297181342 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081248498 0 0) 0x7f846006d710 con 0x7f846006cbb0 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f845f7fe640 1 -- 172.21.15.43:0/2297181342 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8448003530 con 0x7f846006cbb0 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8464d4c640 1 -- 172.21.15.43:0/2297181342 >> v2:172.21.15.113:3300/0 conn(0x7f846015ff40 msgr2=0x7f846006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8464d4c640 1 --2- 172.21.15.43:0/2297181342 >> v2:172.21.15.113:3300/0 conn(0x7f846015ff40 0x7f846006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8464d4c640 1 -- 172.21.15.43:0/2297181342 >> v1:172.21.15.43:6789/0 conn(0x7f846006cbb0 legacy=0x7f8460141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.874+0000 7f8464d4c640 1 -- 172.21.15.43:0/2297181342 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846006d9f0 con 0x7f846013f140 2024-03-30T13:13:47.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8464d4c640 1 --2- 172.21.15.43:0/2297181342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f84601606d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84500097c0 tx=0x7f845001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd1b8d52434be71c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f845f7fe640 1 -- 172.21.15.43:0/2297181342 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84500a7070 con 0x7f846013f140 2024-03-30T13:13:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f845f7fe640 1 -- 172.21.15.43:0/2297181342 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84500b0070 con 0x7f846013f140 2024-03-30T13:13:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f845f7fe640 1 -- 172.21.15.43:0/2297181342 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84500ab870 con 0x7f846013f140 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- 172.21.15.43:0/2297181342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 msgr2=0x7f84601606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 --2- 172.21.15.43:0/2297181342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f84601606d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84500097c0 tx=0x7f845001c470 comp rx=0 tx=0).stop 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- 172.21.15.43:0/2297181342 shutdown_connections 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 --2- 172.21.15.43:0/2297181342 >> v2:172.21.15.113:3300/0 conn(0x7f846015ff40 0x7f846006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 --2- 172.21.15.43:0/2297181342 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f84601606d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- 172.21.15.43:0/2297181342 >> 172.21.15.43:0/2297181342 conn(0x7f84600d00a0 msgr2=0x7f84600d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- 172.21.15.43:0/2297181342 shutdown_connections 2024-03-30T13:13:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- 172.21.15.43:0/2297181342 wait complete. 2024-03-30T13:13:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 Processor -- start 2024-03-30T13:13:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- start start 2024-03-30T13:13:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006cbb0 0x7f8460154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f8460136750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f846015ff40 0x7f846013fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8460160d90 con 0x7f846015ff40 2024-03-30T13:13:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8460160c10 con 0x7f846006cbb0 2024-03-30T13:13:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8466fb0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8460168ff0 con 0x7f846013f140 2024-03-30T13:13:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f846554d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f846015ff40 0x7f846013fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f846554d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f846015ff40 0x7f846013fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59254/0 (socket says 172.21.15.43:59254) 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f846554d640 1 -- 172.21.15.43:0/3855834485 learned_addr learned my addr 172.21.15.43:0/3855834485 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f8464d4c640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006cbb0 0x7f8460154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.878+0000 7f845ffff640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f8460136750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846554d640 1 -- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 msgr2=0x7f8460136750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846554d640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f8460136750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846554d640 1 -- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006cbb0 msgr2=0x7f8460154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846554d640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006cbb0 0x7f8460154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846554d640 1 -- 172.21.15.43:0/3855834485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8460154f50 con 0x7f846015ff40 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846554d640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f846015ff40 0x7f846013fa10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f845800ca50 tx=0x7f845800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f845d7fa640 1 -- 172.21.15.43:0/3855834485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458017070 con 0x7f846015ff40 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f845d7fa640 1 -- 172.21.15.43:0/3855834485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f845800abb0 con 0x7f846015ff40 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846013a930 con 0x7f846015ff40 2024-03-30T13:13:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f845d7fa640 1 -- 172.21.15.43:0/3855834485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458012400 con 0x7f846015ff40 2024-03-30T13:13:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84601552f0 con 0x7f846015ff40 2024-03-30T13:13:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f845d7fa640 1 -- 172.21.15.43:0/3855834485 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f845801a070 con 0x7f846015ff40 2024-03-30T13:13:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f845d7fa640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f844c067fc0 0x7f844c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f845d7fa640 1 -- 172.21.15.43:0/3855834485 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f84580863f0 con 0x7f846015ff40 2024-03-30T13:13:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f8464d4c640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f844c067fc0 0x7f844c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846a132000 1 Processor -- start 2024-03-30T13:13:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f846a132000 1 -- start start 2024-03-30T13:13:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.882+0000 7f8464d4c640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f844c067fc0 0x7f844c06a480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f84500af000 tx=0x7f84500b6040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1cd70 0x55b12ca1d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1d690 0x55b12c9d32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b12c821aa0 con 0x55b12ca1d690 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b12c7a8af0 con 0x55b12c9d3830 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55b12c81da50 con 0x55b12ca1cd70 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1d690 0x55b12c9d32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1d690 0x55b12c9d32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59266/0 (socket says 172.21.15.43:59266) 2024-03-30T13:13:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 -- 172.21.15.43:0/1413851220 learned_addr learned my addr 172.21.15.43:0/1413851220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846554d640 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f8464d4c640 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1cd70 0x55b12ca1d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 -- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1cd70 msgr2=0x55b12ca1d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1cd70 0x55b12ca1d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 -- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 msgr2=0x55b12c9d3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 -- 172.21.15.43:0/1413851220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12c7a6d00 con 0x55b12ca1d690 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f845ffff640 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1d690 0x55b12c9d32f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8454009be0 tx=0x7f8454011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54832d978e75a39 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f84467fc640 1 -- 172.21.15.43:0/1413851220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454019070 con 0x55b12ca1d690 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f84467fc640 1 -- 172.21.15.43:0/1413851220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8454004030 con 0x55b12ca1d690 2024-03-30T13:13:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f84467fc640 1 -- 172.21.15.43:0/1413851220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454004400 con 0x55b12ca1d690 2024-03-30T13:13:47.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- 172.21.15.43:0/1413851220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1d690 msgr2=0x55b12c9d32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1d690 0x55b12c9d32f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8454009be0 tx=0x7f8454011690 comp rx=0 tx=0).stop 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- 172.21.15.43:0/1413851220 shutdown_connections 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1d690 0x55b12c9d32f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- 172.21.15.43:0/1413851220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1cd70 0x55b12ca1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- 172.21.15.43:0/1413851220 >> 172.21.15.43:0/1413851220 conn(0x55b12c9cf7e0 msgr2=0x55b12c9d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- 172.21.15.43:0/1413851220 shutdown_connections 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- 172.21.15.43:0/1413851220 wait complete. 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- start start 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 Processor -- start 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca01da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12ca022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b12c7e9270 con 0x55b12ca1cd70 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55b12c7df6a0 con 0x55b12c9d3830 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55b12c7b9880 con 0x55b12ca1d690 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.886+0000 7f846a132000 10 client.0 _ll_register_callbacks cb 0x55b12c9be490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca01da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca01da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59282/0 (socket says 172.21.15.43:59282) 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 -- 172.21.15.43:0/3857567744 learned_addr learned my addr 172.21.15.43:0/3857567744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f8464d4c640 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846554d640 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12ca022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 -- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 msgr2=0x55b12ca022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12ca022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 -- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 msgr2=0x55b12c9d8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 -- 172.21.15.43:0/3857567744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12c7e6730 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f845ffff640 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca01da0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55b12c8e1c80 tx=0x7f8454004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22928a46fb4a0a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 1 -- 172.21.15.43:0/3857567744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454019040 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 1 -- 172.21.15.43:0/3857567744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f8454003b60 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8454004030 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 1 -- 172.21.15.43:0/3857567744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8454030050 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 10 client.5886 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55b12c7d9ca0 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 1 -- 172.21.15.43:0/3857567744 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8454004030 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 1 -- 172.21.15.43:0/3857567744 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f8454033020 con 0x55b12ca1cd70 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 10 client.5886 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 10 client.5886 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 20 client.5886 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca245d0 0x55b12ca249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 4 client.5886 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b12c7bcc90 con 0x55b12ca245d0 2024-03-30T13:13:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f8464d4c640 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca245d0 0x55b12ca249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f8464d4c640 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca245d0 0x55b12ca249e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a088c212638655 server_cookie=f0bbc243aab27b14 in_seq=0 out_seq=0 2024-03-30T13:13:47.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 10 client.5886 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 1 -- 172.21.15.43:0/3857567744 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55b12c7bcc90 con 0x55b12ca245d0 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f843f7fe640 10 client.5886 handle_command_reply: tid=0 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.890+0000 7f846a132000 1 client.5886 shutdown 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 msgr2=0x55b12ca01da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca01da0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x55b12c8e1c80 tx=0x7f8454004740 comp rx=0 tx=0).stop 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 shutdown_connections 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca245d0 0x55b12ca249e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12ca022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca01da0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 --2- 172.21.15.43:0/3857567744 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f843f7fe640 0 client.5886 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 >> 172.21.15.43:0/3857567744 conn(0x55b12c9cf7e0 msgr2=0x55b12c9d2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 shutdown_connections 2024-03-30T13:13:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 1 -- 172.21.15.43:0/3857567744 wait complete. 2024-03-30T13:13:47.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.894+0000 7f846a132000 20 client.5886 trim_cache size 0 max 16384 2024-03-30T13:13:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 Processor -- start 2024-03-30T13:13:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 -- start start 2024-03-30T13:13:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12c9d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12c9d2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8460201e00 con 0x55b12ca1cd70 2024-03-30T13:13:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8460200de0 con 0x55b12c9d3830 2024-03-30T13:13:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f846013dd00 con 0x55b12ca1d690 2024-03-30T13:13:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12c9d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12c9d2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59292/0 (socket says 172.21.15.43:59292) 2024-03-30T13:13:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f845ffff640 1 -- 172.21.15.43:0/3949300002 learned_addr learned my addr 172.21.15.43:0/3949300002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.906+0000 7f8464d4c640 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846554d640 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12c9d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 -- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 msgr2=0x55b12c9d2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12c9d2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 -- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 msgr2=0x55b12c9d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 -- 172.21.15.43:0/3949300002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b12c7a8d00 con 0x55b12ca1cd70 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12c9d2110 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8454001ec0 tx=0x7f8454024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1423bf099d176b8e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f8445ffb640 1 -- 172.21.15.43:0/3949300002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454019070 con 0x55b12ca1cd70 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f8445ffb640 1 -- 172.21.15.43:0/3949300002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8454000e50 con 0x55b12ca1cd70 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f8445ffb640 1 -- 172.21.15.43:0/3949300002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454022040 con 0x55b12ca1cd70 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- 172.21.15.43:0/3949300002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 msgr2=0x55b12c9d2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12c9d2110 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8454001ec0 tx=0x7f8454024000 comp rx=0 tx=0).stop 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- 172.21.15.43:0/3949300002 shutdown_connections 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12ca1d690 0x55b12c9d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12c9d2110 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 --2- 172.21.15.43:0/3949300002 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12c9d3830 0x55b12c9d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- 172.21.15.43:0/3949300002 >> 172.21.15.43:0/3949300002 conn(0x55b12c9cf7e0 msgr2=0x55b12ca33680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- 172.21.15.43:0/3949300002 shutdown_connections 2024-03-30T13:13:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- 172.21.15.43:0/3949300002 wait complete. 2024-03-30T13:13:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- start start 2024-03-30T13:13:47.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 Processor -- start 2024-03-30T13:13:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12c9d3830 0x55b12ca8ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12ca1d690 0x55b12ca20ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55b12c7b9880 con 0x55b12ca1cd70 2024-03-30T13:13:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f84602007b0 con 0x55b12ca1d690 2024-03-30T13:13:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55b12ca34fb0 con 0x55b12c9d3830 2024-03-30T13:13:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 10 client.0 _ll_register_callbacks cb 0x7f8454009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca8f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59296/0 (socket says 172.21.15.43:59296) 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 -- 172.21.15.43:0/1932739902 learned_addr learned my addr 172.21.15.43:0/1932739902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f8464d4c640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12c9d3830 0x55b12ca8ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846554d640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12ca1d690 0x55b12ca20ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 -- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12c9d3830 msgr2=0x55b12ca8ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12c9d3830 0x55b12ca8ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 -- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12ca1d690 msgr2=0x55b12ca20ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12ca1d690 0x55b12ca20ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 -- 172.21.15.43:0/1932739902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b12c8a2f70 con 0x55b12ca1cd70 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846554d640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12ca1d690 0x55b12ca20ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f845ffff640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca8f180 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f84540046a0 tx=0x7f8454009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340a9b30787ebbb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f843f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f843f7fe640 1 -- 172.21.15.43:0/1932739902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454043060 con 0x55b12ca1cd70 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f843f7fe640 1 -- 172.21.15.43:0/1932739902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f8454025040 con 0x55b12ca1cd70 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8454000e50 con 0x55b12ca1cd70 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f843f7fe640 1 -- 172.21.15.43:0/1932739902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8454049050 con 0x55b12ca1cd70 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 10 client.5892 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.910+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55b12c7c8760 con 0x55b12ca1cd70 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f843f7fe640 1 -- 172.21.15.43:0/1932739902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8454000e50 con 0x55b12ca1cd70 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f843f7fe640 1 -- 172.21.15.43:0/1932739902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f845404c020 con 0x55b12ca1cd70 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 10 client.5892 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 10 client.5892 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 20 client.5892 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca21490 0x55b12ca218a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 4 client.5892 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628287}] 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628287}) -- 0x55b12c82efe0 con 0x55b12ca21490 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8464d4c640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca21490 0x55b12ca218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8464d4c640 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca21490 0x55b12ca218a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc0c80440158591 server_cookie=2dc5ff1c0cc7c458 in_seq=0 out_seq=0 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f843f7fe640 10 client.5892 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f843f7fe640 1 -- 172.21.15.43:0/1932739902 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3815 (crc 0 0 0) 0x55b12c82efe0 con 0x55b12ca21490 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f843f7fe640 10 client.5892 handle_command_reply: tid=0 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 client.5892 shutdown 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 msgr2=0x55b12ca8f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca8f180 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f84540046a0 tx=0x7f8454009190 comp rx=0 tx=0).stop 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 shutdown_connections 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55b12ca21490 0x55b12ca218a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55b12ca1d690 0x55b12ca20ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b12ca1cd70 0x55b12ca8f180 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 --2- 172.21.15.43:0/1932739902 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55b12c9d3830 0x55b12ca8ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f843f7fe640 0 client.5892 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 >> 172.21.15.43:0/1932739902 conn(0x55b12c9cf7e0 msgr2=0x55b12ca33f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 shutdown_connections 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 1 -- 172.21.15.43:0/1932739902 wait complete. 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f846a132000 20 client.5892 trim_cache size 0 max 16384 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/ceph_frag.c", 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628287, 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.898199+0000", 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.241565+0000", 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:47.923 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.924 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 401, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.241565+0000", 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:47.925 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 401, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.898199+0000" 2024-03-30T13:13:47.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 401, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.898199+0000" 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 114, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 16, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:47.927 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.928 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:47.929 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:47.930 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:47.931 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:47.932 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.933 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:47.934 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f844c067fc0 msgr2=0x7f844c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f844c067fc0 0x7f844c06a480 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f84500af000 tx=0x7f84500b6040 comp rx=0 tx=0).stop 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f846015ff40 msgr2=0x7f846013fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f846015ff40 0x7f846013fa10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f845800ca50 tx=0x7f845800cf10 comp rx=0 tx=0).stop 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 shutdown_connections 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f844c067fc0 0x7f844c06a480 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f846015ff40 0x7f846013fa10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f846013f140 0x7f8460136750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 --2- 172.21.15.43:0/3855834485 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f846006cbb0 0x7f8460154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 >> 172.21.15.43:0/3855834485 conn(0x7f84600d00a0 msgr2=0x7f84600b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:47.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 shutdown_connections 2024-03-30T13:13:47.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:47.914+0000 7f8466fb0640 1 -- 172.21.15.43:0/3855834485 wait complete. 2024-03-30T13:13:47.937 DEBUG:tasks.cephfs.test_uninlining:getting inode info #4 of 23:1099511628294 2024-03-30T13:13:47.937 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628294 2024-03-30T13:13:47.937 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628294 2024-03-30T13:13:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f6585c66640 1 Processor -- start 2024-03-30T13:13:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f6585c66640 1 -- start start 2024-03-30T13:13:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f6585c66640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65800ccc00 0x7f65800b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f6585c66640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6580158f20 con 0x7f6580158b40 2024-03-30T13:13:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f6585c66640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65801620f0 con 0x7f65800cc750 2024-03-30T13:13:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f6585c66640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6580162d00 con 0x7f65800ccc00 2024-03-30T13:13:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6580158b40 0x7f65800b4ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47188/0 (socket says 172.21.15.43:47188) 2024-03-30T13:13:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657f7fe640 1 -- 172.21.15.43:0/1633750892 learned_addr learned my addr 172.21.15.43:0/1633750892 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:48.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657ffff640 1 --2- 172.21.15.43:0/1633750892 >> v2:172.21.15.43:3300/0 conn(0x7f65800ccc00 0x7f65800b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657e7fc640 1 -- 172.21.15.43:0/1633750892 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911065743 0 0) 0x7f6580158f20 con 0x7f6580158b40 2024-03-30T13:13:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657e7fc640 1 -- 172.21.15.43:0/1633750892 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f656c003530 con 0x7f6580158b40 2024-03-30T13:13:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657ffff640 1 -- 172.21.15.43:0/1633750892 >> v1:172.21.15.113:6789/0 conn(0x7f65800cc750 legacy=0x7f65800b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657ffff640 1 -- 172.21.15.43:0/1633750892 >> v1:172.21.15.43:6789/0 conn(0x7f6580158b40 legacy=0x7f65800b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.054+0000 7f657ffff640 1 -- 172.21.15.43:0/1633750892 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65801623d0 con 0x7f65800ccc00 2024-03-30T13:13:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f657ffff640 1 --2- 172.21.15.43:0/1633750892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65800ccc00 0x7f65800b7360 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f65700097c0 tx=0x7f657001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6993c1fd24d120 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f657e7fc640 1 -- 172.21.15.43:0/1633750892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700b1070 con 0x7f65800ccc00 2024-03-30T13:13:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f657e7fc640 1 -- 172.21.15.43:0/1633750892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65700a8050 con 0x7f65800ccc00 2024-03-30T13:13:48.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f657e7fc640 1 -- 172.21.15.43:0/1633750892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700ac860 con 0x7f65800ccc00 2024-03-30T13:13:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 -- 172.21.15.43:0/1633750892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65800ccc00 msgr2=0x7f65800b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 --2- 172.21.15.43:0/1633750892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65800ccc00 0x7f65800b7360 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f65700097c0 tx=0x7f657001c3e0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 -- 172.21.15.43:0/1633750892 shutdown_connections 2024-03-30T13:13:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 --2- 172.21.15.43:0/1633750892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65800ccc00 0x7f65800b7360 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 -- 172.21.15.43:0/1633750892 >> 172.21.15.43:0/1633750892 conn(0x7f65800b3cf0 msgr2=0x7f65800b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 -- 172.21.15.43:0/1633750892 shutdown_connections 2024-03-30T13:13:48.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 -- 172.21.15.43:0/1633750892 wait complete. 2024-03-30T13:13:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 Processor -- start 2024-03-30T13:13:48.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.058+0000 7f6585c66640 1 -- start start 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65800cc750 0x7f6580149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65800ccc00 0x7f65800c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580158b40 0x7f65800ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f65800d1d30 con 0x7f6580158b40 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65800b5700 con 0x7f65800cc750 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f65800d1eb0 con 0x7f65800ccc00 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580158b40 0x7f65800ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580158b40 0x7f65800ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59320/0 (socket says 172.21.15.43:59320) 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 -- 172.21.15.43:0/1341903023 learned_addr learned my addr 172.21.15.43:0/1341903023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65800cc750 0x7f6580149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657effd640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65800ccc00 0x7f65800c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 -- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65800ccc00 msgr2=0x7f65800c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65800ccc00 0x7f65800c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 -- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65800cc750 msgr2=0x7f6580149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65800cc750 0x7f6580149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 -- 172.21.15.43:0/1341903023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65800c5a20 con 0x7f6580158b40 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f657ffff640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580158b40 0x7f65800ce780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6570018ff0 tx=0x7f657001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6577fff640 1 -- 172.21.15.43:0/1341903023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700b1040 con 0x7f6580158b40 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6577fff640 1 -- 172.21.15.43:0/1341903023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65700bc070 con 0x7f6580158b40 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658014a950 con 0x7f6580158b40 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6577fff640 1 -- 172.21.15.43:0/1341903023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65700b7450 con 0x7f6580158b40 2024-03-30T13:13:48.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6580150620 con 0x7f6580158b40 2024-03-30T13:13:48.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.062+0000 7f6577fff640 1 -- 172.21.15.43:0/1341903023 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f65700b75f0 con 0x7f6580158b40 2024-03-30T13:13:48.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6577fff640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6560067fc0 0x7f656006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6577fff640 1 -- 172.21.15.43:0/1341903023 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f6570120970 con 0x7f6580158b40 2024-03-30T13:13:48.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6560067fc0 0x7f656006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 Processor -- start 2024-03-30T13:13:48.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- start start 2024-03-30T13:13:48.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6560067fc0 0x7f656006a480 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6578001570 tx=0x7f6578009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef0404d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04003c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef03e51aa0 con 0x55ef0404cd70 2024-03-30T13:13:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ef03dd8af0 con 0x55ef0404d690 2024-03-30T13:13:48.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55ef03e4da50 con 0x55ef04003830 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef0404d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef0404d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59330/0 (socket says 172.21.15.43:59330) 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 -- 172.21.15.43:0/92918850 learned_addr learned my addr 172.21.15.43:0/92918850 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657effd640 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657ffff640 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04003c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 -- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 msgr2=0x55ef04003c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04003c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 -- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 msgr2=0x55ef040032f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040032f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 -- 172.21.15.43:0/92918850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef03dd6d00 con 0x55ef0404cd70 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f657f7fe640 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef0404d150 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6578002240 tx=0x7f6578011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3b77d4f784fdf5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f65757fa640 1 -- 172.21.15.43:0/92918850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578007ae0 con 0x55ef0404cd70 2024-03-30T13:13:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f65757fa640 1 -- 172.21.15.43:0/92918850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6578013630 con 0x55ef0404cd70 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f65757fa640 1 -- 172.21.15.43:0/92918850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65780139e0 con 0x55ef0404cd70 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- 172.21.15.43:0/92918850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 msgr2=0x55ef0404d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef0404d150 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6578002240 tx=0x7f6578011000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- 172.21.15.43:0/92918850 shutdown_connections 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04003c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 --2- 172.21.15.43:0/92918850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef0404d150 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- 172.21.15.43:0/92918850 >> 172.21.15.43:0/92918850 conn(0x55ef03fff7e0 msgr2=0x55ef04001bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- 172.21.15.43:0/92918850 shutdown_connections 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- 172.21.15.43:0/92918850 wait complete. 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 -- start start 2024-03-30T13:13:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.066+0000 7f6588de8000 1 Processor -- start 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef04008c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404cd70 0x55ef04031da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef03e19270 con 0x55ef04003830 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55ef03e0f6a0 con 0x55ef0404d690 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55ef03de9880 con 0x55ef0404cd70 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 10 client.0 _ll_register_callbacks cb 0x55ef03fee490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef04008c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef04008c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59336/0 (socket says 172.21.15.43:59336) 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 -- 172.21.15.43:0/1796663519 learned_addr learned my addr 172.21.15.43:0/1796663519 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657ffff640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657effd640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404cd70 0x55ef04031da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 -- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404cd70 msgr2=0x55ef04031da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404cd70 0x55ef04031da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 -- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 msgr2=0x55ef040322e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040322e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 -- 172.21.15.43:0/1796663519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef03e16730 con 0x55ef04003830 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657ffff640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef04008c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55ef03f11c80 tx=0x7f6578007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268a4e626a17bace server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f655e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f655e7fc640 1 -- 172.21.15.43:0/1796663519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578013da0 con 0x55ef04003830 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f655e7fc640 1 -- 172.21.15.43:0/1796663519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f6578004b70 con 0x55ef04003830 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6578013630 con 0x55ef04003830 2024-03-30T13:13:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f655e7fc640 1 -- 172.21.15.43:0/1796663519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6578032050 con 0x55ef04003830 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 10 client.5907 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55ef03e09ca0 con 0x55ef04003830 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f655e7fc640 1 -- 172.21.15.43:0/1796663519 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6578013630 con 0x55ef04003830 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f655e7fc640 1 -- 172.21.15.43:0/1796663519 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f6578035020 con 0x55ef04003830 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 10 client.5907 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 10 client.5907 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 20 client.5907 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040545d0 0x55ef040549b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 4 client.5907 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef03decc90 con 0x55ef040545d0 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657effd640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040545d0 0x55ef040549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f655e7fc640 10 client.5907 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.070+0000 7f657effd640 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040545d0 0x55ef040549b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe5e8751e6d2b17 server_cookie=348de9cbcfa9b22d in_seq=0 out_seq=0 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f655e7fc640 1 -- 172.21.15.43:0/1796663519 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55ef03decc90 con 0x55ef040545d0 2024-03-30T13:13:48.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f655e7fc640 10 client.5907 handle_command_reply: tid=0 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 client.5907 shutdown 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 msgr2=0x55ef04008c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef04008c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55ef03f11c80 tx=0x7f6578007000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 shutdown_connections 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040545d0 0x55ef040549b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef040322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404cd70 0x55ef04031da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 --2- 172.21.15.43:0/1796663519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef04008c80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f655e7fc640 0 client.5907 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 >> 172.21.15.43:0/1796663519 conn(0x55ef03fff7e0 msgr2=0x55ef04002460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 shutdown_connections 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 1 -- 172.21.15.43:0/1796663519 wait complete. 2024-03-30T13:13:48.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.074+0000 7f6588de8000 20 client.5907 trim_cache size 0 max 16384 2024-03-30T13:13:48.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 Processor -- start 2024-03-30T13:13:48.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- start start 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04030d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef04031290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef04031a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6580203f50 con 0x55ef0404cd70 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6580202ff0 con 0x55ef0404d690 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f65800c7030 con 0x55ef04003830 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef04031290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef04031290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59348/0 (socket says 172.21.15.43:59348) 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 -- 172.21.15.43:0/3374948134 learned_addr learned my addr 172.21.15.43:0/3374948134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657f7fe640 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04030d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657ffff640 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef04031a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 -- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 msgr2=0x55ef04030d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04030d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 -- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 msgr2=0x55ef04031a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef04031a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 -- 172.21.15.43:0/3374948134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef03dd8d00 con 0x55ef0404cd70 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f657effd640 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef04031290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6568000fc0 tx=0x7f6568009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1085a8272beba99 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6574ff9640 1 -- 172.21.15.43:0/3374948134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656801a1f0 con 0x55ef0404cd70 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6574ff9640 1 -- 172.21.15.43:0/3374948134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656801a390 con 0x55ef0404cd70 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6574ff9640 1 -- 172.21.15.43:0/3374948134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656801a7c0 con 0x55ef0404cd70 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- 172.21.15.43:0/3374948134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 msgr2=0x55ef04031290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef04031290 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6568000fc0 tx=0x7f6568009040 comp rx=0 tx=0).stop 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- 172.21.15.43:0/3374948134 shutdown_connections 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404d690 0x55ef04031a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef0404cd70 0x55ef04031290 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- 172.21.15.43:0/3374948134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef04003830 0x55ef04030d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- 172.21.15.43:0/3374948134 >> 172.21.15.43:0/3374948134 conn(0x55ef03fff7e0 msgr2=0x55ef04063680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- 172.21.15.43:0/3374948134 shutdown_connections 2024-03-30T13:13:48.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- 172.21.15.43:0/3374948134 wait complete. 2024-03-30T13:13:48.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- start start 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 Processor -- start 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef040bde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404cd70 0x55ef04050ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404d690 0x55ef04051010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55ef03de9880 con 0x55ef04003830 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f65802029a0 con 0x55ef0404cd70 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.090+0000 7f6588de8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55ef03e50db0 con 0x55ef0404d690 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 10 client.0 _ll_register_callbacks cb 0x7f657800ee00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef040bde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef040bde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59362/0 (socket says 172.21.15.43:59362) 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 -- 172.21.15.43:0/1115934789 learned_addr learned my addr 172.21.15.43:0/1115934789 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657effd640 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404cd70 0x55ef04050ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657ffff640 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404d690 0x55ef04051010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 -- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404d690 msgr2=0x55ef04051010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404d690 0x55ef04051010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 -- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404cd70 msgr2=0x55ef04050ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404cd70 0x55ef04050ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 -- 172.21.15.43:0/1115934789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef03ed2f70 con 0x55ef04003830 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657f7fe640 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef040bde40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6578001e50 tx=0x7f6578013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8026a95253d436b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 1 -- 172.21.15.43:0/1115934789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578013630 con 0x55ef04003830 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 1 -- 172.21.15.43:0/1115934789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f657802d560 con 0x55ef04003830 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656801a390 con 0x55ef04003830 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 1 -- 172.21.15.43:0/1115934789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6578042050 con 0x55ef04003830 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 10 client.5913 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55ef03df8760 con 0x55ef04003830 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 1 -- 172.21.15.43:0/1115934789 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f656801a390 con 0x55ef04003830 2024-03-30T13:13:48.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 1 -- 172.21.15.43:0/1115934789 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f6578045020 con 0x55ef04003830 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 10 client.5913 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 10 client.5913 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 20 client.5913 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040519d0 0x55ef04051db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 4 client.5913 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628294}] 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628294}) -- 0x55ef03e5efe0 con 0x55ef040519d0 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657effd640 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040519d0 0x55ef04051db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f657effd640 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040519d0 0x55ef04051db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f95466a043042ee server_cookie=cea1c3f4820c269e in_seq=0 out_seq=0 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 10 client.5913 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 1 -- 172.21.15.43:0/1115934789 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3808 (crc 0 0 0) 0x55ef03e5efe0 con 0x55ef040519d0 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 10 client.5913 handle_command_reply: tid=0 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 client.5913 shutdown 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 msgr2=0x55ef040bde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef040bde40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6578001e50 tx=0x7f6578013000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 shutdown_connections 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55ef040519d0 0x55ef04051db0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55ef0404d690 0x55ef04051010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55ef0404cd70 0x55ef04050ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 --2- 172.21.15.43:0/1115934789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55ef04003830 0x55ef040bde40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 >> 172.21.15.43:0/1115934789 conn(0x55ef03fff7e0 msgr2=0x55ef04002460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f655e7fc640 0 client.5913 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 shutdown_connections 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 1 -- 172.21.15.43:0/1115934789 wait complete. 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6588de8000 20 client.5913 trim_cache size 0 max 16384 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:48.104 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/io.h", 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628294, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.904422+0000", 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.308243+0000", 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.105 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 387, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.308243+0000", 2024-03-30T13:13:48.106 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.107 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 387, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.904422+0000" 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 387, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.904422+0000" 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 116, 2024-03-30T13:13:48.108 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 30, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:48.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.110 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.111 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.112 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.113 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:48.114 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:48.115 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:48.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6560067fc0 msgr2=0x7f656006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6560067fc0 0x7f656006a480 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6578001570 tx=0x7f6578009040 comp rx=0 tx=0).stop 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580158b40 msgr2=0x7f65800ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580158b40 0x7f65800ce780 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6570018ff0 tx=0x7f657001ac20 comp rx=0 tx=0).stop 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 shutdown_connections 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6560067fc0 0x7f656006a480 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6580158b40 0x7f65800ce780 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f65800ccc00 0x7f65800c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 --2- 172.21.15.43:0/1341903023 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f65800cc750 0x7f6580149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 >> 172.21.15.43:0/1341903023 conn(0x7f65800b3cf0 msgr2=0x7f658014dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 shutdown_connections 2024-03-30T13:13:48.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.094+0000 7f6585c66640 1 -- 172.21.15.43:0/1341903023 wait complete. 2024-03-30T13:13:48.117 DEBUG:tasks.cephfs.test_uninlining:getting inode info #5 of 23:1099511628296 2024-03-30T13:13:48.118 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628296 2024-03-30T13:13:48.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628296 2024-03-30T13:13:48.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b9cfb640 1 Processor -- start 2024-03-30T13:13:48.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b9cfb640 1 -- start start 2024-03-30T13:13:48.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b9cfb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f18b415ff40 0x7f18b406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b9cfb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b406d710 con 0x7f18b406cbb0 2024-03-30T13:13:48.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b9cfb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b406d9f0 con 0x7f18b413f140 2024-03-30T13:13:48.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b9cfb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f18b406e320 con 0x7f18b415ff40 2024-03-30T13:13:48.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f18b406cbb0 0x7f18b40ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47198/0 (socket says 172.21.15.43:47198) 2024-03-30T13:13:48.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b3fff640 1 -- 172.21.15.43:0/2136057964 learned_addr learned my addr 172.21.15.43:0/2136057964 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:48.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/2136057964 >> v2:172.21.15.113:3300/0 conn(0x7f18b415ff40 0x7f18b406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b27fc640 1 -- 172.21.15.43:0/2136057964 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315720894 0 0) 0x7f18b406d710 con 0x7f18b406cbb0 2024-03-30T13:13:48.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.230+0000 7f18b27fc640 1 -- 172.21.15.43:0/2136057964 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f189c003530 con 0x7f18b406cbb0 2024-03-30T13:13:48.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b2ffd640 1 -- 172.21.15.43:0/2136057964 >> v1:172.21.15.113:6789/0 conn(0x7f18b413f140 legacy=0x7f18b41606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b2ffd640 1 -- 172.21.15.43:0/2136057964 >> v1:172.21.15.43:6789/0 conn(0x7f18b406cbb0 legacy=0x7f18b40ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b27fc640 1 -- 172.21.15.43:0/2136057964 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2444412757 0 0) 0x7f189c003530 con 0x7f18b406cbb0 2024-03-30T13:13:48.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b2ffd640 1 -- 172.21.15.43:0/2136057964 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b406dcd0 con 0x7f18b415ff40 2024-03-30T13:13:48.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/2136057964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 0x7f18b406c670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18a80097c0 tx=0x7f18a801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9beefea4df667630 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b27fc640 1 -- 172.21.15.43:0/2136057964 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80b0070 con 0x7f18b415ff40 2024-03-30T13:13:48.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b27fc640 1 -- 172.21.15.43:0/2136057964 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a80ab400 con 0x7f18b415ff40 2024-03-30T13:13:48.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b27fc640 1 -- 172.21.15.43:0/2136057964 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac001680 con 0x7f18b415ff40 2024-03-30T13:13:48.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b9cfb640 1 -- 172.21.15.43:0/2136057964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 msgr2=0x7f18b406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/2136057964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 0x7f18b406c670 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f18a80097c0 tx=0x7f18a801c3c0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b9cfb640 1 -- 172.21.15.43:0/2136057964 shutdown_connections 2024-03-30T13:13:48.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/2136057964 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 0x7f18b406c670 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b9cfb640 1 -- 172.21.15.43:0/2136057964 >> 172.21.15.43:0/2136057964 conn(0x7f18b40d00a0 msgr2=0x7f18b40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b9cfb640 1 -- 172.21.15.43:0/2136057964 shutdown_connections 2024-03-30T13:13:48.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.234+0000 7f18b9cfb640 1 -- 172.21.15.43:0/2136057964 wait complete. 2024-03-30T13:13:48.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 Processor -- start 2024-03-30T13:13:48.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 -- start start 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b406cbb0 0x7f18b4136750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b413f140 0x7f18b413d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 0x7f18b4155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18b40cfc30 con 0x7f18b413f140 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18b40cfab0 con 0x7f18b415ff40 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f18b416a000 con 0x7f18b406cbb0 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b413f140 0x7f18b413d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b413f140 0x7f18b413d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59368/0 (socket says 172.21.15.43:59368) 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 -- 172.21.15.43:0/3445156580 learned_addr learned my addr 172.21.15.43:0/3445156580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b37fe640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b406cbb0 0x7f18b4136750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b3fff640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 0x7f18b4155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 -- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b406cbb0 msgr2=0x7f18b4136750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b406cbb0 0x7f18b4136750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 -- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 msgr2=0x7f18b4155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 0x7f18b4155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 -- 172.21.15.43:0/3445156580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b4155ea0 con 0x7f18b413f140 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b413f140 0x7f18b413d360 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ae040 tx=0x7f18a801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f189bfff640 1 -- 172.21.15.43:0/3445156580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80bb020 con 0x7f18b413f140 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f189bfff640 1 -- 172.21.15.43:0/3445156580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a80b1040 con 0x7f18b413f140 2024-03-30T13:13:48.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f189bfff640 1 -- 172.21.15.43:0/3445156580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80b0050 con 0x7f18b413f140 2024-03-30T13:13:48.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b413dad0 con 0x7f18b413f140 2024-03-30T13:13:48.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.238+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b413e090 con 0x7f18b413f140 2024-03-30T13:13:48.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f189bfff640 1 -- 172.21.15.43:0/3445156580 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f18a80be070 con 0x7f18b413f140 2024-03-30T13:13:48.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f189bfff640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f18a0067ff0 0x7f18a006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f189bfff640 1 -- 172.21.15.43:0/3445156580 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f18a811fe80 con 0x7f18b413f140 2024-03-30T13:13:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b37fe640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f18a0067ff0 0x7f18a006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 Processor -- start 2024-03-30T13:13:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 -- start start 2024-03-30T13:13:48.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b37fe640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f18a0067ff0 0x7f18a006a4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f18b415bfd0 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a6865599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a6865072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686507830 0x55a686507c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a686355aa0 con 0x55a686559ee0 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a6862dcaf0 con 0x55a686507830 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a686351a50 con 0x55a6865595c0 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a6865072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a6865072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59378/0 (socket says 172.21.15.43:59378) 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 -- 172.21.15.43:0/1294772430 learned_addr learned my addr 172.21.15.43:0/1294772430 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a6865599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b3fff640 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686507830 0x55a686507c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 -- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 msgr2=0x55a6865599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a6865599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 -- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686507830 msgr2=0x55a686507c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686507830 0x55a686507c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 -- 172.21.15.43:0/1294772430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6862dad00 con 0x55a686559ee0 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a6865599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b3fff640 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686507830 0x55a686507c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a6865072f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f18a80f8300 tx=0x7f18a8019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2be9f71b6ecddd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18997fa640 1 -- 172.21.15.43:0/1294772430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80efc50 con 0x55a686559ee0 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18997fa640 1 -- 172.21.15.43:0/1294772430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a80f4000 con 0x55a686559ee0 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18997fa640 1 -- 172.21.15.43:0/1294772430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80f8830 con 0x55a686559ee0 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 -- 172.21.15.43:0/1294772430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 msgr2=0x55a6865072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a6865072f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f18a80f8300 tx=0x7f18a8019160 comp rx=0 tx=0).stop 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 -- 172.21.15.43:0/1294772430 shutdown_connections 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686507830 0x55a686507c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a6865072f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.242+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1294772430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a6865599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- 172.21.15.43:0/1294772430 >> 172.21.15.43:0/1294772430 conn(0x55a6865037e0 msgr2=0x55a686505bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- 172.21.15.43:0/1294772430 shutdown_connections 2024-03-30T13:13:48.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- 172.21.15.43:0/1294772430 wait complete. 2024-03-30T13:13:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- start start 2024-03-30T13:13:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 Processor -- start 2024-03-30T13:13:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686507830 0x55a68650cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a68651b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686559ee0 0x55a68651b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a68631d270 con 0x55a686507830 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a6863136a0 con 0x55a686559ee0 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a6862ed880 con 0x55a6865595c0 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 10 client.0 _ll_register_callbacks cb 0x55a6864f2490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686507830 0x55a68650cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686507830 0x55a68650cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59382/0 (socket says 172.21.15.43:59382) 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 -- 172.21.15.43:0/1777552888 learned_addr learned my addr 172.21.15.43:0/1777552888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b3fff640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686559ee0 0x55a68651b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a68651b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 -- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 msgr2=0x55a68651b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a68651b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 -- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686559ee0 msgr2=0x55a68651b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686559ee0 0x55a68651b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 -- 172.21.15.43:0/1777552888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a68631a730 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b3fff640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686559ee0 0x55a68651b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686507830 0x55a68650cc40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f18a400eb50 tx=0x7f18a4001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a479fba1f70958bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18927fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18927fc640 1 -- 172.21.15.43:0/1777552888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a4022430 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18927fc640 1 -- 172.21.15.43:0/1777552888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f18a4021c40 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a80f4000 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18927fc640 1 -- 172.21.15.43:0/1777552888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f18a4021e40 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 10 client.5925 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55a68630dca0 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18927fc640 1 -- 172.21.15.43:0/1777552888 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a80f4000 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18927fc640 1 -- 172.21.15.43:0/1777552888 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f18a4021420 con 0x55a686507830 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 10 client.5925 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 10 client.5925 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 20 client.5925 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a686560c10 0x55a686561020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 4 client.5925 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6862f0c90 con 0x55a686560c10 2024-03-30T13:13:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a686560c10 0x55a686561020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18927fc640 10 client.5925 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.246+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a686560c10 0x55a686561020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a6a043262f5379 server_cookie=da4715c0f8b17d2d in_seq=0 out_seq=0 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18927fc640 1 -- 172.21.15.43:0/1777552888 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55a6862f0c90 con 0x55a686560c10 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18927fc640 10 client.5925 handle_command_reply: tid=0 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 client.5925 shutdown 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686507830 msgr2=0x55a68650cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686507830 0x55a68650cc40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f18a400eb50 tx=0x7f18a4001260 comp rx=0 tx=0).stop 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 shutdown_connections 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a686560c10 0x55a686561020 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a686559ee0 0x55a68651b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18927fc640 0 client.5925 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a6865595c0 0x55a68651b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1777552888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686507830 0x55a68650cc40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 >> 172.21.15.43:0/1777552888 conn(0x55a6865037e0 msgr2=0x55a686506460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 shutdown_connections 2024-03-30T13:13:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 1 -- 172.21.15.43:0/1777552888 wait complete. 2024-03-30T13:13:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.250+0000 7f18bce7d000 20 client.5925 trim_cache size 0 max 16384 2024-03-30T13:13:48.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 Processor -- start 2024-03-30T13:13:48.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- start start 2024-03-30T13:13:48.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a686505bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a686506110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a686506650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4159c80 con 0x55a686559ee0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4158dc0 con 0x55a6865595c0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f18b4158120 con 0x55a686507830 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a686506650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a686506650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59396/0 (socket says 172.21.15.43:59396) 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 -- 172.21.15.43:0/2886847817 learned_addr learned my addr 172.21.15.43:0/2886847817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b37fe640 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a686505bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a686506110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 -- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 msgr2=0x55a686505bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a686505bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 -- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 msgr2=0x55a686506110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a686506110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 -- 172.21.15.43:0/2886847817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6862dcd00 con 0x55a686559ee0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a686506650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0016a0 tx=0x7f18ac006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9045ff6885f0ad3d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f1898ff9640 1 -- 172.21.15.43:0/2886847817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac018020 con 0x55a686559ee0 2024-03-30T13:13:48.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f1898ff9640 1 -- 172.21.15.43:0/2886847817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac0109c0 con 0x55a686559ee0 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f1898ff9640 1 -- 172.21.15.43:0/2886847817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac010db0 con 0x55a686559ee0 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- 172.21.15.43:0/2886847817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 msgr2=0x55a686506650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a686506650 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f18ac0016a0 tx=0x7f18ac006cc0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- 172.21.15.43:0/2886847817 shutdown_connections 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a686506650 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a686506110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- 172.21.15.43:0/2886847817 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a686505bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- 172.21.15.43:0/2886847817 >> 172.21.15.43:0/2886847817 conn(0x55a6865037e0 msgr2=0x55a686570550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- 172.21.15.43:0/2886847817 shutdown_connections 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- 172.21.15.43:0/2886847817 wait complete. 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- start start 2024-03-30T13:13:48.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 Processor -- start 2024-03-30T13:13:48.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a6865caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a68655d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a68655d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a686411b00 con 0x55a686559ee0 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f18b4158770 con 0x55a6865595c0 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a68631d270 con 0x55a686507830 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18bce7d000 10 client.0 _ll_register_callbacks cb 0x7f18a4001ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a68655d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.266+0000 7f18b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a68655d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59408/0 (socket says 172.21.15.43:59408) 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b3fff640 1 -- 172.21.15.43:0/1295095990 learned_addr learned my addr 172.21.15.43:0/1295095990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a68655d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a6865caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b3fff640 1 -- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 msgr2=0x55a6865caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b3fff640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a6865caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b3fff640 1 -- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 msgr2=0x55a68655d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b3fff640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a68655d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b3fff640 1 -- 172.21.15.43:0/1295095990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6863d6f70 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b2ffd640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a68655d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b3fff640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a68655d860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f18ac001980 tx=0x7f18ac000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe508e18bc03b53a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 1 -- 172.21.15.43:0/1295095990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac02c060 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 1 -- 172.21.15.43:0/1295095990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f18ac01f960 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac0109c0 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 1 -- 172.21.15.43:0/1295095990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f18ac032050 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 10 client.5931 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55a6862fc760 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 1 -- 172.21.15.43:0/1295095990 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18ac0109c0 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 1 -- 172.21.15.43:0/1295095990 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f18ac026030 con 0x55a686559ee0 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 10 client.5931 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 10 client.5931 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 20 client.5931 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a68655e580 0x55a6865d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 4 client.5931 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628296}] 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628296}) -- 0x55a686362fe0 con 0x55a68655e580 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a68655e580 0x55a6865d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b37fe640 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a68655e580 0x55a6865d4160 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b58f38d8c92f8 server_cookie=3e5b058c9a278505 in_seq=0 out_seq=0 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 10 client.5931 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 1 -- 172.21.15.43:0/1295095990 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3814 (crc 0 0 0) 0x55a686362fe0 con 0x55a68655e580 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 10 client.5931 handle_command_reply: tid=0 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 client.5931 shutdown 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 msgr2=0x55a68655d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a68655d860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f18ac001980 tx=0x7f18ac000bd0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 shutdown_connections 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a68655e580 0x55a6865d4160 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a686559ee0 0x55a68655d860 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18927fc640 0 client.5931 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a6865595c0 0x55a68655d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 --2- 172.21.15.43:0/1295095990 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a686507830 0x55a6865caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 >> 172.21.15.43:0/1295095990 conn(0x55a6865037e0 msgr2=0x55a686570c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 shutdown_connections 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 1 -- 172.21.15.43:0/1295095990 wait complete. 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18bce7d000 20 client.5931 trim_cache size 0 max 16384 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/ioctl.h", 2024-03-30T13:13:48.281 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628296, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.907399+0000", 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.319083+0000", 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.282 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3724, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:48.283 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.319083+0000", 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:48.284 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3724, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.907399+0000" 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3724, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.907399+0000" 2024-03-30T13:13:48.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 118, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 34, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:48.286 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:48.287 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:48.288 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:48.289 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.290 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:48.291 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:48.292 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:48.293 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f18a0067ff0 msgr2=0x7f18a006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f18a0067ff0 0x7f18a006a4b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f18b415bfd0 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b413f140 msgr2=0x7f18b413d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b413f140 0x7f18b413d360 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ae040 tx=0x7f18a801ac00 comp rx=0 tx=0).stop 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 shutdown_connections 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f18a0067ff0 0x7f18a006a4b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f18b415ff40 0x7f18b4155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18b413f140 0x7f18b413d360 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 --2- 172.21.15.43:0/3445156580 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f18b406cbb0 0x7f18b4136750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 >> 172.21.15.43:0/3445156580 conn(0x7f18b40d00a0 msgr2=0x7f18b40af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 shutdown_connections 2024-03-30T13:13:48.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.270+0000 7f18b9cfb640 1 -- 172.21.15.43:0/3445156580 wait complete. 2024-03-30T13:13:48.295 DEBUG:tasks.cephfs.test_uninlining:getting inode info #6 of 23:1099511628308 2024-03-30T13:13:48.295 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628308 2024-03-30T13:13:48.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628308 2024-03-30T13:13:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 Processor -- start 2024-03-30T13:13:48.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 -- start start 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff00006ca60 0x7ff0000cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff00015d0c0 0x7ff00006def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006e430 0x7ff00006b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff00006e880 con 0x7ff00015d0c0 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff0000cc180 con 0x7ff00006ca60 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff0068ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff0000ccdc0 con 0x7ff00006e430 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff00015d0c0 0x7ff00006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff00015d0c0 0x7ff00006def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59416/0 (socket says 172.21.15.43:59416) 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 -- 172.21.15.43:0/336101779 learned_addr learned my addr 172.21.15.43:0/336101779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7ff004e89640 1 --2- 172.21.15.43:0/336101779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006e430 0x7ff00006b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feffffff640 1 --2- 172.21.15.43:0/336101779 >> v2:172.21.15.113:3300/0 conn(0x7ff00006ca60 0x7ff0000cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 -- 172.21.15.43:0/336101779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006e430 msgr2=0x7ff00006b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 --2- 172.21.15.43:0/336101779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006e430 0x7ff00006b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 -- 172.21.15.43:0/336101779 >> v2:172.21.15.113:3300/0 conn(0x7ff00006ca60 msgr2=0x7ff0000cbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 --2- 172.21.15.43:0/336101779 >> v2:172.21.15.113:3300/0 conn(0x7ff00006ca60 0x7ff0000cbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feff77fe640 1 -- 172.21.15.43:0/336101779 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0000cb160 con 0x7ff00015d0c0 2024-03-30T13:13:48.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.418+0000 7feffffff640 1 --2- 172.21.15.43:0/336101779 >> v2:172.21.15.113:3300/0 conn(0x7ff00006ca60 0x7ff0000cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7feff77fe640 1 --2- 172.21.15.43:0/336101779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00015d0c0 0x7ff00006def0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff000072cd0 tx=0x7fefe801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea16e635e26fefdd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7fefff7fe640 1 -- 172.21.15.43:0/336101779 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80b2070 con 0x7ff00015d0c0 2024-03-30T13:13:48.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7fefff7fe640 1 -- 172.21.15.43:0/336101779 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe80ad640 con 0x7ff00015d0c0 2024-03-30T13:13:48.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7fefff7fe640 1 -- 172.21.15.43:0/336101779 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80ada10 con 0x7ff00015d0c0 2024-03-30T13:13:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 -- 172.21.15.43:0/336101779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00015d0c0 msgr2=0x7ff00006def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 --2- 172.21.15.43:0/336101779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00015d0c0 0x7ff00006def0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff000072cd0 tx=0x7fefe801c520 comp rx=0 tx=0).stop 2024-03-30T13:13:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 -- 172.21.15.43:0/336101779 shutdown_connections 2024-03-30T13:13:48.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 --2- 172.21.15.43:0/336101779 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006e430 0x7ff00006b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 --2- 172.21.15.43:0/336101779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00015d0c0 0x7ff00006def0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 --2- 172.21.15.43:0/336101779 >> v2:172.21.15.113:3300/0 conn(0x7ff00006ca60 0x7ff0000cbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 -- 172.21.15.43:0/336101779 >> 172.21.15.43:0/336101779 conn(0x7ff0000cd170 msgr2=0x7ff0000cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 -- 172.21.15.43:0/336101779 shutdown_connections 2024-03-30T13:13:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.422+0000 7ff0068ec640 1 -- 172.21.15.43:0/336101779 wait complete. 2024-03-30T13:13:48.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 Processor -- start 2024-03-30T13:13:48.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 -- start start 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006ca60 0x7ff0000c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00006e430 0x7ff000147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00015d0c0 0x7ff0001595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff0000cca00 con 0x7ff00006e430 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff00015d570 con 0x7ff00015d0c0 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff0000ccb80 con 0x7ff00006ca60 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00006e430 0x7ff000147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00006e430 0x7ff000147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59422/0 (socket says 172.21.15.43:59422) 2024-03-30T13:13:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 -- 172.21.15.43:0/3222102094 learned_addr learned my addr 172.21.15.43:0/3222102094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff004e89640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00015d0c0 0x7ff0001595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feffffff640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006ca60 0x7ff0000c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 -- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006ca60 msgr2=0x7ff0000c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006ca60 0x7ff0000c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 -- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00015d0c0 msgr2=0x7ff0001595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00015d0c0 0x7ff0001595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 -- 172.21.15.43:0/3222102094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00015a4a0 con 0x7ff00006e430 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feff77fe640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00006e430 0x7ff000147040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fefe80b0040 tx=0x7fefe80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feffd7fa640 1 -- 172.21.15.43:0/3222102094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80bd070 con 0x7ff00006e430 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feffd7fa640 1 -- 172.21.15.43:0/3222102094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe80ab550 con 0x7ff00006e430 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0000c8d40 con 0x7ff00006e430 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7feffd7fa640 1 -- 172.21.15.43:0/3222102094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe80b2070 con 0x7ff00006e430 2024-03-30T13:13:48.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.426+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0000c9300 con 0x7ff00006e430 2024-03-30T13:13:48.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feffd7fa640 1 -- 172.21.15.43:0/3222102094 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fefe80c0070 con 0x7ff00006e430 2024-03-30T13:13:48.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feffd7fa640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fefe0067fc0 0x7fefe006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feffd7fa640 1 -- 172.21.15.43:0/3222102094 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fefe81202d0 con 0x7ff00006e430 2024-03-30T13:13:48.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feffffff640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fefe0067fc0 0x7fefe006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 Processor -- start 2024-03-30T13:13:48.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- start start 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feffffff640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fefe0067fc0 0x7fefe006a480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feff000b990 tx=0x7feff0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d47c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc67b95aa0 con 0x55fc67d47830 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fc67b1caf0 con 0x55fc67d91690 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fc67b91a50 con 0x55fc67d90d70 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d47c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d47c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59428/0 (socket says 172.21.15.43:59428) 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 -- 172.21.15.43:0/699318467 learned_addr learned my addr 172.21.15.43:0/699318467 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feffffff640 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feff77fe640 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 -- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 msgr2=0x55fc67d91150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d91150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 -- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 msgr2=0x55fc67d472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 -- 172.21.15.43:0/699318467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc67b1ad00 con 0x55fc67d47830 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feffffff640 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d91150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff004e89640 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d47c10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feff80074c0 tx=0x7feff8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da76db6df8c149f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feff5ffb640 1 -- 172.21.15.43:0/699318467 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8019020 con 0x55fc67d47830 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feff5ffb640 1 -- 172.21.15.43:0/699318467 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff8015600 con 0x55fc67d47830 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7feff5ffb640 1 -- 172.21.15.43:0/699318467 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff80159d0 con 0x55fc67d47830 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- 172.21.15.43:0/699318467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 msgr2=0x55fc67d47c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d47c10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7feff80074c0 tx=0x7feff8009750 comp rx=0 tx=0).stop 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- 172.21.15.43:0/699318467 shutdown_connections 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d47c10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 --2- 172.21.15.43:0/699318467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d91150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- 172.21.15.43:0/699318467 >> 172.21.15.43:0/699318467 conn(0x55fc67d437e0 msgr2=0x55fc67d45bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.430+0000 7ff009a6e000 1 -- 172.21.15.43:0/699318467 shutdown_connections 2024-03-30T13:13:48.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- 172.21.15.43:0/699318467 wait complete. 2024-03-30T13:13:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- start start 2024-03-30T13:13:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 Processor -- start 2024-03-30T13:13:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d4cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc67b5d270 con 0x55fc67d47830 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fc67b536a0 con 0x55fc67d91690 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fc67b2d880 con 0x55fc67d90d70 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 10 client.0 _ll_register_callbacks cb 0x55fc67d32490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d4cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d4cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59438/0 (socket says 172.21.15.43:59438) 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 -- 172.21.15.43:0/135852108 learned_addr learned my addr 172.21.15.43:0/135852108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff004e89640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feff77fe640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 -- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 msgr2=0x55fc67d75da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d75da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 -- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 msgr2=0x55fc67d762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 -- 172.21.15.43:0/135852108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc67b5a730 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff004e89640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feffffff640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d4cc10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feff0010010 tx=0x7feff0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fdb9b66f6370e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7fefdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7fefdeffd640 1 -- 172.21.15.43:0/135852108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0025020 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff8015600 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7fefdeffd640 1 -- 172.21.15.43:0/135852108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7feff0021830 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7fefdeffd640 1 -- 172.21.15.43:0/135852108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feff002f050 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 10 client.5943 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55fc67b4dca0 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7fefdeffd640 1 -- 172.21.15.43:0/135852108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff8015600 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7fefdeffd640 1 -- 172.21.15.43:0/135852108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7feff0032020 con 0x55fc67d47830 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 10 client.5943 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 10 client.5943 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 20 client.5943 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d985d0 0x55fc67d989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 4 client.5943 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc67b30c90 con 0x55fc67d985d0 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feff77fe640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d985d0 0x55fc67d989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7fefdeffd640 10 client.5943 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.434+0000 7feff77fe640 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d985d0 0x55fc67d989e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc9915035a03ac28 server_cookie=aa82dbd9c0569576 in_seq=0 out_seq=0 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7fefdeffd640 1 -- 172.21.15.43:0/135852108 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55fc67b30c90 con 0x55fc67d985d0 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7fefdeffd640 10 client.5943 handle_command_reply: tid=0 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 client.5943 shutdown 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 msgr2=0x55fc67d4cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d4cc10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7feff0010010 tx=0x7feff0004740 comp rx=0 tx=0).stop 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 shutdown_connections 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d985d0 0x55fc67d989e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d91690 0x55fc67d762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67d75da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 --2- 172.21.15.43:0/135852108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d4cc10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7fefdeffd640 0 client.5943 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 >> 172.21.15.43:0/135852108 conn(0x55fc67d437e0 msgr2=0x55fc67d46460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 shutdown_connections 2024-03-30T13:13:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 1 -- 172.21.15.43:0/135852108 wait complete. 2024-03-30T13:13:48.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.438+0000 7ff009a6e000 20 client.5943 trim_cache size 0 max 16384 2024-03-30T13:13:48.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 Processor -- start 2024-03-30T13:13:48.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- start start 2024-03-30T13:13:48.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d45bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d90d70 0x55fc67d46110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d91690 0x55fc67d46650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff000201b60 con 0x55fc67d47830 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff000200b40 con 0x55fc67d90d70 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff000155c90 con 0x55fc67d91690 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d45bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d45bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59450/0 (socket says 172.21.15.43:59450) 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 -- 172.21.15.43:0/173106788 learned_addr learned my addr 172.21.15.43:0/173106788 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff004e89640 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d91690 0x55fc67d46650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feff77fe640 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d90d70 0x55fc67d46110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 -- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d91690 msgr2=0x55fc67d46650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d91690 0x55fc67d46650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 -- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d90d70 msgr2=0x55fc67d46110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d90d70 0x55fc67d46110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 -- 172.21.15.43:0/173106788 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc67b1cd00 con 0x55fc67d47830 2024-03-30T13:13:48.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feff77fe640 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d90d70 0x55fc67d46110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feffffff640 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d45bd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7feff0000c10 tx=0x7feff0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81717dee7e41c2c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feff57fa640 1 -- 172.21.15.43:0/173106788 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0025050 con 0x55fc67d47830 2024-03-30T13:13:48.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feff57fa640 1 -- 172.21.15.43:0/173106788 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff001c040 con 0x55fc67d47830 2024-03-30T13:13:48.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7feff57fa640 1 -- 172.21.15.43:0/173106788 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff001b070 con 0x55fc67d47830 2024-03-30T13:13:48.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- 172.21.15.43:0/173106788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 msgr2=0x55fc67d45bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d45bd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7feff0000c10 tx=0x7feff0020000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- 172.21.15.43:0/173106788 shutdown_connections 2024-03-30T13:13:48.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d91690 0x55fc67d46650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d90d70 0x55fc67d46110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 --2- 172.21.15.43:0/173106788 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d47830 0x55fc67d45bd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- 172.21.15.43:0/173106788 >> 172.21.15.43:0/173106788 conn(0x55fc67d437e0 msgr2=0x55fc67da7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- 172.21.15.43:0/173106788 shutdown_connections 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- 172.21.15.43:0/173106788 wait complete. 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 -- start start 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.454+0000 7ff009a6e000 1 Processor -- start 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d47830 0x55fc67e034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67e03a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d91690 0x55fc67d94ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fc67b536a0 con 0x55fc67d91690 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001562e0 con 0x55fc67d47830 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fc67b5d270 con 0x55fc67d90d70 2024-03-30T13:13:48.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 10 client.0 _ll_register_callbacks cb 0x7feff000b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d91690 0x55fc67d94ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d91690 0x55fc67d94ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59452/0 (socket says 172.21.15.43:59452) 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 -- 172.21.15.43:0/3886251061 learned_addr learned my addr 172.21.15.43:0/3886251061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7feffffff640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d47830 0x55fc67e034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7feff77fe640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67e03a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 -- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 msgr2=0x55fc67e03a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67e03a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 -- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d47830 msgr2=0x55fc67e034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d47830 0x55fc67e034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 -- 172.21.15.43:0/3886251061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc67c16f70 con 0x55fc67d91690 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7feffffff640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d47830 0x55fc67e034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff004e89640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d91690 0x55fc67d94ad0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7feff8017000 tx=0x7feff8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2992f2d036e6e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 1 -- 172.21.15.43:0/3886251061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8019040 con 0x55fc67d91690 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff001c040 con 0x55fc67d91690 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 1 -- 172.21.15.43:0/3886251061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7feff8029430 con 0x55fc67d91690 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 1 -- 172.21.15.43:0/3886251061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feff8033050 con 0x55fc67d91690 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 10 client.5949 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55fc67b3c760 con 0x55fc67d91690 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 1 -- 172.21.15.43:0/3886251061 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff001c040 con 0x55fc67d91690 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 1 -- 172.21.15.43:0/3886251061 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7feff8036020 con 0x55fc67d91690 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 10 client.5949 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 10 client.5949 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 20 client.5949 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d95850 0x55fc67d95c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 4 client.5949 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628308}] 2024-03-30T13:13:48.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628308}) -- 0x55fc67ba2fe0 con 0x55fc67d95850 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7feffffff640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d95850 0x55fc67d95c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7feffffff640 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d95850 0x55fc67d95c60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239f37a545802c88 server_cookie=b0d282fa55e2ce7a in_seq=0 out_seq=0 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 10 client.5949 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 1 -- 172.21.15.43:0/3886251061 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3812 (crc 0 0 0) 0x55fc67ba2fe0 con 0x55fc67d95850 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 10 client.5949 handle_command_reply: tid=0 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 client.5949 shutdown 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d91690 msgr2=0x55fc67d94ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d91690 0x55fc67d94ad0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7feff8017000 tx=0x7feff8014000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 shutdown_connections 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fc67d95850 0x55fc67d95c60 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fc67d91690 0x55fc67d94ad0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fc67d90d70 0x55fc67e03a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 --2- 172.21.15.43:0/3886251061 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fc67d47830 0x55fc67e034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7fefdeffd640 0 client.5949 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 >> 172.21.15.43:0/3886251061 conn(0x55fc67d437e0 msgr2=0x55fc67da8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 shutdown_connections 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 1 -- 172.21.15.43:0/3886251061 wait complete. 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.458+0000 7ff009a6e000 20 client.5949 trim_cache size 0 max 16384 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/Kconfig", 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628308, 2024-03-30T13:13:48.470 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.918402+0000", 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.438086+0000", 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.471 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1240, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:48.472 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.438086+0000", 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:48.473 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1240, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.918402+0000" 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1240, 2024-03-30T13:13:48.474 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.918402+0000" 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 68, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 2, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:48.475 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:48.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:48.477 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.478 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.479 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.480 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:48.481 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:48.482 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:48.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fefe0067fc0 msgr2=0x7fefe006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fefe0067fc0 0x7fefe006a480 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feff000b990 tx=0x7feff0008040 comp rx=0 tx=0).stop 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00006e430 msgr2=0x7ff000147040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00006e430 0x7ff000147040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fefe80b0040 tx=0x7fefe80192c0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 shutdown_connections 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fefe0067fc0 0x7fefe006a480 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff00015d0c0 0x7ff0001595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff00006e430 0x7ff000147040 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 --2- 172.21.15.43:0/3222102094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff00006ca60 0x7ff0000c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 >> 172.21.15.43:0/3222102094 conn(0x7ff0000cd170 msgr2=0x7ff0000b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 shutdown_connections 2024-03-30T13:13:48.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.462+0000 7ff0068ec640 1 -- 172.21.15.43:0/3222102094 wait complete. 2024-03-30T13:13:48.484 DEBUG:tasks.cephfs.test_uninlining:getting inode info #7 of 23:1099511628326 2024-03-30T13:13:48.485 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628326 2024-03-30T13:13:48.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628326 2024-03-30T13:13:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 Processor -- start 2024-03-30T13:13:48.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 -- start start 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce44147e00 0x7fce440b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fce440b4f30 0x7fce4406c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce4406c5b0 0x7fce4406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fce440b5470 con 0x7fce4406c5b0 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fce440b5750 con 0x7fce440b4f30 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce48fd6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fce440b5a30 con 0x7fce44147e00 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce4406c5b0 0x7fce4406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce4406c5b0 0x7fce4406f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59466/0 (socket says 172.21.15.43:59466) 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 -- 172.21.15.43:0/1283855294 learned_addr learned my addr 172.21.15.43:0/1283855294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce41d9b640 1 --2- 172.21.15.43:0/1283855294 >> v2:172.21.15.113:3300/0 conn(0x7fce440b4f30 0x7fce4406c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce4259c640 1 --2- 172.21.15.43:0/1283855294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce44147e00 0x7fce440b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 -- 172.21.15.43:0/1283855294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce44147e00 msgr2=0x7fce440b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1283855294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce44147e00 0x7fce440b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 -- 172.21.15.43:0/1283855294 >> v2:172.21.15.113:3300/0 conn(0x7fce440b4f30 msgr2=0x7fce4406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1283855294 >> v2:172.21.15.113:3300/0 conn(0x7fce440b4f30 0x7fce4406c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.602+0000 7fce42d9d640 1 -- 172.21.15.43:0/1283855294 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce440b5d10 con 0x7fce4406c5b0 2024-03-30T13:13:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1283855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce4406f9e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009e50 tx=0x7fce3c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a42912827c24f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4159a640 1 -- 172.21.15.43:0/1283855294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0ac070 con 0x7fce4406c5b0 2024-03-30T13:13:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4159a640 1 -- 172.21.15.43:0/1283855294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3c0a8420 con 0x7fce4406c5b0 2024-03-30T13:13:48.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4159a640 1 -- 172.21.15.43:0/1283855294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0a87f0 con 0x7fce4406c5b0 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- 172.21.15.43:0/1283855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 msgr2=0x7fce4406f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 --2- 172.21.15.43:0/1283855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce4406f9e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fce3c009e50 tx=0x7fce3c01c730 comp rx=0 tx=0).stop 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- 172.21.15.43:0/1283855294 shutdown_connections 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 --2- 172.21.15.43:0/1283855294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce4406f9e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 --2- 172.21.15.43:0/1283855294 >> v2:172.21.15.113:3300/0 conn(0x7fce440b4f30 0x7fce4406c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 --2- 172.21.15.43:0/1283855294 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce44147e00 0x7fce440b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- 172.21.15.43:0/1283855294 >> 172.21.15.43:0/1283855294 conn(0x7fce440d1070 msgr2=0x7fce440d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- 172.21.15.43:0/1283855294 shutdown_connections 2024-03-30T13:13:48.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- 172.21.15.43:0/1283855294 wait complete. 2024-03-30T13:13:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 Processor -- start 2024-03-30T13:13:48.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- start start 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce440ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce440b4f30 0x7fce44142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce44147e00 0x7fce44141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce440ae740 con 0x7fce4406c5b0 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce440ae5c0 con 0x7fce44147e00 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce48fd6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fce440ae8c0 con 0x7fce440b4f30 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce440ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce440ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59482/0 (socket says 172.21.15.43:59482) 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 -- 172.21.15.43:0/685184270 learned_addr learned my addr 172.21.15.43:0/685184270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce42d9d640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce44147e00 0x7fce44141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce41d9b640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce440b4f30 0x7fce44142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 -- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce440b4f30 msgr2=0x7fce44142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce440b4f30 0x7fce44142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 -- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce44147e00 msgr2=0x7fce44141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce44147e00 0x7fce44141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce4259c640 1 -- 172.21.15.43:0/685184270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce4415f5c0 con 0x7fce4406c5b0 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce42d9d640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce44147e00 0x7fce44141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.606+0000 7fce41d9b640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce440b4f30 0x7fce44142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4259c640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce440ce890 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fce3400d5f0 tx=0x7fce3400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce32ffd640 1 -- 172.21.15.43:0/685184270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340125f0 con 0x7fce4406c5b0 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce32ffd640 1 -- 172.21.15.43:0/685184270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce34019070 con 0x7fce4406c5b0 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce4415f8a0 con 0x7fce4406c5b0 2024-03-30T13:13:48.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce32ffd640 1 -- 172.21.15.43:0/685184270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce340129b0 con 0x7fce4406c5b0 2024-03-30T13:13:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce4415fe60 con 0x7fce4406c5b0 2024-03-30T13:13:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 Processor -- start 2024-03-30T13:13:48.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 -- start start 2024-03-30T13:13:48.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce32ffd640 1 -- 172.21.15.43:0/685184270 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fce34012b50 con 0x7fce4406c5b0 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce32ffd640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce24067fc0 0x7fce2406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce32ffd640 1 -- 172.21.15.43:0/685184270 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fce34086a10 con 0x7fce4406c5b0 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa17b7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b7690 0x564aa176d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa176dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564aa15bbaa0 con 0x564aa17b6d70 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564aa1542af0 con 0x564aa176d830 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x564aa15b7a50 con 0x564aa17b7690 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa17b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa17b7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59486/0 (socket says 172.21.15.43:59486) 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.610+0000 7fce4259c640 1 -- 172.21.15.43:0/1819353866 learned_addr learned my addr 172.21.15.43:0/1819353866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce41d9b640 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b7690 0x564aa176d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce24067fc0 0x7fce2406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa176dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4259c640 1 -- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b7690 msgr2=0x564aa176d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4259c640 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b7690 0x564aa176d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4259c640 1 -- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 msgr2=0x564aa176dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4259c640 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa176dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4259c640 1 -- 172.21.15.43:0/1819353866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa1540d00 con 0x564aa17b6d70 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa176dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4259c640 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa17b7150 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fce3405f040 tx=0x7fce34011480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c862b2706dda840e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce23fff640 1 -- 172.21.15.43:0/1819353866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34083070 con 0x564aa17b6d70 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce23fff640 1 -- 172.21.15.43:0/1819353866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3405ad40 con 0x564aa17b6d70 2024-03-30T13:13:48.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce23fff640 1 -- 172.21.15.43:0/1819353866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3405f570 con 0x564aa17b6d70 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- 172.21.15.43:0/1819353866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 msgr2=0x564aa17b7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce24067fc0 0x7fce2406a480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0ae000 tx=0x7fce3c004ac0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa17b7150 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fce3405f040 tx=0x7fce34011480 comp rx=0 tx=0).stop 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- 172.21.15.43:0/1819353866 shutdown_connections 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa176dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b7690 0x564aa176d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 --2- 172.21.15.43:0/1819353866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa17b7150 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- 172.21.15.43:0/1819353866 >> 172.21.15.43:0/1819353866 conn(0x564aa17697e0 msgr2=0x564aa176bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- 172.21.15.43:0/1819353866 shutdown_connections 2024-03-30T13:13:48.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- 172.21.15.43:0/1819353866 wait complete. 2024-03-30T13:13:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- start start 2024-03-30T13:13:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 Processor -- start 2024-03-30T13:13:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa1772c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b6d70 0x564aa179bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa179c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564aa1583270 con 0x564aa17b7690 2024-03-30T13:13:48.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564aa15796a0 con 0x564aa176d830 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x564aa1553880 con 0x564aa17b6d70 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 10 client.0 _ll_register_callbacks cb 0x564aa1758490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa179c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa179c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59494/0 (socket says 172.21.15.43:59494) 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 -- 172.21.15.43:0/1862655339 learned_addr learned my addr 172.21.15.43:0/1862655339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4259c640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa1772c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce41d9b640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b6d70 0x564aa179bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 -- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b6d70 msgr2=0x564aa179bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b6d70 0x564aa179bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 -- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 msgr2=0x564aa1772c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa1772c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 -- 172.21.15.43:0/1862655339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa1580730 con 0x564aa17b7690 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce41d9b640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b6d70 0x564aa179bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce42d9d640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa179c2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01c770 tx=0x7fce3c01af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f979b47764a444a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce217fa640 1 -- 172.21.15.43:0/1862655339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0ac040 con 0x564aa17b7690 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce217fa640 1 -- 172.21.15.43:0/1862655339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fce3c0c9cb0 con 0x564aa17b7690 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce3405ad40 con 0x564aa17b7690 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce217fa640 1 -- 172.21.15.43:0/1862655339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce3c0d0050 con 0x564aa17b7690 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 10 client.5961 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.614+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x564aa1573ca0 con 0x564aa17b7690 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce217fa640 1 -- 172.21.15.43:0/1862655339 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce3405ad40 con 0x564aa17b7690 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce217fa640 1 -- 172.21.15.43:0/1862655339 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fce3c0d3020 con 0x564aa17b7690 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 10 client.5961 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 10 client.5961 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 20 client.5961 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17be5d0 0x564aa17be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 4 client.5961 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:48.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aa1556c90 con 0x564aa17be5d0 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4259c640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17be5d0 0x564aa17be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce217fa640 10 client.5961 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4259c640 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17be5d0 0x564aa17be9b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db4ccf4fb25f420 server_cookie=ebfd25b1417a3c5d in_seq=0 out_seq=0 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce217fa640 1 -- 172.21.15.43:0/1862655339 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x564aa1556c90 con 0x564aa17be5d0 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce217fa640 10 client.5961 handle_command_reply: tid=0 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 client.5961 shutdown 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 msgr2=0x564aa179c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa179c2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01c770 tx=0x7fce3c01af70 comp rx=0 tx=0).stop 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 shutdown_connections 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17be5d0 0x564aa17be9b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa179c2e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa17b6d70 0x564aa179bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce217fa640 0 client.5961 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 --2- 172.21.15.43:0/1862655339 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa176d830 0x564aa1772c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 >> 172.21.15.43:0/1862655339 conn(0x564aa17697e0 msgr2=0x564aa176c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 shutdown_connections 2024-03-30T13:13:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 1 -- 172.21.15.43:0/1862655339 wait complete. 2024-03-30T13:13:48.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.618+0000 7fce4c158000 20 client.5961 trim_cache size 0 max 16384 2024-03-30T13:13:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 Processor -- start 2024-03-30T13:13:48.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 -- start start 2024-03-30T13:13:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa179c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa176bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b7690 0x564aa176c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fce44204610 con 0x564aa17b6d70 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce442035f0 con 0x564aa17b7690 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fce441441b0 con 0x564aa176d830 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa176bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce42d9d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b7690 0x564aa176c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4259c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa179c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa176bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59498/0 (socket says 172.21.15.43:59498) 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 -- 172.21.15.43:0/112062568 learned_addr learned my addr 172.21.15.43:0/112062568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 -- 172.21.15.43:0/112062568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 msgr2=0x564aa179c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 --2- 172.21.15.43:0/112062568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa179c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 -- 172.21.15.43:0/112062568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b7690 msgr2=0x564aa176c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 --2- 172.21.15.43:0/112062568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b7690 0x564aa176c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 -- 172.21.15.43:0/112062568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa1542d00 con 0x564aa17b6d70 2024-03-30T13:13:48.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce41d9b640 1 --2- 172.21.15.43:0/112062568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa176bbd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fce380108d0 tx=0x7fce38010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1babd0a8edb707 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce237fe640 1 -- 172.21.15.43:0/112062568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce38019030 con 0x564aa17b6d70 2024-03-30T13:13:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce237fe640 1 -- 172.21.15.43:0/112062568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce38012a60 con 0x564aa17b6d70 2024-03-30T13:13:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce237fe640 1 -- 172.21.15.43:0/112062568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3800f040 con 0x564aa17b6d70 2024-03-30T13:13:48.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 -- 172.21.15.43:0/112062568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 msgr2=0x564aa176bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.634+0000 7fce4c158000 1 --2- 172.21.15.43:0/112062568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa176bbd0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fce380108d0 tx=0x7fce38010d90 comp rx=0 tx=0).stop 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- 172.21.15.43:0/112062568 shutdown_connections 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 --2- 172.21.15.43:0/112062568 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b7690 0x564aa176c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 --2- 172.21.15.43:0/112062568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b6d70 0x564aa176bbd0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 --2- 172.21.15.43:0/112062568 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa179c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- 172.21.15.43:0/112062568 >> 172.21.15.43:0/112062568 conn(0x564aa17697e0 msgr2=0x564aa17cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- 172.21.15.43:0/112062568 shutdown_connections 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- 172.21.15.43:0/112062568 wait complete. 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- start start 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 Processor -- start 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa1827ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b6d70 0x564aa17baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa17bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564aa15badb0 con 0x564aa17b7690 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fce44144800 con 0x564aa17b6d70 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x564aa1553880 con 0x564aa176d830 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 10 client.0 _ll_register_callbacks cb 0x7fce3c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa17bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa17bb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59508/0 (socket says 172.21.15.43:59508) 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 -- 172.21.15.43:0/2145754754 learned_addr learned my addr 172.21.15.43:0/2145754754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce41d9b640 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b6d70 0x564aa17baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4259c640 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa1827ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 -- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 msgr2=0x564aa1827ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa1827ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 -- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b6d70 msgr2=0x564aa17baad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b6d70 0x564aa17baad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 -- 172.21.15.43:0/2145754754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa163cf70 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce42d9d640 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa17bb010 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01cca0 tx=0x7fce3c0b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7982b38981a90721 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce217fa640 1 -- 172.21.15.43:0/2145754754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3c0d9060 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce217fa640 1 -- 172.21.15.43:0/2145754754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fce3c0c9c20 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce38012a60 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce217fa640 1 -- 172.21.15.43:0/2145754754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fce3c0df050 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 10 client.5967 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x564aa1562760 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce217fa640 1 -- 172.21.15.43:0/2145754754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce38012a60 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce217fa640 1 -- 172.21.15.43:0/2145754754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fce3c0e2020 con 0x564aa17b7690 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 10 client.5967 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 10 client.5967 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 20 client.5967 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17bb550 0x564aa17bb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 4 client.5967 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628326}] 2024-03-30T13:13:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628326}) -- 0x564aa15c8fe0 con 0x564aa17bb550 2024-03-30T13:13:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.638+0000 7fce4259c640 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17bb550 0x564aa17bb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4259c640 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17bb550 0x564aa17bb930 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4b91481933ed63 server_cookie=76b194912e5d288e in_seq=0 out_seq=0 2024-03-30T13:13:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce217fa640 10 client.5967 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce217fa640 1 -- 172.21.15.43:0/2145754754 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3818 (crc 0 0 0) 0x564aa15c8fe0 con 0x564aa17bb550 2024-03-30T13:13:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce217fa640 10 client.5967 handle_command_reply: tid=0 2024-03-30T13:13:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 client.5967 shutdown 2024-03-30T13:13:48.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 msgr2=0x564aa17bb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa17bb010 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fce3c01cca0 tx=0x7fce3c0b7000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 shutdown_connections 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564aa17bb550 0x564aa17bb930 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564aa17b7690 0x564aa17bb010 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564aa17b6d70 0x564aa17baad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce217fa640 0 client.5967 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 --2- 172.21.15.43:0/2145754754 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564aa176d830 0x564aa1827ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 >> 172.21.15.43:0/2145754754 conn(0x564aa17697e0 msgr2=0x564aa17ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 shutdown_connections 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 1 -- 172.21.15.43:0/2145754754 wait complete. 2024-03-30T13:13:48.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce4c158000 20 client.5967 trim_cache size 0 max 16384 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr_user.c", 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628326, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.922430+0000", 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.565906+0000", 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1232, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:48.652 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.565906+0000", 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:48.653 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1232, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.922430+0000" 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.654 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1232, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.922430+0000" 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 72, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 38, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:48.655 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:48.656 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:48.657 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.658 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:48.659 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:48.660 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:48.661 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:48.662 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:48.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce24067fc0 msgr2=0x7fce2406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce24067fc0 0x7fce2406a480 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0ae000 tx=0x7fce3c004ac0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 msgr2=0x7fce440ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce440ce890 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fce3400d5f0 tx=0x7fce3400dab0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 shutdown_connections 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fce24067fc0 0x7fce2406a480 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fce44147e00 0x7fce44141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fce440b4f30 0x7fce44142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 --2- 172.21.15.43:0/685184270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce4406c5b0 0x7fce440ce890 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 >> 172.21.15.43:0/685184270 conn(0x7fce440d1070 msgr2=0x7fce44146b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 shutdown_connections 2024-03-30T13:13:48.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.642+0000 7fce48fd6640 1 -- 172.21.15.43:0/685184270 wait complete. 2024-03-30T13:13:48.664 DEBUG:tasks.cephfs.test_uninlining:getting inode info #8 of 23:1099511628324 2024-03-30T13:13:48.665 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628324 2024-03-30T13:13:48.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628324 2024-03-30T13:13:48.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.778+0000 7fb6b867f640 1 Processor -- start 2024-03-30T13:13:48.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.778+0000 7fb6b867f640 1 -- start start 2024-03-30T13:13:48.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b867f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6b0160610 0x7fb6b00cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b867f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb6b00d02d0 0x7fb6b006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b867f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6b00d0b50 con 0x7fb6b01450f0 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b867f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b00d0e30 con 0x7fb6b0160610 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b867f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb6b006b700 con 0x7fb6b00d02d0 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6b0160610 0x7fb6b00cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6b0160610 0x7fb6b00cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59524/0 (socket says 172.21.15.43:59524) 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/1336769798 learned_addr learned my addr 172.21.15.43:0/1336769798 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b641b640 1 --1- 172.21.15.43:0/1336769798 >> v1:172.21.15.43:6789/0 conn(0x7fb6b01450f0 0x7fb6b015ff00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47212/0 (socket says 172.21.15.43:47212) 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/1336769798 >> v2:172.21.15.113:3300/0 conn(0x7fb6b00d02d0 0x7fb6b006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/1336769798 >> v2:172.21.15.113:3300/0 conn(0x7fb6b00d02d0 msgr2=0x7fb6b006c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/1336769798 >> v2:172.21.15.113:3300/0 conn(0x7fb6b00d02d0 0x7fb6b006c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/1336769798 >> v1:172.21.15.43:6789/0 conn(0x7fb6b01450f0 legacy=0x7fb6b015ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5419640 1 -- 172.21.15.43:0/1336769798 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091903513 0 0) 0x7fb6b00d0b50 con 0x7fb6b01450f0 2024-03-30T13:13:48.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/1336769798 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b006b9e0 con 0x7fb6b0160610 2024-03-30T13:13:48.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/1336769798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b0160610 0x7fb6b00cfd90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00ce640 tx=0x7fb6a001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38b46692b4cdab5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5419640 1 -- 172.21.15.43:0/1336769798 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00aa670 con 0x7fb6b0160610 2024-03-30T13:13:48.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5419640 1 -- 172.21.15.43:0/1336769798 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a00af070 con 0x7fb6b0160610 2024-03-30T13:13:48.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.782+0000 7fb6b5419640 1 -- 172.21.15.43:0/1336769798 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00aaa30 con 0x7fb6b0160610 2024-03-30T13:13:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- 172.21.15.43:0/1336769798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b0160610 msgr2=0x7fb6b00cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 --2- 172.21.15.43:0/1336769798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b0160610 0x7fb6b00cfd90 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00ce640 tx=0x7fb6a001c4f0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- 172.21.15.43:0/1336769798 shutdown_connections 2024-03-30T13:13:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 --2- 172.21.15.43:0/1336769798 >> v2:172.21.15.113:3300/0 conn(0x7fb6b00d02d0 0x7fb6b006c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 --2- 172.21.15.43:0/1336769798 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b0160610 0x7fb6b00cfd90 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- 172.21.15.43:0/1336769798 >> 172.21.15.43:0/1336769798 conn(0x7fb6b00d1070 msgr2=0x7fb6b00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- 172.21.15.43:0/1336769798 shutdown_connections 2024-03-30T13:13:48.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- 172.21.15.43:0/1336769798 wait complete. 2024-03-30T13:13:48.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 Processor -- start 2024-03-30T13:13:48.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- start start 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6b00d02d0 0x7fb6b015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b01450f0 0x7fb6b013dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6b0160610 0x7fb6b00b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b006cf30 con 0x7fb6b01450f0 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b00d0930 con 0x7fb6b00d02d0 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b006d0b0 con 0x7fb6b0160610 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b01450f0 0x7fb6b013dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b01450f0 0x7fb6b013dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59538/0 (socket says 172.21.15.43:59538) 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/3546705648 learned_addr learned my addr 172.21.15.43:0/3546705648 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b641b640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6b00d02d0 0x7fb6b015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6b0160610 0x7fb6b00b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6b0160610 msgr2=0x7fb6b00b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6b0160610 0x7fb6b00b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6b00d02d0 msgr2=0x7fb6b015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6b00d02d0 0x7fb6b015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/3546705648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b015c290 con 0x7fb6b01450f0 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6b0160610 0x7fb6b00b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b01450f0 0x7fb6b013dfc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00ac040 tx=0x7fb6a0019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6a6ffd640 1 -- 172.21.15.43:0/3546705648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00be610 con 0x7fb6b01450f0 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6a6ffd640 1 -- 172.21.15.43:0/3546705648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b01456e0 con 0x7fb6b01450f0 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6a6ffd640 1 -- 172.21.15.43:0/3546705648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00bea10 con 0x7fb6b01450f0 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b013e9a0 con 0x7fb6b01450f0 2024-03-30T13:13:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.786+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b013ef60 con 0x7fb6b01450f0 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6a6ffd640 1 -- 172.21.15.43:0/3546705648 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb6a00bc020 con 0x7fb6b01450f0 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6a6ffd640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb694067ff0 0x7fb69406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6a6ffd640 1 -- 172.21.15.43:0/3546705648 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fb6a01207e0 con 0x7fb6b01450f0 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b641b640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb694067ff0 0x7fb69406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 Processor -- start 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 -- start start 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b641b640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb694067ff0 0x7fb69406a4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac000f70 tx=0x7fb6ac007b80 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bbc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbc690 0x55be27b722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be279c0aa0 con 0x55be27bbc690 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55be27947af0 con 0x55be27bbbd70 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55be279bca50 con 0x55be27b72830 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbc690 0x55be27b722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbc690 0x55be27b722f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59550/0 (socket says 172.21.15.43:59550) 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/4127973139 learned_addr learned my addr 172.21.15.43:0/4127973139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b641b640 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bbc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 msgr2=0x55be27b72c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b72c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 msgr2=0x55be27bbc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bbc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/4127973139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be27945d00 con 0x55be27bbc690 2024-03-30T13:13:48.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbc690 0x55be27b722f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00aafd0 tx=0x7fb6a00f0020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cf2187ba0a5756 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb693fff640 1 -- 172.21.15.43:0/4127973139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00f4a70 con 0x55be27bbc690 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb693fff640 1 -- 172.21.15.43:0/4127973139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a00af050 con 0x55be27bbc690 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.790+0000 7fb693fff640 1 -- 172.21.15.43:0/4127973139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a00f92a0 con 0x55be27bbc690 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/4127973139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbc690 msgr2=0x55be27b722f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbc690 0x55be27b722f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00aafd0 tx=0x7fb6a00f0020 comp rx=0 tx=0).stop 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/4127973139 shutdown_connections 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b72c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbc690 0x55be27b722f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- 172.21.15.43:0/4127973139 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bbc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/4127973139 >> 172.21.15.43:0/4127973139 conn(0x55be27b6e7e0 msgr2=0x55be27b70bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/4127973139 shutdown_connections 2024-03-30T13:13:48.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/4127973139 wait complete. 2024-03-30T13:13:48.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- start start 2024-03-30T13:13:48.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 Processor -- start 2024-03-30T13:13:48.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27b72830 0x55be27b77c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27bbbd70 0x55be27ba0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27ba16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be27988270 con 0x55be27b72830 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55be2797e6a0 con 0x55be27ba12e0 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55be27958880 con 0x55be27bbbd70 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 10 client.0 _ll_register_callbacks cb 0x55be27b5d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27b72830 0x55be27b77c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27b72830 0x55be27b77c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59564/0 (socket says 172.21.15.43:59564) 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 -- 172.21.15.43:0/1420979416 learned_addr learned my addr 172.21.15.43:0/1420979416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27bbbd70 0x55be27ba0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27ba16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 -- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27bbbd70 msgr2=0x55be27ba0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27bbbd70 0x55be27ba0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 -- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 msgr2=0x55be27ba16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27ba16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 -- 172.21.15.43:0/1420979416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be27985730 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27ba16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b641b640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27b72830 0x55be27b77c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac001c70 tx=0x7fb6ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9be5492165f641 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6917fa640 1 -- 172.21.15.43:0/1420979416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac027060 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6917fa640 1 -- 172.21.15.43:0/1420979416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fb6ac010040 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a00af050 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6917fa640 1 -- 172.21.15.43:0/1420979416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb6ac015110 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 10 client.5982 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55be27978ca0 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6917fa640 1 -- 172.21.15.43:0/1420979416 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a00af050 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6917fa640 1 -- 172.21.15.43:0/1420979416 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fb6ac013420 con 0x55be27b72830 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 10 client.5982 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 10 client.5982 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 20 client.5982 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc3930 0x55be27bce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 4 client.5982 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be2795bc90 con 0x55be27bc3930 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.794+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc3930 0x55be27bce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc3930 0x55be27bce660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1664775d621fb226 server_cookie=76f3b039159fddbc in_seq=0 out_seq=0 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6917fa640 10 client.5982 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6917fa640 1 -- 172.21.15.43:0/1420979416 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55be2795bc90 con 0x55be27bc3930 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6917fa640 10 client.5982 handle_command_reply: tid=0 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 client.5982 shutdown 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27b72830 msgr2=0x55be27b77c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27b72830 0x55be27b77c60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac001c70 tx=0x7fb6ac011000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 shutdown_connections 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc3930 0x55be27bce660 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27ba16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27bbbd70 0x55be27ba0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 --2- 172.21.15.43:0/1420979416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27b72830 0x55be27b77c60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 >> 172.21.15.43:0/1420979416 conn(0x55be27b6e7e0 msgr2=0x55be27b71460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6917fa640 0 client.5982 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 shutdown_connections 2024-03-30T13:13:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 1 -- 172.21.15.43:0/1420979416 wait complete. 2024-03-30T13:13:48.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.798+0000 7fb6bb801000 20 client.5982 trim_cache size 0 max 16384 2024-03-30T13:13:48.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 Processor -- start 2024-03-30T13:13:48.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- start start 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b71be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27ba12e0 0x55be27b72120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bd38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0203ca0 con 0x55be27ba12e0 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0202ce0 con 0x55be27bbbd70 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b015f9b0 con 0x55be27b72830 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27ba12e0 0x55be27b72120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27ba12e0 0x55be27b72120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59576/0 (socket says 172.21.15.43:59576) 2024-03-30T13:13:48.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/2035770507 learned_addr learned my addr 172.21.15.43:0/2035770507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b641b640 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b71be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bd38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 msgr2=0x55be27b71be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b71be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 msgr2=0x55be27bd38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bd38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 -- 172.21.15.43:0/2035770507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be27947d00 con 0x55be27ba12e0 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bd38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27ba12e0 0x55be27b72120 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00aafd0 tx=0x7fb6a00b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653e7a853cc3298a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6937fe640 1 -- 172.21.15.43:0/2035770507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a011c030 con 0x55be27ba12e0 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6937fe640 1 -- 172.21.15.43:0/2035770507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a00af040 con 0x55be27ba12e0 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6937fe640 1 -- 172.21.15.43:0/2035770507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0115e60 con 0x55be27ba12e0 2024-03-30T13:13:48.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- 172.21.15.43:0/2035770507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27ba12e0 msgr2=0x55be27b72120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27ba12e0 0x55be27b72120 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00aafd0 tx=0x7fb6a00b5000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- 172.21.15.43:0/2035770507 shutdown_connections 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27bbbd70 0x55be27bd38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27ba12e0 0x55be27b72120 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- 172.21.15.43:0/2035770507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27b71be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- 172.21.15.43:0/2035770507 >> 172.21.15.43:0/2035770507 conn(0x55be27b6e7e0 msgr2=0x55be27b9fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- 172.21.15.43:0/2035770507 shutdown_connections 2024-03-30T13:13:48.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- 172.21.15.43:0/2035770507 wait complete. 2024-03-30T13:13:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- start start 2024-03-30T13:13:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 Processor -- start 2024-03-30T13:13:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27c2df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27c2e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbbd70 0x55be27bbfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55be27958880 con 0x55be27bbbd70 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0202690 con 0x55be27ba12e0 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55be27a7cb00 con 0x55be27b72830 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.814+0000 7fb6bb801000 10 client.0 _ll_register_callbacks cb 0x7fb6ac007d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbbd70 0x55be27bbfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbbd70 0x55be27bbfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59580/0 (socket says 172.21.15.43:59580) 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 -- 172.21.15.43:0/597873701 learned_addr learned my addr 172.21.15.43:0/597873701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b641b640 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27c2df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b5c1a640 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27c2e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 -- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 msgr2=0x55be27c2df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27c2df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 -- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 msgr2=0x55be27c2e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27c2e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 -- 172.21.15.43:0/597873701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be27a41f70 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b6c1c640 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbbd70 0x55be27bbfad0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80079c0 tx=0x7fb6a8007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcf20ddafb389a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 1 -- 172.21.15.43:0/597873701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a801b070 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 1 -- 172.21.15.43:0/597873701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fb6a8013b60 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a00af040 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 1 -- 172.21.15.43:0/597873701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb6a8026050 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 10 client.5988 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55be27967760 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 1 -- 172.21.15.43:0/597873701 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a00af040 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 1 -- 172.21.15.43:0/597873701 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fb6a8029020 con 0x55be27bbbd70 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 10 client.5988 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 10 client.5988 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 20 client.5988 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc0810 0x55be27bc0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 4 client.5988 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628324}] 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628324}) -- 0x55be279cdfe0 con 0x55be27bc0810 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b641b640 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc0810 0x55be27bc0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b641b640 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc0810 0x55be27bc0c20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=775d27472c08b3da server_cookie=336251a2aa32787a in_seq=0 out_seq=0 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 10 client.5988 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 1 -- 172.21.15.43:0/597873701 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3822 (crc 0 0 0) 0x55be279cdfe0 con 0x55be27bc0810 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 10 client.5988 handle_command_reply: tid=0 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 client.5988 shutdown 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbbd70 msgr2=0x55be27bbfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbbd70 0x55be27bbfad0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80079c0 tx=0x7fb6a8007e80 comp rx=0 tx=0).stop 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 shutdown_connections 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55be27bc0810 0x55be27bc0c20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55be27bbbd70 0x55be27bbfad0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55be27ba12e0 0x55be27c2e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 --2- 172.21.15.43:0/597873701 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55be27b72830 0x55be27c2df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6917fa640 0 client.5988 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 >> 172.21.15.43:0/597873701 conn(0x55be27b6e7e0 msgr2=0x55be27ba05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 shutdown_connections 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 1 -- 172.21.15.43:0/597873701 wait complete. 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6bb801000 20 client.5988 trim_cache size 0 max 16384 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr_security.c", 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628324, 2024-03-30T13:13:48.829 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.928531+0000", 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.556866+0000", 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:48.830 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1473, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.556866+0000", 2024-03-30T13:13:48.831 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.832 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1473, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.928531+0000" 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1473, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.928531+0000" 2024-03-30T13:13:48.833 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 78, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 34, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:48.834 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:48.835 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:48.836 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.837 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.838 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:48.839 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:48.840 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:48.841 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:48.841 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb694067ff0 msgr2=0x7fb69406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.818+0000 7fb6b867f640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb694067ff0 0x7fb69406a4b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac000f70 tx=0x7fb6ac007b80 comp rx=0 tx=0).stop 2024-03-30T13:13:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b01450f0 msgr2=0x7fb6b013dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b01450f0 0x7fb6b013dfc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb6a00ac040 tx=0x7fb6a0019290 comp rx=0 tx=0).stop 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 shutdown_connections 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb694067ff0 0x7fb69406a4b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb6b0160610 0x7fb6b00b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6b01450f0 0x7fb6b013dfc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 --2- 172.21.15.43:0/3546705648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb6b00d02d0 0x7fb6b015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 >> 172.21.15.43:0/3546705648 conn(0x7fb6b00d1070 msgr2=0x7fb6b0067370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 shutdown_connections 2024-03-30T13:13:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.822+0000 7fb6b867f640 1 -- 172.21.15.43:0/3546705648 wait complete. 2024-03-30T13:13:48.843 DEBUG:tasks.cephfs.test_uninlining:getting inode info #9 of 23:1099511628309 2024-03-30T13:13:48.844 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628309 2024-03-30T13:13:48.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628309 2024-03-30T13:13:48.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7603e05640 1 Processor -- start 2024-03-30T13:13:48.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7603e05640 1 -- start start 2024-03-30T13:13:48.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7603e05640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f75fc0cc750 0x7f75fc0ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7603e05640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f75fc0b6c50 0x7f75fc0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7603e05640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75fc158f20 con 0x7f75fc158b40 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7603e05640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc162d00 con 0x7f75fc0cc750 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7603e05640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f75fc1620f0 con 0x7f75fc0b6c50 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f76013a0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f75fc0cc750 0x7f75fc0ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7601ba1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f75fc158b40 0x7f75fc0b4ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47224/0 (socket says 172.21.15.43:47224) 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f76013a0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f75fc0cc750 0x7f75fc0ccc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59592/0 (socket says 172.21.15.43:59592) 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f7601ba1640 1 -- 172.21.15.43:0/4043433048 learned_addr learned my addr 172.21.15.43:0/4043433048 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:48.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f76023a2640 1 --2- 172.21.15.43:0/4043433048 >> v2:172.21.15.113:3300/0 conn(0x7f75fc0b6c50 0x7f75fc0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f76013a0640 1 -- 172.21.15.43:0/4043433048 >> v2:172.21.15.113:3300/0 conn(0x7f75fc0b6c50 msgr2=0x7f75fc0b7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f76013a0640 1 --2- 172.21.15.43:0/4043433048 >> v2:172.21.15.113:3300/0 conn(0x7f75fc0b6c50 0x7f75fc0b7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f76013a0640 1 -- 172.21.15.43:0/4043433048 >> v1:172.21.15.43:6789/0 conn(0x7f75fc158b40 legacy=0x7f75fc0b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.958+0000 7f76013a0640 1 -- 172.21.15.43:0/4043433048 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc1623d0 con 0x7f75fc0cc750 2024-03-30T13:13:48.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f76013a0640 1 --2- 172.21.15.43:0/4043433048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0ccc00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f75fc162e80 tx=0x7f75ec01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140fdddfeb2018fb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7600b9f640 1 -- 172.21.15.43:0/4043433048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b0070 con 0x7f75fc0cc750 2024-03-30T13:13:48.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7600b9f640 1 -- 172.21.15.43:0/4043433048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec0ab600 con 0x7f75fc0cc750 2024-03-30T13:13:48.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7600b9f640 1 -- 172.21.15.43:0/4043433048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0ab9d0 con 0x7f75fc0cc750 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- 172.21.15.43:0/4043433048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 msgr2=0x7f75fc0ccc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 --2- 172.21.15.43:0/4043433048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0ccc00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f75fc162e80 tx=0x7f75ec01c3d0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- 172.21.15.43:0/4043433048 shutdown_connections 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 --2- 172.21.15.43:0/4043433048 >> v2:172.21.15.113:3300/0 conn(0x7f75fc0b6c50 0x7f75fc0b7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 --2- 172.21.15.43:0/4043433048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0ccc00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- 172.21.15.43:0/4043433048 >> 172.21.15.43:0/4043433048 conn(0x7f75fc0b3cf0 msgr2=0x7f75fc0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- 172.21.15.43:0/4043433048 shutdown_connections 2024-03-30T13:13:48.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- 172.21.15.43:0/4043433048 wait complete. 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 Processor -- start 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- start start 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75fc0b6c50 0x7f75fc14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75fc158b40 0x7f75fc14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0d1eb0 con 0x7f75fc0cc750 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0d1d30 con 0x7f75fc0b6c50 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f7603e05640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc0b7dc0 con 0x7f75fc158b40 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f76013a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f76013a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59596/0 (socket says 172.21.15.43:59596) 2024-03-30T13:13:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.962+0000 7f76013a0640 1 -- 172.21.15.43:0/1495564977 learned_addr learned my addr 172.21.15.43:0/1495564977 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7601ba1640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75fc0b6c50 0x7f75fc14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f76023a2640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75fc158b40 0x7f75fc14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f76013a0640 1 -- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75fc158b40 msgr2=0x7f75fc14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f76013a0640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75fc158b40 0x7f75fc14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f76013a0640 1 -- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75fc0b6c50 msgr2=0x7f75fc14e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f76013a0640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75fc0b6c50 0x7f75fc14e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f76013a0640 1 -- 172.21.15.43:0/1495564977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc14ffa0 con 0x7f75fc0cc750 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7601ba1640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75fc0b6c50 0x7f75fc14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f76013a0640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0cb2e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01a000 tx=0x7f75ec019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f75f27fc640 1 -- 172.21.15.43:0/1495564977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b0040 con 0x7f75fc0cc750 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f75f27fc640 1 -- 172.21.15.43:0/1495564977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec0b9070 con 0x7f75fc0cc750 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc14e9c0 con 0x7f75fc0cc750 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f75f27fc640 1 -- 172.21.15.43:0/1495564977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b4650 con 0x7f75fc0cc750 2024-03-30T13:13:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc14ef80 con 0x7f75fc0cc750 2024-03-30T13:13:48.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f75f27fc640 1 -- 172.21.15.43:0/1495564977 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f75ec0b47f0 con 0x7f75fc0cc750 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f75f27fc640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f75e0067fc0 0x7f75e006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f75f27fc640 1 -- 172.21.15.43:0/1495564977 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f75ec120400 con 0x7f75fc0cc750 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7601ba1640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f75e0067fc0 0x7f75e006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7606f87000 1 Processor -- start 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7606f87000 1 -- start start 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.966+0000 7f7601ba1640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f75e0067fc0 0x7f75e006a480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000f70 tx=0x7f75f8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2d40150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2cf62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cf6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fcb2b44aa0 con 0x55fcb2d3fd70 2024-03-30T13:13:48.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fcb2acbaf0 con 0x55fcb2cf6830 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fcb2b40a50 con 0x55fcb2d40690 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2d40150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2d40150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59600/0 (socket says 172.21.15.43:59600) 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 -- 172.21.15.43:0/2501674402 learned_addr learned my addr 172.21.15.43:0/2501674402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76013a0640 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2cf62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cf6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 -- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 msgr2=0x55fcb2cf62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2cf62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 -- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 msgr2=0x55fcb2cf6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cf6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 -- 172.21.15.43:0/2501674402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcb2ac9d00 con 0x55fcb2d3fd70 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2d40150 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001e70 tx=0x7f75f8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f023f7abd23c305 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f75df7fe640 1 -- 172.21.15.43:0/2501674402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8007710 con 0x55fcb2d3fd70 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f75df7fe640 1 -- 172.21.15.43:0/2501674402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f800e7e0 con 0x55fcb2d3fd70 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f75df7fe640 1 -- 172.21.15.43:0/2501674402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f800eb90 con 0x55fcb2d3fd70 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- 172.21.15.43:0/2501674402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 msgr2=0x55fcb2d40150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2d40150 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001e70 tx=0x7f75f8011000 comp rx=0 tx=0).stop 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- 172.21.15.43:0/2501674402 shutdown_connections 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cf6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2cf62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- 172.21.15.43:0/2501674402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2d40150 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- 172.21.15.43:0/2501674402 >> 172.21.15.43:0/2501674402 conn(0x55fcb2cf27e0 msgr2=0x55fcb2cf4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- 172.21.15.43:0/2501674402 shutdown_connections 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- 172.21.15.43:0/2501674402 wait complete. 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- start start 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 Processor -- start 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cfbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2d24da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d40690 0x55fcb2d252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fcb2b0c270 con 0x55fcb2d40690 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fcb2b026a0 con 0x55fcb2cf6830 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fcb2adc880 con 0x55fcb2d3fd70 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7606f87000 10 client.0 _ll_register_callbacks cb 0x55fcb2ce1490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d40690 0x55fcb2d252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d40690 0x55fcb2d252e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59602/0 (socket says 172.21.15.43:59602) 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 -- 172.21.15.43:0/2086930120 learned_addr learned my addr 172.21.15.43:0/2086930120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f7601ba1640 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cfbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76013a0640 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2d24da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 -- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 msgr2=0x55fcb2d24da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2d24da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 -- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 msgr2=0x55fcb2cfbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cfbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.970+0000 7f76023a2640 1 -- 172.21.15.43:0/2086930120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcb2b09730 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f76023a2640 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d40690 0x55fcb2d252e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007a70 tx=0x7f75f4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62712b0b5dbebf66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 1 -- 172.21.15.43:0/2086930120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f401b020 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 1 -- 172.21.15.43:0/2086930120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f75f4011b40 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f800e7e0 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 1 -- 172.21.15.43:0/2086930120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f75f4025050 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 10 client.6003 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55fcb2afcca0 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 1 -- 172.21.15.43:0/2086930120 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f800e7e0 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 1 -- 172.21.15.43:0/2086930120 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f75f4028020 con 0x55fcb2d40690 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 10 client.6003 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:48.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 10 client.6003 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 20 client.6003 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d475d0 0x55fcb2d479b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 4 client.6003 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcb2adfc90 con 0x55fcb2d475d0 2024-03-30T13:13:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7601ba1640 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d475d0 0x55fcb2d479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7601ba1640 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d475d0 0x55fcb2d479b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99c6d465322bbf1f server_cookie=afe506c10aab7867 in_seq=0 out_seq=0 2024-03-30T13:13:48.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 10 client.6003 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 1 -- 172.21.15.43:0/2086930120 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55fcb2adfc90 con 0x55fcb2d475d0 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 10 client.6003 handle_command_reply: tid=0 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 client.6003 shutdown 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d40690 msgr2=0x55fcb2d252e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d40690 0x55fcb2d252e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007a70 tx=0x7f75f4007f30 comp rx=0 tx=0).stop 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 shutdown_connections 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d475d0 0x55fcb2d479b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d40690 0x55fcb2d252e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2d24da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 --2- 172.21.15.43:0/2086930120 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2cfbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 >> 172.21.15.43:0/2086930120 conn(0x55fcb2cf27e0 msgr2=0x55fcb2cf5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:48.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f75dcff9640 0 client.6003 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:48.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 shutdown_connections 2024-03-30T13:13:48.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 1 -- 172.21.15.43:0/2086930120 wait complete. 2024-03-30T13:13:48.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.974+0000 7f7606f87000 20 client.6003 trim_cache size 0 max 16384 2024-03-30T13:13:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 Processor -- start 2024-03-30T13:13:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 -- start start 2024-03-30T13:13:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2cf6830 0x55fcb2d25180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2cf4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2d40690 0x55fcb2cf5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc203e50 con 0x55fcb2cf6830 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc202ea0 con 0x55fcb2d40690 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc153c30 con 0x55fcb2d3fd70 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2cf6830 0x55fcb2d25180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2cf6830 0x55fcb2d25180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59606/0 (socket says 172.21.15.43:59606) 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 -- 172.21.15.43:0/3123092761 learned_addr learned my addr 172.21.15.43:0/3123092761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f76023a2640 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2d40690 0x55fcb2cf5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 -- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 msgr2=0x55fcb2cf4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f76013a0640 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2cf4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2cf4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 -- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2d40690 msgr2=0x55fcb2cf5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2d40690 0x55fcb2cf5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 -- 172.21.15.43:0/3123092761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcb2acbd00 con 0x55fcb2cf6830 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f76023a2640 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2d40690 0x55fcb2cf5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f7601ba1640 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2cf6830 0x55fcb2d25180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001ec0 tx=0x7f75f8017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5916a5a8589140 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f75deffd640 1 -- 172.21.15.43:0/3123092761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f800eb20 con 0x55fcb2cf6830 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f75deffd640 1 -- 172.21.15.43:0/3123092761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f800ecc0 con 0x55fcb2cf6830 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.990+0000 7f75deffd640 1 -- 172.21.15.43:0/3123092761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f802e230 con 0x55fcb2cf6830 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/3123092761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2cf6830 msgr2=0x55fcb2d25180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2cf6830 0x55fcb2d25180 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f75f8001ec0 tx=0x7f75f8017070 comp rx=0 tx=0).stop 2024-03-30T13:13:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/3123092761 shutdown_connections 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2d40690 0x55fcb2cf5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d3fd70 0x55fcb2cf4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- 172.21.15.43:0/3123092761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2cf6830 0x55fcb2d25180 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/3123092761 >> 172.21.15.43:0/3123092761 conn(0x55fcb2cf27e0 msgr2=0x55fcb2d56f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/3123092761 shutdown_connections 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/3123092761 wait complete. 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- start start 2024-03-30T13:13:49.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 Processor -- start 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2db1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2db2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2d43ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fcb2b0c270 con 0x55fcb2d3fd70 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc0c9e30 con 0x55fcb2cf6830 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fcb2c00b00 con 0x55fcb2d40690 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 10 client.0 _ll_register_callbacks cb 0x7f75f4009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2db2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2db2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59612/0 (socket says 172.21.15.43:59612) 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 -- 172.21.15.43:0/734238395 learned_addr learned my addr 172.21.15.43:0/734238395 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7601ba1640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2db1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76023a2640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2d43ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 -- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 msgr2=0x55fcb2d43ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2d43ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 -- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 msgr2=0x55fcb2db1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2db1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 -- 172.21.15.43:0/734238395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcb2bc5f70 con 0x55fcb2d3fd70 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7601ba1640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2db1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f76013a0640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2db2320 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f75ec097d90 tx=0x7f75ec0f0ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a5a6db70474614 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f75dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f75dcff9640 1 -- 172.21.15.43:0/734238395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0f8ed0 con 0x55fcb2d3fd70 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f75dcff9640 1 -- 172.21.15.43:0/734238395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f75ec0f0550 con 0x55fcb2d3fd70 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f800ecc0 con 0x55fcb2d3fd70 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f75dcff9640 1 -- 172.21.15.43:0/734238395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f75ec0f0750 con 0x55fcb2d3fd70 2024-03-30T13:13:49.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 10 client.6009 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55fcb2aeb760 con 0x55fcb2d3fd70 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f75dcff9640 1 -- 172.21.15.43:0/734238395 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f800ecc0 con 0x55fcb2d3fd70 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f75dcff9640 1 -- 172.21.15.43:0/734238395 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f75ec0b3070 con 0x55fcb2d3fd70 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 10 client.6009 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 10 client.6009 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 20 client.6009 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d44490 0x55fcb2d448a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 4 client.6009 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628309}] 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628309}) -- 0x55fcb2b51fe0 con 0x55fcb2d44490 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.994+0000 7f7601ba1640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d44490 0x55fcb2d448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f75dcff9640 10 client.6009 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7601ba1640 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d44490 0x55fcb2d448a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e299d7063986433 server_cookie=356a4505d20abfd3 in_seq=0 out_seq=0 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f75dcff9640 1 -- 172.21.15.43:0/734238395 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3810 (crc 0 0 0) 0x55fcb2b51fe0 con 0x55fcb2d44490 2024-03-30T13:13:49.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f75dcff9640 10 client.6009 handle_command_reply: tid=0 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 client.6009 shutdown 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 msgr2=0x55fcb2db2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2db2320 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f75ec097d90 tx=0x7f75ec0f0ab0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 shutdown_connections 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fcb2d44490 0x55fcb2d448a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fcb2d40690 0x55fcb2d43ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb2d3fd70 0x55fcb2db2320 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f75dcff9640 0 client.6009 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 --2- 172.21.15.43:0/734238395 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fcb2cf6830 0x55fcb2db1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 >> 172.21.15.43:0/734238395 conn(0x55fcb2cf27e0 msgr2=0x55fcb2d57620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 shutdown_connections 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 1 -- 172.21.15.43:0/734238395 wait complete. 2024-03-30T13:13:49.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7606f87000 20 client.6009 trim_cache size 0 max 16384 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/Makefile", 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628309, 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.951109+0000", 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.443696+0000", 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:49.008 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:49.009 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 378, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.443696+0000", 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:49.010 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 378, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.951109+0000" 2024-03-30T13:13:49.011 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 378, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.951109+0000" 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 84, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 4, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:49.012 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:49.013 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:49.014 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:49.015 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.016 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.017 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.018 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:49.019 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:49.020 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:49.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f75e0067fc0 msgr2=0x7f75e006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f75e0067fc0 0x7f75e006a480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f75f8000f70 tx=0x7f75f8009040 comp rx=0 tx=0).stop 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 msgr2=0x7f75fc0cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0cb2e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f75ec01a000 tx=0x7f75ec019170 comp rx=0 tx=0).stop 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 shutdown_connections 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f75e0067fc0 0x7f75e006a480 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f75fc158b40 0x7f75fc14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75fc0cc750 0x7f75fc0cb2e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 --2- 172.21.15.43:0/1495564977 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f75fc0b6c50 0x7f75fc14e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 >> 172.21.15.43:0/1495564977 conn(0x7f75fc0b3cf0 msgr2=0x7f75fc0b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 shutdown_connections 2024-03-30T13:13:49.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:48.998+0000 7f7603e05640 1 -- 172.21.15.43:0/1495564977 wait complete. 2024-03-30T13:13:49.023 DEBUG:tasks.cephfs.test_uninlining:getting inode info #10 of 23:1099511628321 2024-03-30T13:13:49.023 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628321 2024-03-30T13:13:49.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628321 2024-03-30T13:13:49.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.138+0000 7f5f0dac5640 1 Processor -- start 2024-03-30T13:13:49.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.138+0000 7f5f0dac5640 1 -- start start 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f0dac5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f0806cd80 0x7f5f08165de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f0dac5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f080b3660 con 0x7f5f080b4870 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f0dac5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f080b3940 con 0x7f5f08144930 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f0dac5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5f080cdb90 con 0x7f5f0806cd80 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f07fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f0806cd80 0x7f5f08165de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5efffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5f080b4870 0x7f5f0806c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47234/0 (socket says 172.21.15.43:47234) 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5efffff640 1 -- 172.21.15.43:0/2737318183 learned_addr learned my addr 172.21.15.43:0/2737318183 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f06ffd640 1 -- 172.21.15.43:0/2737318183 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405035040 0 0) 0x7f5f080b3660 con 0x7f5f080b4870 2024-03-30T13:13:49.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f06ffd640 1 -- 172.21.15.43:0/2737318183 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ef4003530 con 0x7f5f080b4870 2024-03-30T13:13:49.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f07fff640 1 -- 172.21.15.43:0/2737318183 >> v1:172.21.15.113:6789/0 conn(0x7f5f08144930 legacy=0x7f5f080b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f07fff640 1 -- 172.21.15.43:0/2737318183 >> v1:172.21.15.43:6789/0 conn(0x7f5f080b4870 legacy=0x7f5f0806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f07fff640 1 -- 172.21.15.43:0/2737318183 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f080b3c20 con 0x7f5f0806cd80 2024-03-30T13:13:49.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.142+0000 7f5f07fff640 1 --2- 172.21.15.43:0/2737318183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f08165de0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009d90 tx=0x7f5f0001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9b3918df1a1f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f06ffd640 1 -- 172.21.15.43:0/2737318183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000b2070 con 0x7f5f0806cd80 2024-03-30T13:13:49.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f06ffd640 1 -- 172.21.15.43:0/2737318183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f000a5960 con 0x7f5f0806cd80 2024-03-30T13:13:49.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f06ffd640 1 -- 172.21.15.43:0/2737318183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000a5d30 con 0x7f5f0806cd80 2024-03-30T13:13:49.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2737318183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 msgr2=0x7f5f08165de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2737318183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f08165de0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009d90 tx=0x7f5f0001c880 comp rx=0 tx=0).stop 2024-03-30T13:13:49.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2737318183 shutdown_connections 2024-03-30T13:13:49.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2737318183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f08165de0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2737318183 >> 172.21.15.43:0/2737318183 conn(0x7f5f080d00a0 msgr2=0x7f5f080d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2737318183 shutdown_connections 2024-03-30T13:13:49.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2737318183 wait complete. 2024-03-30T13:13:49.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 Processor -- start 2024-03-30T13:13:49.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- start start 2024-03-30T13:13:49.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f080aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f080b4870 0x7f5f080cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f08144930 0x7f5f0815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08168ed0 con 0x7f5f0806cd80 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08168d50 con 0x7f5f080b4870 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5f08169050 con 0x7f5f08144930 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f080aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f080aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59632/0 (socket says 172.21.15.43:59632) 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 -- 172.21.15.43:0/2756295515 learned_addr learned my addr 172.21.15.43:0/2756295515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5efffff640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f080b4870 0x7f5f080cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f07fff640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f08144930 0x7f5f0815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 -- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f08144930 msgr2=0x7f5f0815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f08144930 0x7f5f0815a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 -- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f080b4870 msgr2=0x7f5f080cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f080b4870 0x7f5f080cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 -- 172.21.15.43:0/2756295515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f080cbdf0 con 0x7f5f0806cd80 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f077fe640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f080aefc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c370 tx=0x7f5ef800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f04ff9640 1 -- 172.21.15.43:0/2756295515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80052d0 con 0x7f5f0806cd80 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f04ff9640 1 -- 172.21.15.43:0/2756295515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef8005470 con 0x7f5f0806cd80 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f081f7a90 con 0x7f5f0806cd80 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f04ff9640 1 -- 172.21.15.43:0/2756295515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8007260 con 0x7f5f0806cd80 2024-03-30T13:13:49.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.146+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f081f8050 con 0x7f5f0806cd80 2024-03-30T13:13:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f04ff9640 1 -- 172.21.15.43:0/2756295515 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5ef8007400 con 0x7f5f0806cd80 2024-03-30T13:13:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f04ff9640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5ee8067fc0 0x7f5ee806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f04ff9640 1 -- 172.21.15.43:0/2756295515 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f5ef8085b70 con 0x7f5f0806cd80 2024-03-30T13:13:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5efffff640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5ee8067fc0 0x7f5ee806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 Processor -- start 2024-03-30T13:13:49.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 -- start start 2024-03-30T13:13:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5efffff640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5ee8067fc0 0x7f5ee806a480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f080af500 tx=0x7f5ef0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b115150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56470af19aa0 con 0x56470b0cb830 2024-03-30T13:13:49.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56470aea0af0 con 0x56470b114d70 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56470af15a50 con 0x56470b115690 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59640/0 (socket says 172.21.15.43:59640) 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f07fff640 1 -- 172.21.15.43:0/1362156409 learned_addr learned my addr 172.21.15.43:0/1362156409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5efffff640 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.150+0000 7f5f077fe640 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b115150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 -- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 msgr2=0x56470b0cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 -- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 msgr2=0x56470b115150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b115150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 -- 172.21.15.43:0/1362156409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470ae9ed00 con 0x56470b0cb830 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0cbc10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5f000b0040 tx=0x7f5f00004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7ef8f0ec17a685 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5efd7fa640 1 -- 172.21.15.43:0/1362156409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000b2040 con 0x56470b0cb830 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5efd7fa640 1 -- 172.21.15.43:0/1362156409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f000a5ac0 con 0x56470b0cb830 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5efd7fa640 1 -- 172.21.15.43:0/1362156409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000b8440 con 0x56470b0cb830 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- 172.21.15.43:0/1362156409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 msgr2=0x56470b0cbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0cbc10 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5f000b0040 tx=0x7f5f00004ac0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- 172.21.15.43:0/1362156409 shutdown_connections 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0cbc10 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1362156409 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b115150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- 172.21.15.43:0/1362156409 >> 172.21.15.43:0/1362156409 conn(0x56470b0c77e0 msgr2=0x56470b0c9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- 172.21.15.43:0/1362156409 shutdown_connections 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- 172.21.15.43:0/1362156409 wait complete. 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- start start 2024-03-30T13:13:49.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 Processor -- start 2024-03-30T13:13:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b115690 0x56470b0fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56470aee1270 con 0x56470b115690 2024-03-30T13:13:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56470aed76a0 con 0x56470b114d70 2024-03-30T13:13:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56470aeb1880 con 0x56470b0cb830 2024-03-30T13:13:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 10 client.0 _ll_register_callbacks cb 0x56470b0b6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b115690 0x56470b0fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b115690 0x56470b0fa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59656/0 (socket says 172.21.15.43:59656) 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 -- 172.21.15.43:0/1554778053 learned_addr learned my addr 172.21.15.43:0/1554778053 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5efffff640 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f077fe640 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 -- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 msgr2=0x56470b0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b0d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 -- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 msgr2=0x56470b0f9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0f9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 -- 172.21.15.43:0/1554778053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470aede730 con 0x56470b115690 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f07fff640 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b115690 0x56470b0fa2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5f00000f90 tx=0x7f5f00004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60de6310ccfbbfe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5ee67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5ee67fc640 1 -- 172.21.15.43:0/1554778053 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f000039a0 con 0x56470b115690 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5ee67fc640 1 -- 172.21.15.43:0/1554778053 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f5f00003b40 con 0x56470b115690 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f000a5ac0 con 0x56470b115690 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5ee67fc640 1 -- 172.21.15.43:0/1554778053 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5f000cc050 con 0x56470b115690 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 10 client.6024 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.154+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56470aed1ca0 con 0x56470b115690 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5ee67fc640 1 -- 172.21.15.43:0/1554778053 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f000a5ac0 con 0x56470b115690 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5ee67fc640 1 -- 172.21.15.43:0/1554778053 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f5f000cf020 con 0x56470b115690 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 10 client.6024 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 10 client.6024 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 20 client.6024 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b11c5d0 0x56470b11c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 4 client.6024 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56470aeb4c90 con 0x56470b11c5d0 2024-03-30T13:13:49.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f077fe640 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b11c5d0 0x56470b11c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f077fe640 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b11c5d0 0x56470b11c9b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c5394ab5566957 server_cookie=dd290c43ef2747c in_seq=0 out_seq=0 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5ee67fc640 10 client.6024 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5ee67fc640 1 -- 172.21.15.43:0/1554778053 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x56470aeb4c90 con 0x56470b11c5d0 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5ee67fc640 10 client.6024 handle_command_reply: tid=0 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 client.6024 shutdown 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b115690 msgr2=0x56470b0fa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b115690 0x56470b0fa2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5f00000f90 tx=0x7f5f00004a50 comp rx=0 tx=0).stop 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 shutdown_connections 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b11c5d0 0x56470b11c9b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b115690 0x56470b0fa2e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5ee67fc640 0 client.6024 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0f9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1554778053 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 >> 172.21.15.43:0/1554778053 conn(0x56470b0c77e0 msgr2=0x56470b0ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 shutdown_connections 2024-03-30T13:13:49.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 1 -- 172.21.15.43:0/1554778053 wait complete. 2024-03-30T13:13:49.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.158+0000 7f5f10c47000 20 client.6024 trim_cache size 0 max 16384 2024-03-30T13:13:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 Processor -- start 2024-03-30T13:13:49.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- start start 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08204050 con 0x56470b0cb830 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08203130 con 0x56470b114d70 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5f080b04b0 con 0x56470b115690 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59658/0 (socket says 172.21.15.43:59658) 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 -- 172.21.15.43:0/1217339525 learned_addr learned my addr 172.21.15.43:0/1217339525 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5efffff640 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f07fff640 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 -- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 msgr2=0x56470b0ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 -- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 msgr2=0x56470b0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 -- 172.21.15.43:0/1217339525 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470aea0d00 con 0x56470b0cb830 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5efffff640 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f077fe640 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0c9bd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8051e60 tx=0x7f5ef805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377d9ee852a1a4ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5efcff9640 1 -- 172.21.15.43:0/1217339525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8053d50 con 0x56470b0cb830 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5efcff9640 1 -- 172.21.15.43:0/1217339525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef80520c0 con 0x56470b0cb830 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5efcff9640 1 -- 172.21.15.43:0/1217339525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef805c2d0 con 0x56470b0cb830 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- 172.21.15.43:0/1217339525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 msgr2=0x56470b0c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0c9bd0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8051e60 tx=0x7f5ef805c040 comp rx=0 tx=0).stop 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- 172.21.15.43:0/1217339525 shutdown_connections 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b115690 0x56470b0ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b114d70 0x56470b0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 --2- 172.21.15.43:0/1217339525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b0cb830 0x56470b0c9bd0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- 172.21.15.43:0/1217339525 >> 172.21.15.43:0/1217339525 conn(0x56470b0c77e0 msgr2=0x56470b12b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- 172.21.15.43:0/1217339525 shutdown_connections 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.174+0000 7f5f10c47000 1 -- 172.21.15.43:0/1217339525 wait complete. 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 -- start start 2024-03-30T13:13:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 Processor -- start 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b186c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b114d70 0x56470b1871b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b115690 0x56470b118ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56470aed0720 con 0x56470b114d70 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5f080ae3a0 con 0x56470b115690 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56470b12cfb0 con 0x56470b0cb830 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 10 client.0 _ll_register_callbacks cb 0x7f5ef4003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b114d70 0x56470b1871b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b114d70 0x56470b1871b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59666/0 (socket says 172.21.15.43:59666) 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 -- 172.21.15.43:0/494666633 learned_addr learned my addr 172.21.15.43:0/494666633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f07fff640 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b115690 0x56470b118ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f077fe640 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b186c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 -- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 msgr2=0x56470b186c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b186c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 -- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b115690 msgr2=0x56470b118ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b115690 0x56470b118ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 -- 172.21.15.43:0/494666633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470af9af70 con 0x56470b114d70 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5efffff640 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b114d70 0x56470b1871b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0005cf0 tx=0x7f5ef0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f4580b59afba4e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 1 -- 172.21.15.43:0/494666633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0029060 con 0x56470b114d70 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 1 -- 172.21.15.43:0/494666633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f5ef0010040 con 0x56470b114d70 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef80520c0 con 0x56470b114d70 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 1 -- 172.21.15.43:0/494666633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5ef002f050 con 0x56470b114d70 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 10 client.6030 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56470aec0760 con 0x56470b114d70 2024-03-30T13:13:49.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 1 -- 172.21.15.43:0/494666633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef80520c0 con 0x56470b114d70 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 1 -- 172.21.15.43:0/494666633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f5ef0032020 con 0x56470b114d70 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 10 client.6030 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 10 client.6030 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 20 client.6030 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b1198b0 0x56470b119cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 4 client.6030 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628321}] 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628321}) -- 0x56470af26fe0 con 0x56470b1198b0 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f077fe640 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b1198b0 0x56470b119cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 10 client.6030 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5f077fe640 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b1198b0 0x56470b119cc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c318ed9ea6d023ed server_cookie=27a9fed7f48cde6c in_seq=0 out_seq=0 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.178+0000 7f5ee67fc640 1 -- 172.21.15.43:0/494666633 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3812 (crc 0 0 0) 0x56470af26fe0 con 0x56470b1198b0 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5ee67fc640 10 client.6030 handle_command_reply: tid=0 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 client.6030 shutdown 2024-03-30T13:13:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b114d70 msgr2=0x56470b1871b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b114d70 0x56470b1871b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0005cf0 tx=0x7f5ef0000ee0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 shutdown_connections 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56470b1198b0 0x56470b119cc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56470b115690 0x56470b118ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56470b114d70 0x56470b1871b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5ee67fc640 0 client.6030 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 --2- 172.21.15.43:0/494666633 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56470b0cb830 0x56470b186c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 >> 172.21.15.43:0/494666633 conn(0x56470b0c77e0 msgr2=0x56470b12bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 shutdown_connections 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 1 -- 172.21.15.43:0/494666633 wait complete. 2024-03-30T13:13:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f10c47000 20 client.6030 trim_cache size 0 max 16384 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/symlink.c", 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628321, 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.953490+0000", 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.541065+0000", 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 916, 2024-03-30T13:13:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.541065+0000", 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 916, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.953490+0000" 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 916, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.953490+0000" 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 86, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 28, 2024-03-30T13:13:49.195 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:49.196 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.197 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:49.198 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:49.199 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:49.200 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.201 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:49.202 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5ee8067fc0 msgr2=0x7f5ee806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5ee8067fc0 0x7f5ee806a480 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f080af500 tx=0x7f5ef0009040 comp rx=0 tx=0).stop 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 msgr2=0x7f5f080aefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f080aefc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c370 tx=0x7f5ef800c830 comp rx=0 tx=0).stop 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 shutdown_connections 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5ee8067fc0 0x7f5ee806a480 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5f08144930 0x7f5f0815a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5f080b4870 0x7f5f080cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 --2- 172.21.15.43:0/2756295515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f0806cd80 0x7f5f080aefc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 >> 172.21.15.43:0/2756295515 conn(0x7f5f080d00a0 msgr2=0x7f5f0806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 shutdown_connections 2024-03-30T13:13:49.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.182+0000 7f5f0dac5640 1 -- 172.21.15.43:0/2756295515 wait complete. 2024-03-30T13:13:49.205 DEBUG:tasks.cephfs.test_uninlining:getting inode info #11 of 23:1099511628311 2024-03-30T13:13:49.205 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628311 2024-03-30T13:13:49.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628311 2024-03-30T13:13:49.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cbcbf3640 1 Processor -- start 2024-03-30T13:13:49.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cbcbf3640 1 -- start start 2024-03-30T13:13:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cbcbf3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cb806c670 0x7f9cb806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cbcbf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb80b40b0 0x7f9cb80b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cbcbf3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb814c680 con 0x7f9cb814c2a0 2024-03-30T13:13:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cbcbf3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb80b4a60 con 0x7f9cb806c670 2024-03-30T13:13:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cbcbf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb8164750 con 0x7f9cb80b40b0 2024-03-30T13:13:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb5d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cb806c670 0x7f9cb806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb5d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cb806c670 0x7f9cb806cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59672/0 (socket says 172.21.15.43:59672) 2024-03-30T13:13:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/4241479023 learned_addr learned my addr 172.21.15.43:0/4241479023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/4241479023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb80b40b0 0x7f9cb80b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/4241479023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb80b40b0 msgr2=0x7f9cb80b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/4241479023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb80b40b0 0x7f9cb80b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/4241479023 >> v1:172.21.15.113:6789/0 conn(0x7f9cb814c2a0 legacy=0x7f9cb8164040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.318+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/4241479023 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb80b3310 con 0x7f9cb806c670 2024-03-30T13:13:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/4241479023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb806c670 0x7f9cb806cb20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009870 tx=0x7f9cac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8482fe6d597b57 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb559a640 1 -- 172.21.15.43:0/4241479023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0a7070 con 0x7f9cb806c670 2024-03-30T13:13:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb559a640 1 -- 172.21.15.43:0/4241479023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cac0b0070 con 0x7f9cb806c670 2024-03-30T13:13:49.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb559a640 1 -- 172.21.15.43:0/4241479023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0ab890 con 0x7f9cb806c670 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/4241479023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb806c670 msgr2=0x7f9cb806cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/4241479023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb806c670 0x7f9cb806cb20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9cac009870 tx=0x7f9cac01c560 comp rx=0 tx=0).stop 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/4241479023 shutdown_connections 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/4241479023 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb80b40b0 0x7f9cb80b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/4241479023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb806c670 0x7f9cb806cb20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/4241479023 >> 172.21.15.43:0/4241479023 conn(0x7f9cb80d00a0 msgr2=0x7f9cb80d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/4241479023 shutdown_connections 2024-03-30T13:13:49.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/4241479023 wait complete. 2024-03-30T13:13:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 Processor -- start 2024-03-30T13:13:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- start start 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb806c670 0x7f9cb8165e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb80b40b0 0x7f9cb815a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb814c2a0 0x7f9cb81488e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8163b20 con 0x7f9cb814c2a0 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb81639a0 con 0x7f9cb80b40b0 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cbcbf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb8163ca0 con 0x7f9cb806c670 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb814c2a0 0x7f9cb81488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb814c2a0 0x7f9cb81488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59688/0 (socket says 172.21.15.43:59688) 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/3056534770 learned_addr learned my addr 172.21.15.43:0/3056534770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb80b40b0 0x7f9cb815a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb659c640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb806c670 0x7f9cb8165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb806c670 msgr2=0x7f9cb8165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb806c670 0x7f9cb8165e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb80b40b0 msgr2=0x7f9cb815a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb80b40b0 0x7f9cb815a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/3056534770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb8162640 con 0x7f9cb814c2a0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb80b40b0 0x7f9cb815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.322+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb814c2a0 0x7f9cb81488e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d800 tx=0x7f9ca800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9ca6ffd640 1 -- 172.21.15.43:0/3056534770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800b660 con 0x7f9cb814c2a0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9ca6ffd640 1 -- 172.21.15.43:0/3056534770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca8019070 con 0x7f9cb814c2a0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb81618c0 con 0x7f9cb814c2a0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9ca6ffd640 1 -- 172.21.15.43:0/3056534770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca800ba40 con 0x7f9cb814c2a0 2024-03-30T13:13:49.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb8161e40 con 0x7f9cb814c2a0 2024-03-30T13:13:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9ca6ffd640 1 -- 172.21.15.43:0/3056534770 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9ca800bbe0 con 0x7f9cb814c2a0 2024-03-30T13:13:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9ca6ffd640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c98067ff0 0x7f9c9806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9ca6ffd640 1 -- 172.21.15.43:0/3056534770 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f9ca8086ac0 con 0x7f9cb814c2a0 2024-03-30T13:13:49.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cb659c640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c98067ff0 0x7f9c9806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 Processor -- start 2024-03-30T13:13:49.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 -- start start 2024-03-30T13:13:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cb659c640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c98067ff0 0x7f9c9806a4b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8140e30 tx=0x7f9cb0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a5309d7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53098d830 0x55a53098dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a5307dbaa0 con 0x55a53098d830 2024-03-30T13:13:49.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a530762af0 con 0x55a5309d6d70 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a5307d7a50 con 0x55a5309d7690 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.326+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53098d830 0x55a53098dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53098d830 0x55a53098dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59700/0 (socket says 172.21.15.43:59700) 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2936074463 learned_addr learned my addr 172.21.15.43:0/2936074463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb659c640 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a5309d7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 msgr2=0x55a53098d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 msgr2=0x55a5309d7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a5309d7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2936074463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a530760d00 con 0x55a53098d830 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb659c640 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a5309d7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53098d830 0x55a53098dc10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9ca805f060 tx=0x7f9ca80570b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d4f4557096bd5d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9c97fff640 1 -- 172.21.15.43:0/2936074463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8083070 con 0x55a53098d830 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9c97fff640 1 -- 172.21.15.43:0/2936074463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca805ad60 con 0x55a53098d830 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9c97fff640 1 -- 172.21.15.43:0/2936074463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca805f590 con 0x55a53098d830 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2936074463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53098d830 msgr2=0x55a53098dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53098d830 0x55a53098dc10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9ca805f060 tx=0x7f9ca80570b0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2936074463 shutdown_connections 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a53098d830 0x55a53098dc10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2936074463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a5309d7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2936074463 >> 172.21.15.43:0/2936074463 conn(0x55a5309897e0 msgr2=0x55a53098bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2936074463 shutdown_connections 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2936074463 wait complete. 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- start start 2024-03-30T13:13:49.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 Processor -- start 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a530992c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d6d70 0x55a5309bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a5307a3270 con 0x55a5309d7690 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a5307996a0 con 0x55a53098d830 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a530773880 con 0x55a5309d6d70 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 10 client.0 _ll_register_callbacks cb 0x55a530978490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309bc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59708/0 (socket says 172.21.15.43:59708) 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2774689792 learned_addr learned my addr 172.21.15.43:0/2774689792 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d6d70 0x55a5309bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb659c640 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a530992c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d6d70 msgr2=0x55a5309bbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d6d70 0x55a5309bbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 msgr2=0x55a530992c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a530992c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/2774689792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5307a0730 con 0x55a5309d7690 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309bc2e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8057260 tx=0x7f9ca8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100c29035c80f7a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9c957fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9c957fa640 1 -- 172.21.15.43:0/2774689792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8083070 con 0x55a5309d7690 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9c957fa640 1 -- 172.21.15.43:0/2774689792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f9ca8075130 con 0x55a5309d7690 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca805ad60 con 0x55a5309d7690 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9c957fa640 1 -- 172.21.15.43:0/2774689792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9ca8075350 con 0x55a5309d7690 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 10 client.6042 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.330+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55a530793ca0 con 0x55a5309d7690 2024-03-30T13:13:49.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9c957fa640 1 -- 172.21.15.43:0/2774689792 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca805ad60 con 0x55a5309d7690 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9c957fa640 1 -- 172.21.15.43:0/2774689792 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f9ca8057ce0 con 0x55a5309d7690 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 10 client.6042 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 10 client.6042 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 20 client.6042 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309de5d0 0x55a5309de9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 4 client.6042 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a530776c90 con 0x55a5309de5d0 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cb659c640 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309de5d0 0x55a5309de9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9c957fa640 10 client.6042 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cb659c640 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309de5d0 0x55a5309de9b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea15d904d8620cb2 server_cookie=aa2b408152856407 in_seq=0 out_seq=0 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9c957fa640 1 -- 172.21.15.43:0/2774689792 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55a530776c90 con 0x55a5309de5d0 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9c957fa640 10 client.6042 handle_command_reply: tid=0 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 client.6042 shutdown 2024-03-30T13:13:49.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 msgr2=0x55a5309bc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309bc2e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8057260 tx=0x7f9ca8000bd0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 shutdown_connections 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309de5d0 0x55a5309de9b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9c957fa640 0 client.6042 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309bc2e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d6d70 0x55a5309bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/2774689792 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a530992c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 >> 172.21.15.43:0/2774689792 conn(0x55a5309897e0 msgr2=0x55a53098c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 shutdown_connections 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 1 -- 172.21.15.43:0/2774689792 wait complete. 2024-03-30T13:13:49.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.334+0000 7f9cbfd75000 20 client.6042 trim_cache size 0 max 16384 2024-03-30T13:13:49.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 Processor -- start 2024-03-30T13:13:49.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- start start 2024-03-30T13:13:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a5309bc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d6d70 0x55a53098bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8208410 con 0x55a5309d6d70 2024-03-30T13:13:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8207450 con 0x55a53098d830 2024-03-30T13:13:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb8147f90 con 0x55a5309d7690 2024-03-30T13:13:49.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d6d70 0x55a53098bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d6d70 0x55a53098bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59722/0 (socket says 172.21.15.43:59722) 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/18497918 learned_addr learned my addr 172.21.15.43:0/18497918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb659c640 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a5309bc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 msgr2=0x55a53098c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 msgr2=0x55a5309bc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a5309bc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 -- 172.21.15.43:0/18497918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a530762d00 con 0x55a5309d6d70 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d6d70 0x55a53098bbd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9cac01a190 tx=0x7f9cac0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff4462afc850e54 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9c977fe640 1 -- 172.21.15.43:0/18497918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0ab990 con 0x55a5309d6d70 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9c977fe640 1 -- 172.21.15.43:0/18497918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cac0c3070 con 0x55a5309d6d70 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9c977fe640 1 -- 172.21.15.43:0/18497918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0abd70 con 0x55a5309d6d70 2024-03-30T13:13:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- 172.21.15.43:0/18497918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d6d70 msgr2=0x55a53098bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d6d70 0x55a53098bbd0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9cac01a190 tx=0x7f9cac0a7000 comp rx=0 tx=0).stop 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- 172.21.15.43:0/18497918 shutdown_connections 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a5309d7690 0x55a53098c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d6d70 0x55a53098bbd0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/18497918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a53098d830 0x55a5309bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- 172.21.15.43:0/18497918 >> 172.21.15.43:0/18497918 conn(0x55a5309897e0 msgr2=0x55a5309edf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- 172.21.15.43:0/18497918 shutdown_connections 2024-03-30T13:13:49.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.350+0000 7f9cbfd75000 1 -- 172.21.15.43:0/18497918 wait complete. 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 -- start start 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 Processor -- start 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a53098d830 0x55a530a48ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a530a49400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309daad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8206ed0 con 0x55a5309d7690 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a5307dadb0 con 0x55a5309d6d70 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a5307996a0 con 0x55a53098d830 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 10 client.0 _ll_register_callbacks cb 0x7f9ca8081070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309daad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309daad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59726/0 (socket says 172.21.15.43:59726) 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/1100157375 learned_addr learned my addr 172.21.15.43:0/1100157375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb659c640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a53098d830 0x55a530a48ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a530a49400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a53098d830 msgr2=0x55a530a48ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a53098d830 0x55a530a48ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 msgr2=0x55a530a49400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a530a49400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 -- 172.21.15.43:0/1100157375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a53085cf70 con 0x55a5309d7690 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb5d9b640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a530a49400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb659c640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a53098d830 0x55a530a48ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb6d9d640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309daad0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80566b0 tx=0x7f9ca8052c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08f818258c4efc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 1 -- 172.21.15.43:0/1100157375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca807e580 con 0x55a5309d7690 2024-03-30T13:13:49.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 1 -- 172.21.15.43:0/1100157375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f9ca801fcd0 con 0x55a5309d7690 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9cac0c3070 con 0x55a5309d7690 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 1 -- 172.21.15.43:0/1100157375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9ca805ad60 con 0x55a5309d7690 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 10 client.6048 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55a530782760 con 0x55a5309d7690 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 1 -- 172.21.15.43:0/1100157375 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cac0c3070 con 0x55a5309d7690 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 1 -- 172.21.15.43:0/1100157375 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f9ca8013070 con 0x55a5309d7690 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 10 client.6048 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 10 client.6048 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 20 client.6048 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309db460 0x55a5309db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 4 client.6048 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628311}] 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628311}) -- 0x55a5307e8fe0 con 0x55a5309db460 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb659c640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309db460 0x55a5309db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9cb659c640 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309db460 0x55a5309db870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b9d205562fea56 server_cookie=4daaacbfc9bd85c4 in_seq=0 out_seq=0 2024-03-30T13:13:49.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 10 client.6048 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 1 -- 172.21.15.43:0/1100157375 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3810 (crc 0 0 0) 0x55a5307e8fe0 con 0x55a5309db460 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.354+0000 7f9c957fa640 10 client.6048 handle_command_reply: tid=0 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 client.6048 shutdown 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 msgr2=0x55a5309daad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309daad0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9ca80566b0 tx=0x7f9ca8052c50 comp rx=0 tx=0).stop 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 shutdown_connections 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a5309db460 0x55a5309db870 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a5309d7690 0x55a5309daad0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a5309d6d70 0x55a530a49400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 --2- 172.21.15.43:0/1100157375 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a53098d830 0x55a530a48ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 >> 172.21.15.43:0/1100157375 conn(0x55a5309897e0 msgr2=0x55a5309ee620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9c957fa640 0 client.6048 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 shutdown_connections 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 1 -- 172.21.15.43:0/1100157375 wait complete. 2024-03-30T13:13:49.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbfd75000 20 client.6048 trim_cache size 0 max 16384 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/acl.h", 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628311, 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.957756+0000", 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.459836+0000", 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:49.366 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:49.367 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1502, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.459836+0000", 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:49.368 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1502, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.957756+0000" 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:49.369 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1502, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.957756+0000" 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 88, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 8, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:49.370 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:49.371 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:49.372 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:49.373 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.374 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.375 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:49.376 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:49.377 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c98067ff0 msgr2=0x7f9c9806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c98067ff0 0x7f9c9806a4b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8140e30 tx=0x7f9cb0009040 comp rx=0 tx=0).stop 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb814c2a0 msgr2=0x7f9cb81488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb814c2a0 0x7f9cb81488e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800d800 tx=0x7f9ca800dcc0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 shutdown_connections 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c98067ff0 0x7f9c9806a4b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cb814c2a0 0x7f9cb81488e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9cb80b40b0 0x7f9cb815a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 --2- 172.21.15.43:0/3056534770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9cb806c670 0x7f9cb8165e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 >> 172.21.15.43:0/3056534770 conn(0x7f9cb80d00a0 msgr2=0x7f9cb8165730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 shutdown_connections 2024-03-30T13:13:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.358+0000 7f9cbcbf3640 1 -- 172.21.15.43:0/3056534770 wait complete. 2024-03-30T13:13:49.380 DEBUG:tasks.cephfs.test_uninlining:getting inode info #12 of 23:1099511628325 2024-03-30T13:13:49.380 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628325 2024-03-30T13:13:49.380 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628325 2024-03-30T13:13:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efddd681640 1 Processor -- start 2024-03-30T13:13:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efddd681640 1 -- start start 2024-03-30T13:13:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efddd681640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7efdd815e4c0 0x7efdd80b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efddd681640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd80cf6c0 0x7efdd80d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efddd681640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdd806c850 con 0x7efdd806c070 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efddd681640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7efdd815e8a0 con 0x7efdd815e4c0 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efddd681640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efdd806cb30 con 0x7efdd80cf6c0 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd67fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efdd806c070 0x7efdd80cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47242/0 (socket says 172.21.15.43:47242) 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd67fc640 1 -- 172.21.15.43:0/1140539795 learned_addr learned my addr 172.21.15.43:0/1140539795 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1140539795 >> v2:172.21.15.113:3300/0 conn(0x7efdd815e4c0 0x7efdd80b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd77fe640 1 --2- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd80cf6c0 0x7efdd80d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd5ffb640 1 -- 172.21.15.43:0/1140539795 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3091497799 0 0) 0x7efdd806c850 con 0x7efdd806c070 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd5ffb640 1 -- 172.21.15.43:0/1140539795 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdb8003530 con 0x7efdd806c070 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd80cf6c0 msgr2=0x7efdd80d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd80cf6c0 0x7efdd80d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1140539795 >> v1:172.21.15.43:6789/0 conn(0x7efdd806c070 legacy=0x7efdd80cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.494+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1140539795 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd806ce10 con 0x7efdd815e4c0 2024-03-30T13:13:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd80b9e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdd00097c0 tx=0x7efdd001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778c8f647c03a36c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efdd5ffb640 1 -- 172.21.15.43:0/1140539795 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b0070 con 0x7efdd815e4c0 2024-03-30T13:13:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efdd5ffb640 1 -- 172.21.15.43:0/1140539795 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd00ab600 con 0x7efdd815e4c0 2024-03-30T13:13:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efdd5ffb640 1 -- 172.21.15.43:0/1140539795 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00ab9b0 con 0x7efdd815e4c0 2024-03-30T13:13:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 -- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 msgr2=0x7efdd80b9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 --2- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd80b9e30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efdd00097c0 tx=0x7efdd001c4a0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 -- 172.21.15.43:0/1140539795 shutdown_connections 2024-03-30T13:13:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 --2- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd80cf6c0 0x7efdd80d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 --2- 172.21.15.43:0/1140539795 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd80b9e30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 -- 172.21.15.43:0/1140539795 >> 172.21.15.43:0/1140539795 conn(0x7efdd80b6410 msgr2=0x7efdd80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 -- 172.21.15.43:0/1140539795 shutdown_connections 2024-03-30T13:13:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.498+0000 7efddd681640 1 -- 172.21.15.43:0/1140539795 wait complete. 2024-03-30T13:13:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 Processor -- start 2024-03-30T13:13:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 -- start start 2024-03-30T13:13:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd806c070 0x7efdd81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd80cf6c0 0x7efdd81592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd815b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80d3ed0 con 0x7efdd80cf6c0 2024-03-30T13:13:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efdd80d3d50 con 0x7efdd815e4c0 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efdd80d4050 con 0x7efdd806c070 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd80cf6c0 0x7efdd81592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd80cf6c0 0x7efdd81592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59740/0 (socket says 172.21.15.43:59740) 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 -- 172.21.15.43:0/1548719961 learned_addr learned my addr 172.21.15.43:0/1548719961 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd77fe640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd815b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd806c070 0x7efdd81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 -- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd806c070 msgr2=0x7efdd81566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd806c070 0x7efdd81566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 -- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 msgr2=0x7efdd815b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd815b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 -- 172.21.15.43:0/1548719961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd80ca8e0 con 0x7efdd80cf6c0 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd77fe640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdd67fc640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd80cf6c0 0x7efdd81592b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7efdc000caa0 tx=0x7efdc000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdcf7fe640 1 -- 172.21.15.43:0/1548719961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0017070 con 0x7efdd80cf6c0 2024-03-30T13:13:49.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdcf7fe640 1 -- 172.21.15.43:0/1548719961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdc000aba0 con 0x7efdd80cf6c0 2024-03-30T13:13:49.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdcf7fe640 1 -- 172.21.15.43:0/1548719961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc0012400 con 0x7efdd80cf6c0 2024-03-30T13:13:49.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd8155c00 con 0x7efdd80cf6c0 2024-03-30T13:13:49.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd80cabc0 con 0x7efdd80cf6c0 2024-03-30T13:13:49.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.502+0000 7efdcf7fe640 1 -- 172.21.15.43:0/1548719961 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7efdc001a070 con 0x7efdd80cf6c0 2024-03-30T13:13:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdcf7fe640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdbc067fc0 0x7efdbc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdcf7fe640 1 -- 172.21.15.43:0/1548719961 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7efdc0086480 con 0x7efdd80cf6c0 2024-03-30T13:13:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdbc067fc0 0x7efdbc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 Processor -- start 2024-03-30T13:13:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- start start 2024-03-30T13:13:49.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdbc067fc0 0x7efdbc06a480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ad000 tx=0x7efdd00a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d4dd70 0x563d93d4e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d04830 0x563d93d04c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d93b52aa0 con 0x563d93d4dd70 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563d93ad9af0 con 0x563d93d04830 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563d93b4ea50 con 0x563d93d4e690 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d4dd70 0x563d93d4e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d4dd70 0x563d93d4e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59750/0 (socket says 172.21.15.43:59750) 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 -- 172.21.15.43:0/2940144948 learned_addr learned my addr 172.21.15.43:0/2940144948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd77fe640 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d04830 0x563d93d04c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd67fc640 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 -- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 msgr2=0x563d93d042f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d042f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 -- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d04830 msgr2=0x563d93d04c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d04830 0x563d93d04c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 -- 172.21.15.43:0/2940144948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d93ad7d00 con 0x563d93d4dd70 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd67fc640 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd77fe640 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d04830 0x563d93d04c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d4dd70 0x563d93d4e150 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efdd001ab50 tx=0x7efdd0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cc425740fefb36 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdccff9640 1 -- 172.21.15.43:0/2940144948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00c6070 con 0x563d93d4dd70 2024-03-30T13:13:49.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdccff9640 1 -- 172.21.15.43:0/2940144948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0003af0 con 0x563d93d4dd70 2024-03-30T13:13:49.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efdccff9640 1 -- 172.21.15.43:0/2940144948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00b0040 con 0x563d93d4dd70 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- 172.21.15.43:0/2940144948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d4dd70 msgr2=0x563d93d4e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d4dd70 0x563d93d4e150 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7efdd001ab50 tx=0x7efdd0004740 comp rx=0 tx=0).stop 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- 172.21.15.43:0/2940144948 shutdown_connections 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d04830 0x563d93d04c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 --2- 172.21.15.43:0/2940144948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d4dd70 0x563d93d4e150 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- 172.21.15.43:0/2940144948 >> 172.21.15.43:0/2940144948 conn(0x563d93d007e0 msgr2=0x563d93d02bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- 172.21.15.43:0/2940144948 shutdown_connections 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.506+0000 7efde0803000 1 -- 172.21.15.43:0/2940144948 wait complete. 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 -- start start 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 Processor -- start 2024-03-30T13:13:49.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d09c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 0x563d93d32da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d93b1a270 con 0x563d93d04830 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563d93b106a0 con 0x563d93d4dd70 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563d93aea880 con 0x563d93d4e690 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 10 client.0 _ll_register_callbacks cb 0x563d93cef490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d09c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d09c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59762/0 (socket says 172.21.15.43:59762) 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 -- 172.21.15.43:0/3149978938 learned_addr learned my addr 172.21.15.43:0/3149978938 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd67fc640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 0x563d93d32da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd77fe640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 -- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 msgr2=0x563d93d332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 -- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 msgr2=0x563d93d32da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 0x563d93d32da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 -- 172.21.15.43:0/3149978938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d93b17730 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd77fe640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d09c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efdd001ab50 tx=0x7efdd001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a859c65a9a765f89 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdc5ffb640 1 -- 172.21.15.43:0/3149978938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00cf060 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdc5ffb640 1 -- 172.21.15.43:0/3149978938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7efdd00c16b0 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd0003af0 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdc5ffb640 1 -- 172.21.15.43:0/3149978938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efdd00b0070 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 10 client.6060 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x563d93b0aca0 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdc5ffb640 1 -- 172.21.15.43:0/3149978938 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0003af0 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdc5ffb640 1 -- 172.21.15.43:0/3149978938 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7efdd00059a0 con 0x563d93d04830 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 10 client.6060 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 10 client.6060 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 20 client.6060 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d555d0 0x563d93d559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 4 client.6060 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d93aedc90 con 0x563d93d555d0 2024-03-30T13:13:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd67fc640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d555d0 0x563d93d559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdd67fc640 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d555d0 0x563d93d559e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80987f41b5359ce2 server_cookie=bb1c3ce289544e17 in_seq=0 out_seq=0 2024-03-30T13:13:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.510+0000 7efdc5ffb640 10 client.6060 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efdc5ffb640 1 -- 172.21.15.43:0/3149978938 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x563d93aedc90 con 0x563d93d555d0 2024-03-30T13:13:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efdc5ffb640 10 client.6060 handle_command_reply: tid=0 2024-03-30T13:13:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 client.6060 shutdown 2024-03-30T13:13:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 msgr2=0x563d93d09c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d09c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7efdd001ab50 tx=0x7efdd001ace0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 shutdown_connections 2024-03-30T13:13:49.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d555d0 0x563d93d559e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 0x563d93d32da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efdc5ffb640 0 client.6060 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 --2- 172.21.15.43:0/3149978938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d09c80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 >> 172.21.15.43:0/3149978938 conn(0x563d93d007e0 msgr2=0x563d93d03460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 shutdown_connections 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 1 -- 172.21.15.43:0/3149978938 wait complete. 2024-03-30T13:13:49.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.514+0000 7efde0803000 20 client.6060 trim_cache size 0 max 16384 2024-03-30T13:13:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 Processor -- start 2024-03-30T13:13:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- start start 2024-03-30T13:13:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d33180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 0x563d93d50a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d50fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8203bd0 con 0x563d93d04830 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efdd8202bf0 con 0x563d93d4dd70 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7efdd8152f30 con 0x563d93d4e690 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d33180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d33180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59770/0 (socket says 172.21.15.43:59770) 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1812149678 learned_addr learned my addr 172.21.15.43:0/1812149678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1812149678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 msgr2=0x563d93d50fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1812149678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d50fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1812149678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 msgr2=0x563d93d50a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1812149678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 0x563d93d50a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1812149678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d93ad9d00 con 0x563d93d04830 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1812149678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d33180 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efdd001a0d0 tx=0x7efdd00c6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a231a2b848b3621 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdc7fff640 1 -- 172.21.15.43:0/1812149678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00c6070 con 0x563d93d04830 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdc7fff640 1 -- 172.21.15.43:0/1812149678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd00039f0 con 0x563d93d04830 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdc7fff640 1 -- 172.21.15.43:0/1812149678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00c1900 con 0x563d93d04830 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- 172.21.15.43:0/1812149678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 msgr2=0x563d93d33180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- 172.21.15.43:0/1812149678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d33180 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7efdd001a0d0 tx=0x7efdd00c6000 comp rx=0 tx=0).stop 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- 172.21.15.43:0/1812149678 shutdown_connections 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- 172.21.15.43:0/1812149678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4e690 0x563d93d50fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- 172.21.15.43:0/1812149678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4dd70 0x563d93d50a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- 172.21.15.43:0/1812149678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93d33180 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- 172.21.15.43:0/1812149678 >> 172.21.15.43:0/1812149678 conn(0x563d93d007e0 msgr2=0x563d93d64f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- 172.21.15.43:0/1812149678 shutdown_connections 2024-03-30T13:13:49.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- 172.21.15.43:0/1812149678 wait complete. 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- start start 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 Processor -- start 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93dbfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4dd70 0x563d93dc0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4e690 0x563d93d51ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d93b1a270 con 0x563d93d04830 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7efdd82025a0 con 0x563d93d4e690 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563d93c0eb00 con 0x563d93d4dd70 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efde0803000 10 client.0 _ll_register_callbacks cb 0x7efdd00c4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93dbfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93dbfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59784/0 (socket says 172.21.15.43:59784) 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1668989467 learned_addr learned my addr 172.21.15.43:0/1668989467 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd77fe640 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4e690 0x563d93d51ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.530+0000 7efdd67fc640 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4dd70 0x563d93dc0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4dd70 msgr2=0x563d93dc0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4dd70 0x563d93dc0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4e690 msgr2=0x563d93d51ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4e690 0x563d93d51ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd6ffd640 1 -- 172.21.15.43:0/1668989467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d93bd3f70 con 0x563d93d04830 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd6ffd640 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93dbfbd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efdd001ab50 tx=0x7efdd00a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba84b3f65275f544 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 1 -- 172.21.15.43:0/1668989467 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd00c6040 con 0x563d93d04830 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 1 -- 172.21.15.43:0/1668989467 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7efdd00bd070 con 0x563d93d04830 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd00039f0 con 0x563d93d04830 2024-03-30T13:13:49.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 1 -- 172.21.15.43:0/1668989467 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7efdd00e3050 con 0x563d93d04830 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 10 client.6066 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x563d93af9760 con 0x563d93d04830 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 1 -- 172.21.15.43:0/1668989467 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd00039f0 con 0x563d93d04830 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 1 -- 172.21.15.43:0/1668989467 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7efdd00d2040 con 0x563d93d04830 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 10 client.6066 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 10 client.6066 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 20 client.6066 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d52430 0x563d93d52810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 4 client.6066 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628325}] 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628325}) -- 0x563d93b5ffe0 con 0x563d93d52430 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd67fc640 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d52430 0x563d93d52810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 10 client.6066 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdd67fc640 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d52430 0x563d93d52810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2442498e66fa346 server_cookie=b9addff225de9f40 in_seq=0 out_seq=0 2024-03-30T13:13:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 1 -- 172.21.15.43:0/1668989467 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3821 (crc 0 0 0) 0x563d93b5ffe0 con 0x563d93d52430 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 10 client.6066 handle_command_reply: tid=0 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 client.6066 shutdown 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 msgr2=0x563d93dbfbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93dbfbd0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efdd001ab50 tx=0x7efdd00a8000 comp rx=0 tx=0).stop 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 shutdown_connections 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d93d52430 0x563d93d52810 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d93d4e690 0x563d93d51ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d93d4dd70 0x563d93dc0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efdc5ffb640 0 client.6066 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 --2- 172.21.15.43:0/1668989467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d93d04830 0x563d93dbfbd0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 >> 172.21.15.43:0/1668989467 conn(0x563d93d007e0 msgr2=0x563d93d65620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 shutdown_connections 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 1 -- 172.21.15.43:0/1668989467 wait complete. 2024-03-30T13:13:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efde0803000 20 client.6066 trim_cache size 0 max 16384 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr_trusted.c", 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628325, 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.967093+0000", 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.561229+0000", 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:49.544 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1094, 2024-03-30T13:13:49.545 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.561229+0000", 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:49.546 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1094, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.967093+0000" 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.547 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1094, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.967093+0000" 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 90, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 36, 2024-03-30T13:13:49.548 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:49.549 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:49.550 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.551 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:49.552 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:49.553 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:49.554 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:49.555 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdbc067fc0 msgr2=0x7efdbc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdbc067fc0 0x7efdbc06a480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efdd00ad000 tx=0x7efdd00a6000 comp rx=0 tx=0).stop 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd80cf6c0 msgr2=0x7efdd81592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd80cf6c0 0x7efdd81592b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7efdc000caa0 tx=0x7efdc000cf60 comp rx=0 tx=0).stop 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 shutdown_connections 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7efdbc067fc0 0x7efdbc06a480 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7efdd815e4c0 0x7efdd815b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdd80cf6c0 0x7efdd81592b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 --2- 172.21.15.43:0/1548719961 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7efdd806c070 0x7efdd81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 >> 172.21.15.43:0/1548719961 conn(0x7efdd80b6410 msgr2=0x7efdd80b8030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 shutdown_connections 2024-03-30T13:13:49.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.534+0000 7efddd681640 1 -- 172.21.15.43:0/1548719961 wait complete. 2024-03-30T13:13:49.558 DEBUG:tasks.cephfs.test_uninlining:getting inode info #13 of 23:1099511628323 2024-03-30T13:13:49.558 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628323 2024-03-30T13:13:49.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628323 2024-03-30T13:13:49.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.670+0000 7f9233a91640 1 Processor -- start 2024-03-30T13:13:49.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.670+0000 7f9233a91640 1 -- start start 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f9233a91640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f922c13f140 0x7f922c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f9233a91640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c06cd80 0x7f922c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f9233a91640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f922c06d710 con 0x7f922c15ff40 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f9233a91640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f922c06e320 con 0x7f922c13f140 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f9233a91640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f922c06d9f0 con 0x7f922c06cd80 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923202e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c06cd80 0x7f922c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923182d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f922c13f140 0x7f922c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923202e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c06cd80 0x7f922c0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:44062/0 (socket says 172.21.15.43:44062) 2024-03-30T13:13:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923202e640 1 -- 172.21.15.43:0/546494875 learned_addr learned my addr 172.21.15.43:0/546494875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923082b640 1 -- 172.21.15.43:0/546494875 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2942211236 0 0) 0x7f922c06d710 con 0x7f922c15ff40 2024-03-30T13:13:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923082b640 1 -- 172.21.15.43:0/546494875 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9218003530 con 0x7f922c15ff40 2024-03-30T13:13:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923202e640 1 -- 172.21.15.43:0/546494875 >> v2:172.21.15.113:3300/0 conn(0x7f922c13f140 msgr2=0x7f922c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923202e640 1 --2- 172.21.15.43:0/546494875 >> v2:172.21.15.113:3300/0 conn(0x7f922c13f140 0x7f922c1606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923202e640 1 -- 172.21.15.43:0/546494875 >> v1:172.21.15.113:6789/0 conn(0x7f922c15ff40 legacy=0x7f922c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923202e640 1 -- 172.21.15.43:0/546494875 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c06dcd0 con 0x7f922c06cd80 2024-03-30T13:13:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.674+0000 7f923182d640 1 --2- 172.21.15.43:0/546494875 >> v2:172.21.15.113:3300/0 conn(0x7f922c13f140 0x7f922c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:13:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f923202e640 1 --2- 172.21.15.43:0/546494875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c06cd80 0x7f922c0ce940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f922401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc153d9d5caec9db server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f923082b640 1 -- 172.21.15.43:0/546494875 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240a58f0 con 0x7f922c06cd80 2024-03-30T13:13:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f923082b640 1 -- 172.21.15.43:0/546494875 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92240a5a90 con 0x7f922c06cd80 2024-03-30T13:13:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f923082b640 1 -- 172.21.15.43:0/546494875 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240a5e60 con 0x7f922c06cd80 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 -- 172.21.15.43:0/546494875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c06cd80 msgr2=0x7f922c0ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 --2- 172.21.15.43:0/546494875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c06cd80 0x7f922c0ce940 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f922401ca00 comp rx=0 tx=0).stop 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 -- 172.21.15.43:0/546494875 shutdown_connections 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 --2- 172.21.15.43:0/546494875 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c06cd80 0x7f922c0ce940 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 --2- 172.21.15.43:0/546494875 >> v2:172.21.15.113:3300/0 conn(0x7f922c13f140 0x7f922c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 -- 172.21.15.43:0/546494875 >> 172.21.15.43:0/546494875 conn(0x7f922c0d00a0 msgr2=0x7f922c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 -- 172.21.15.43:0/546494875 shutdown_connections 2024-03-30T13:13:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 -- 172.21.15.43:0/546494875 wait complete. 2024-03-30T13:13:49.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 Processor -- start 2024-03-30T13:13:49.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.678+0000 7f9233a91640 1 -- start start 2024-03-30T13:13:49.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f922c06cd80 0x7f922c13d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c13f140 0x7f922c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c15ff40 0x7f922c163180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f922c0cf810 con 0x7f922c13f140 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f922c0cf690 con 0x7f922c06cd80 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f922c0cf990 con 0x7f922c15ff40 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c13f140 0x7f922c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c13f140 0x7f922c154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59800/0 (socket says 172.21.15.43:59800) 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 -- 172.21.15.43:0/762179357 learned_addr learned my addr 172.21.15.43:0/762179357 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923202e640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c15ff40 0x7f922c163180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923182d640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f922c06cd80 0x7f922c13d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 -- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c15ff40 msgr2=0x7f922c163180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c15ff40 0x7f922c163180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 -- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f922c06cd80 msgr2=0x7f922c13d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f922c06cd80 0x7f922c13d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 -- 172.21.15.43:0/762179357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922c153bb0 con 0x7f922c13f140 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923202e640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c15ff40 0x7f922c163180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f923102c640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c13f140 0x7f922c154c50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f921c00c3c0 tx=0x7f921c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f92227fc640 1 -- 172.21.15.43:0/762179357 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c017070 con 0x7f922c13f140 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f92227fc640 1 -- 172.21.15.43:0/762179357 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c0052d0 con 0x7f922c13f140 2024-03-30T13:13:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922c153e90 con 0x7f922c13f140 2024-03-30T13:13:49.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f92227fc640 1 -- 172.21.15.43:0/762179357 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c003840 con 0x7f922c13f140 2024-03-30T13:13:49.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922c154450 con 0x7f922c13f140 2024-03-30T13:13:49.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f92227fc640 1 -- 172.21.15.43:0/762179357 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f921c01a070 con 0x7f922c13f140 2024-03-30T13:13:49.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.682+0000 7f92227fc640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9210067fc0 0x7f921006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f92227fc640 1 -- 172.21.15.43:0/762179357 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f921c085960 con 0x7f922c13f140 2024-03-30T13:13:49.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923182d640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9210067fc0 0x7f921006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 Processor -- start 2024-03-30T13:13:49.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- start start 2024-03-30T13:13:49.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923182d640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9210067fc0 0x7f921006a480 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9228009580 tx=0x7f9228009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e3d70 0x555fba2e4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba29a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba29a830 0x555fba29ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555fba0e8aa0 con 0x555fba2e4690 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555fba06faf0 con 0x555fba2e3d70 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x555fba0e4a50 con 0x555fba29a830 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba29a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba29a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59812/0 (socket says 172.21.15.43:59812) 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 -- 172.21.15.43:0/3780518814 learned_addr learned my addr 172.21.15.43:0/3780518814 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923182d640 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e3d70 0x555fba2e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923202e640 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba29a830 0x555fba29ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 -- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba29a830 msgr2=0x555fba29ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba29a830 0x555fba29ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 -- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e3d70 msgr2=0x555fba2e4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e3d70 0x555fba2e4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 -- 172.21.15.43:0/3780518814 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fba06dd00 con 0x555fba2e4690 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f923102c640 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba29a2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f921c05df00 tx=0x7f921c055a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad32bf5e889b6baa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f920f7fe640 1 -- 172.21.15.43:0/3780518814 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c053c00 con 0x555fba2e4690 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f920f7fe640 1 -- 172.21.15.43:0/3780518814 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c051f70 con 0x555fba2e4690 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f920f7fe640 1 -- 172.21.15.43:0/3780518814 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c056d80 con 0x555fba2e4690 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- 172.21.15.43:0/3780518814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 msgr2=0x555fba29a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba29a2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f921c05df00 tx=0x7f921c055a40 comp rx=0 tx=0).stop 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- 172.21.15.43:0/3780518814 shutdown_connections 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba29a830 0x555fba29ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba29a2f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 --2- 172.21.15.43:0/3780518814 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e3d70 0x555fba2e4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- 172.21.15.43:0/3780518814 >> 172.21.15.43:0/3780518814 conn(0x555fba2967e0 msgr2=0x555fba298bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- 172.21.15.43:0/3780518814 shutdown_connections 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- 172.21.15.43:0/3780518814 wait complete. 2024-03-30T13:13:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 -- start start 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.686+0000 7f9236c13000 1 Processor -- start 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba29a830 0x555fba29fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba2c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e4690 0x555fba2c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555fba0b0270 con 0x555fba29a830 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x555fba0a66a0 con 0x555fba2e4690 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x555fba080880 con 0x555fba2e3d70 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 10 client.0 _ll_register_callbacks cb 0x555fba285490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba29a830 0x555fba29fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba29a830 0x555fba29fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59814/0 (socket says 172.21.15.43:59814) 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 -- 172.21.15.43:0/4258938889 learned_addr learned my addr 172.21.15.43:0/4258938889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923102c640 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba2c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923202e640 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e4690 0x555fba2c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 -- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 msgr2=0x555fba2c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba2c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 -- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e4690 msgr2=0x555fba2c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e4690 0x555fba2c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 -- 172.21.15.43:0/4258938889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fba0ad730 con 0x555fba29a830 2024-03-30T13:13:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923182d640 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba29a830 0x555fba29fc80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9228003620 tx=0x7f92280046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d9a7321f7a3392 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f920cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f920cff9640 1 -- 172.21.15.43:0/4258938889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228026070 con 0x555fba29a830 2024-03-30T13:13:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f920cff9640 1 -- 172.21.15.43:0/4258938889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f9228014440 con 0x555fba29a830 2024-03-30T13:13:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f921c051f70 con 0x555fba29a830 2024-03-30T13:13:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f920cff9640 1 -- 172.21.15.43:0/4258938889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9228014680 con 0x555fba29a830 2024-03-30T13:13:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 10 client.6075 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x555fba0a0ca0 con 0x555fba29a830 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f920cff9640 1 -- 172.21.15.43:0/4258938889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f921c051f70 con 0x555fba29a830 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f920cff9640 1 -- 172.21.15.43:0/4258938889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f9228034080 con 0x555fba29a830 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 10 client.6075 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 10 client.6075 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 20 client.6075 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba2eb5d0 0x555fba2eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 4 client.6075 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fba083c90 con 0x555fba2eb5d0 2024-03-30T13:13:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923102c640 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba2eb5d0 0x555fba2eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f923102c640 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba2eb5d0 0x555fba2eb9b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae28077cd36f854 server_cookie=cc5ecf38977531f4 in_seq=0 out_seq=0 2024-03-30T13:13:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.690+0000 7f920cff9640 10 client.6075 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f920cff9640 1 -- 172.21.15.43:0/4258938889 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x555fba083c90 con 0x555fba2eb5d0 2024-03-30T13:13:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f920cff9640 10 client.6075 handle_command_reply: tid=0 2024-03-30T13:13:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 client.6075 shutdown 2024-03-30T13:13:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba29a830 msgr2=0x555fba29fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba29a830 0x555fba29fc80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9228003620 tx=0x7f92280046d0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 shutdown_connections 2024-03-30T13:13:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba2eb5d0 0x555fba2eb9b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba2e4690 0x555fba2c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba2c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f920cff9640 0 client.6075 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 --2- 172.21.15.43:0/4258938889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba29a830 0x555fba29fc80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 >> 172.21.15.43:0/4258938889 conn(0x555fba2967e0 msgr2=0x555fba299460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 shutdown_connections 2024-03-30T13:13:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 1 -- 172.21.15.43:0/4258938889 wait complete. 2024-03-30T13:13:49.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.694+0000 7f9236c13000 20 client.6075 trim_cache size 0 max 16384 2024-03-30T13:13:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.706+0000 7f9236c13000 1 Processor -- start 2024-03-30T13:13:49.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.706+0000 7f9236c13000 1 -- start start 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba298bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba299110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f922c203de0 con 0x555fba2e4690 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f922c202e90 con 0x555fba29a830 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f922c161c30 con 0x555fba2e3d70 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba299110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba299110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59818/0 (socket says 172.21.15.43:59818) 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 -- 172.21.15.43:0/209856456 learned_addr learned my addr 172.21.15.43:0/209856456 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923182d640 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba298bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 -- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 msgr2=0x555fba298bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba298bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 -- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 msgr2=0x555fba2c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 -- 172.21.15.43:0/209856456 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fba06fd00 con 0x555fba2e4690 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba299110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f9224004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb5553a0d0b2517 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f920effd640 1 -- 172.21.15.43:0/209856456 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240a58f0 con 0x555fba2e4690 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f920effd640 1 -- 172.21.15.43:0/209856456 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92240a5a90 con 0x555fba2e4690 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f920effd640 1 -- 172.21.15.43:0/209856456 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92240c1690 con 0x555fba2e4690 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- 172.21.15.43:0/209856456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 msgr2=0x555fba299110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba299110 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f9224009e50 tx=0x7f9224004ac0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- 172.21.15.43:0/209856456 shutdown_connections 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e4690 0x555fba299110 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e3d70 0x555fba298bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- 172.21.15.43:0/209856456 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- 172.21.15.43:0/209856456 >> 172.21.15.43:0/209856456 conn(0x555fba2967e0 msgr2=0x555fba2faf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- 172.21.15.43:0/209856456 shutdown_connections 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- 172.21.15.43:0/209856456 wait complete. 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- start start 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 Processor -- start 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2e7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e3d70 0x555fba2e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e4690 0x555fba2e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x555fba0e7db0 con 0x555fba2e3d70 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f922c162280 con 0x555fba29a830 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x555fba1a4b00 con 0x555fba2e4690 2024-03-30T13:13:49.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f9236c13000 10 client.0 _ll_register_callbacks cb 0x7f9228011a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e3d70 0x555fba2e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e3d70 0x555fba2e8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59826/0 (socket says 172.21.15.43:59826) 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 -- 172.21.15.43:0/1013855600 learned_addr learned my addr 172.21.15.43:0/1013855600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923182d640 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2e7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923202e640 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e4690 0x555fba2e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 -- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e4690 msgr2=0x555fba2e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e4690 0x555fba2e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 -- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 msgr2=0x555fba2e7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2e7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.710+0000 7f923102c640 1 -- 172.21.15.43:0/1013855600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fba169f70 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f923102c640 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e3d70 0x555fba2e8010 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f921c055600 tx=0x7f921c014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c91d32461352232 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 1 -- 172.21.15.43:0/1013855600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c051ba0 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 1 -- 172.21.15.43:0/1013855600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f921c051f70 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92240a5a90 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 1 -- 172.21.15.43:0/1013855600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f921c056b80 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 10 client.6081 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x555fba08f760 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 1 -- 172.21.15.43:0/1013855600 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92240a5a90 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 1 -- 172.21.15.43:0/1013855600 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f921c07d130 con 0x555fba2e3d70 2024-03-30T13:13:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 10 client.6081 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 10 client.6081 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 20 client.6081 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba352700 0x555fba2e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 4 client.6081 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628323}] 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628323}) -- 0x555fba0f5fe0 con 0x555fba352700 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f923182d640 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba352700 0x555fba2e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 10 client.6081 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f923182d640 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba352700 0x555fba2e8a90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec01f733ba14939b server_cookie=7efddc131d0162a5 in_seq=0 out_seq=0 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 1 -- 172.21.15.43:0/1013855600 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3813 (crc 0 0 0) 0x555fba0f5fe0 con 0x555fba352700 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 10 client.6081 handle_command_reply: tid=0 2024-03-30T13:13:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 client.6081 shutdown 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e3d70 msgr2=0x555fba2e8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e3d70 0x555fba2e8010 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f921c055600 tx=0x7f921c014070 comp rx=0 tx=0).stop 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 shutdown_connections 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x555fba352700 0x555fba2e8a90 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x555fba2e4690 0x555fba2e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555fba2e3d70 0x555fba2e8010 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 --2- 172.21.15.43:0/1013855600 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x555fba29a830 0x555fba2e7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 >> 172.21.15.43:0/1013855600 conn(0x555fba2967e0 msgr2=0x555fba2fb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f920cff9640 0 client.6081 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 shutdown_connections 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 1 -- 172.21.15.43:0/1013855600 wait complete. 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9236c13000 20 client.6081 trim_cache size 0 max 16384 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr.h", 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628323, 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.968801+0000", 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.552317+0000", 2024-03-30T13:13:49.727 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.728 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3340, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.552317+0000", 2024-03-30T13:13:49.729 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:49.730 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3340, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.968801+0000" 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3340, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.731 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.968801+0000" 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 92, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 32, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.732 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:49.733 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:49.734 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.736 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.737 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:49.738 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:49.739 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:49.739 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:49.739 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.739 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:49.739 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:49.739 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:49.740 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:49.740 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:49.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9210067fc0 msgr2=0x7f921006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9210067fc0 0x7f921006a480 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9228009580 tx=0x7f9228009190 comp rx=0 tx=0).stop 2024-03-30T13:13:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c13f140 msgr2=0x7f922c154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c13f140 0x7f922c154c50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f921c00c3c0 tx=0x7f921c00c880 comp rx=0 tx=0).stop 2024-03-30T13:13:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 shutdown_connections 2024-03-30T13:13:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9210067fc0 0x7f921006a480 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f922c15ff40 0x7f922c163180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922c13f140 0x7f922c154c50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 --2- 172.21.15.43:0/762179357 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f922c06cd80 0x7f922c13d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 >> 172.21.15.43:0/762179357 conn(0x7f922c0d00a0 msgr2=0x7f922c15f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 shutdown_connections 2024-03-30T13:13:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.714+0000 7f9233a91640 1 -- 172.21.15.43:0/762179357 wait complete. 2024-03-30T13:13:49.743 DEBUG:tasks.cephfs.test_uninlining:getting inode info #14 of 23:1099511628328 2024-03-30T13:13:49.743 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628328 2024-03-30T13:13:49.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628328 2024-03-30T13:13:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bf26d640 1 Processor -- start 2024-03-30T13:13:49.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bf26d640 1 -- start start 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bf26d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 0x7fd6b806bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bf26d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b815f7d0 con 0x7fd6b80cb210 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bf26d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b806c390 con 0x7fd6b815ec30 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bf26d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b806c670 con 0x7fd6b80cb5f0 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd6b80cb210 0x7fd6b806b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47256/0 (socket says 172.21.15.43:47256) 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 -- 172.21.15.43:0/1855508549 learned_addr learned my addr 172.21.15.43:0/1855508549 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bc808640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 0x7fd6b806bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3052684677 0 0) 0x7fd6b815f7d0 con 0x7fd6b80cb210 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a4003530 con 0x7fd6b80cb210 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 45597452 0 0) 0x7fd6a4003530 con 0x7fd6b80cb210 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 msgr2=0x7fd6b806bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 0x7fd6b806bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 >> v1:172.21.15.113:6789/0 conn(0x7fd6b815ec30 legacy=0x7fd6b815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b806c950 con 0x7fd6b80cb210 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fd6b4001680 con 0x7fd6b80cb210 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 >> v1:172.21.15.43:6789/0 conn(0x7fd6b80cb210 legacy=0x7fd6b806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a40039f0 0x7fd6a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 0x7fd6a4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b806c670 con 0x7fd6a40039f0 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a400bb00 con 0x7fd6a4004810 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a400bd00 con 0x7fd6b80cb5f0 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a40039f0 0x7fd6a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bc808640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 0x7fd6a4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 -- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 msgr2=0x7fd6a4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 0x7fd6a4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 -- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a4004810 msgr2=0x7fd6a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd009640 1 -- 172.21.15.43:0/1855508549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a4003530 con 0x7fd6a40039f0 2024-03-30T13:13:49.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6b80cb5f0 0x7fd6a4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.862+0000 7fd6bc808640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6a4004810 0x7fd6a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a40039f0 0x7fd6a4003e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c240 tx=0x7fd6b401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9899a199ba1cec97 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4007d20 con 0x7fd6a40039f0 2024-03-30T13:13:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b40a9070 con 0x7fd6a40039f0 2024-03-30T13:13:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6affff640 1 -- 172.21.15.43:0/1855508549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4005d40 con 0x7fd6a40039f0 2024-03-30T13:13:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1855508549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a40039f0 msgr2=0x7fd6a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 --2- 172.21.15.43:0/1855508549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6a40039f0 0x7fd6a4003e10 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd6b400c240 tx=0x7fd6b401cc50 comp rx=0 tx=0).stop 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bd80a640 1 -- 172.21.15.43:0/1855508549 reap_dead start 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1855508549 shutdown_connections 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1855508549 >> 172.21.15.43:0/1855508549 conn(0x7fd6b80cd1d0 msgr2=0x7fd6b80cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1855508549 shutdown_connections 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1855508549 wait complete. 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 Processor -- start 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- start start 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6a40039f0 0x7fd6b80b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8151f70 0x7fd6b8146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8156db0 0x7fd6b8157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b806fa10 con 0x7fd6b8156db0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b806f890 con 0x7fd6b8151f70 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bf26d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b806fb90 con 0x7fd6a40039f0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bd80a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8156db0 0x7fd6b8157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bd80a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8156db0 0x7fd6b8157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59840/0 (socket says 172.21.15.43:59840) 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bd80a640 1 -- 172.21.15.43:0/1055644881 learned_addr learned my addr 172.21.15.43:0/1055644881 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.866+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6a40039f0 0x7fd6b80b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bc808640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8151f70 0x7fd6b8146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd80a640 1 -- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6a40039f0 msgr2=0x7fd6b80b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6a40039f0 0x7fd6b80b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd80a640 1 -- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8151f70 msgr2=0x7fd6b8146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8151f70 0x7fd6b8146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd80a640 1 -- 172.21.15.43:0/1055644881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b8152610 con 0x7fd6b8156db0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8156db0 0x7fd6b8157230 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8068120 tx=0x7fd6b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6adffb640 1 -- 172.21.15.43:0/1055644881 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0006d60 con 0x7fd6b8156db0 2024-03-30T13:13:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6adffb640 1 -- 172.21.15.43:0/1055644881 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0003980 con 0x7fd6b8156db0 2024-03-30T13:13:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6adffb640 1 -- 172.21.15.43:0/1055644881 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0003db0 con 0x7fd6b8156db0 2024-03-30T13:13:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b80ada20 con 0x7fd6b8156db0 2024-03-30T13:13:49.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b8147010 con 0x7fd6b8156db0 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6adffb640 1 -- 172.21.15.43:0/1055644881 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd6b001b020 con 0x7fd6b8156db0 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6adffb640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd69c067fc0 0x7fd69c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6adffb640 1 -- 172.21.15.43:0/1055644881 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fd6b00860b0 con 0x7fd6b8156db0 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd69c067fc0 0x7fd69c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd69c067fc0 0x7fd69c06a480 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40ab000 tx=0x7fd6b40b1040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6c23ef000 1 Processor -- start 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.870+0000 7fd6c23ef000 1 -- start start 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d14d70 0x564262d15150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d15690 0x564262ccb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262ccbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564262b19aa0 con 0x564262d14d70 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564262aa0af0 con 0x564262ccb830 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x564262b15a50 con 0x564262d15690 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d14d70 0x564262d15150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d14d70 0x564262d15150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59846/0 (socket says 172.21.15.43:59846) 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 -- 172.21.15.43:0/2666701197 learned_addr learned my addr 172.21.15.43:0/2666701197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262ccbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bc808640 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d15690 0x564262ccb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 -- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d15690 msgr2=0x564262ccb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d15690 0x564262ccb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 -- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 msgr2=0x564262ccbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262ccbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 -- 172.21.15.43:0/2666701197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564262a9ed00 con 0x564262d14d70 2024-03-30T13:13:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d14d70 0x564262d15150 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4098560 tx=0x7fd6b8158ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6724afe79969ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd69affd640 1 -- 172.21.15.43:0/2666701197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4004d40 con 0x564262d14d70 2024-03-30T13:13:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd69affd640 1 -- 172.21.15.43:0/2666701197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b40a9070 con 0x564262d14d70 2024-03-30T13:13:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd69affd640 1 -- 172.21.15.43:0/2666701197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4007080 con 0x564262d14d70 2024-03-30T13:13:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2666701197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d14d70 msgr2=0x564262d15150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d14d70 0x564262d15150 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4098560 tx=0x7fd6b8158ea0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2666701197 shutdown_connections 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262ccbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d15690 0x564262ccb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2666701197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d14d70 0x564262d15150 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2666701197 >> 172.21.15.43:0/2666701197 conn(0x564262cc77e0 msgr2=0x564262cc9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2666701197 shutdown_connections 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2666701197 wait complete. 2024-03-30T13:13:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- start start 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 Processor -- start 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262cd0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262cfa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564262ae1270 con 0x564262ccb830 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x564262ad76a0 con 0x564262d15690 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x564262ab1880 con 0x564262d14d70 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6c23ef000 10 client.0 _ll_register_callbacks cb 0x564262cb6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262cd0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262cd0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59862/0 (socket says 172.21.15.43:59862) 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bd009640 1 -- 172.21.15.43:0/1911796918 learned_addr learned my addr 172.21.15.43:0/1911796918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.874+0000 7fd6bc808640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262cfa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd009640 1 -- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 msgr2=0x564262cf9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd009640 1 -- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 msgr2=0x564262cfa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262cfa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd009640 1 -- 172.21.15.43:0/1911796918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564262ade730 con 0x564262ccb830 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262cfa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bd009640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262cd0c40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401c6c0 tx=0x7fd6b40199f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fcf03c4a0dc5ca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 1 -- 172.21.15.43:0/1911796918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4007440 con 0x564262ccb830 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b40a9070 con 0x564262ccb830 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 1 -- 172.21.15.43:0/1911796918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fd6b40075e0 con 0x564262ccb830 2024-03-30T13:13:49.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 1 -- 172.21.15.43:0/1911796918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd6b40d2050 con 0x564262ccb830 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 10 client.6093 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x564262ad1ca0 con 0x564262ccb830 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 1 -- 172.21.15.43:0/1911796918 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b40a9070 con 0x564262ccb830 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 1 -- 172.21.15.43:0/1911796918 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fd6b40d5020 con 0x564262ccb830 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 10 client.6093 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 10 client.6093 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 20 client.6093 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d1c5d0 0x564262d1c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 4 client.6093 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564262ab4c90 con 0x564262d1c5d0 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bc808640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d1c5d0 0x564262d1c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 10 client.6093 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6bc808640 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d1c5d0 0x564262d1c9e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df16c1a2a1add771 server_cookie=711bad52dbb34f42 in_seq=0 out_seq=0 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 1 -- 172.21.15.43:0/1911796918 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x564262ab4c90 con 0x564262d1c5d0 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 10 client.6093 handle_command_reply: tid=0 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 client.6093 shutdown 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 msgr2=0x564262cd0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262cd0c40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401c6c0 tx=0x7fd6b40199f0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 shutdown_connections 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d1c5d0 0x564262d1c9e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262cfa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/1911796918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262cd0c40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd693fff640 0 client.6093 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 >> 172.21.15.43:0/1911796918 conn(0x564262cc77e0 msgr2=0x564262cca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 shutdown_connections 2024-03-30T13:13:49.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 1 -- 172.21.15.43:0/1911796918 wait complete. 2024-03-30T13:13:49.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.878+0000 7fd6c23ef000 20 client.6093 trim_cache size 0 max 16384 2024-03-30T13:13:49.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 Processor -- start 2024-03-30T13:13:49.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 -- start start 2024-03-30T13:13:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262cf8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d15690 0x564262cf9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8204850 con 0x564262d15690 2024-03-30T13:13:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b82038b0 con 0x564262ccb830 2024-03-30T13:13:49.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b8202c10 con 0x564262d14d70 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d15690 0x564262cf9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d15690 0x564262cf9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59866/0 (socket says 172.21.15.43:59866) 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 -- 172.21.15.43:0/79925248 learned_addr learned my addr 172.21.15.43:0/79925248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bc808640 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd009640 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262cf8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 -- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 msgr2=0x564262cf9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 -- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 msgr2=0x564262cf8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262cf8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 -- 172.21.15.43:0/79925248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564262aa0d00 con 0x564262d15690 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd009640 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262cf8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d15690 0x564262cf9a90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0001540 tx=0x7fd6b00567f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33435ec1caf761f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd69a7fc640 1 -- 172.21.15.43:0/79925248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b005ec10 con 0x564262d15690 2024-03-30T13:13:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd69a7fc640 1 -- 172.21.15.43:0/79925248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0054330 con 0x564262d15690 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.894+0000 7fd69a7fc640 1 -- 172.21.15.43:0/79925248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b006e1d0 con 0x564262d15690 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/79925248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d15690 msgr2=0x564262cf9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d15690 0x564262cf9a90 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0001540 tx=0x7fd6b00567f0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/79925248 shutdown_connections 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262d15690 0x564262cf9a90 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262cf9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/79925248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262ccb830 0x564262cf8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/79925248 >> 172.21.15.43:0/79925248 conn(0x564262cc77e0 msgr2=0x564262d2b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/79925248 shutdown_connections 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/79925248 wait complete. 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- start start 2024-03-30T13:13:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 Processor -- start 2024-03-30T13:13:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262d86980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262d86ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262d18ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x564262ab1880 con 0x564262ccb830 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b8203260 con 0x564262d15690 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x564262b15a50 con 0x564262d14d70 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 10 client.0 _ll_register_callbacks cb 0x7fd6a4005770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262d86980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262d86980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59882/0 (socket says 172.21.15.43:59882) 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 -- 172.21.15.43:0/2026556157 learned_addr learned my addr 172.21.15.43:0/2026556157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bc808640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262d86ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262d18ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 -- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 msgr2=0x564262d86ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262d86ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 -- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 msgr2=0x564262d18ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262d18ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 -- 172.21.15.43:0/2026556157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564262b9af70 con 0x564262ccb830 2024-03-30T13:13:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd80a640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262d18ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bd009640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262d86980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401b4d0 tx=0x7fd6b40a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b0df291152d5c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd693fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd693fff640 1 -- 172.21.15.43:0/2026556157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40db060 con 0x564262ccb830 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd693fff640 1 -- 172.21.15.43:0/2026556157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fd6b40af470 con 0x564262ccb830 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6b0054330 con 0x564262ccb830 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd693fff640 1 -- 172.21.15.43:0/2026556157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd6b4001650 con 0x564262ccb830 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 10 client.6099 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x564262ac0760 con 0x564262ccb830 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd693fff640 1 -- 172.21.15.43:0/2026556157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0054330 con 0x564262ccb830 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd693fff640 1 -- 172.21.15.43:0/2026556157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fd6b40d5070 con 0x564262ccb830 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 10 client.6099 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 10 client.6099 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 20 client.6099 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d19490 0x564262d19870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 4 client.6099 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628328}] 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628328}) -- 0x564262b26fe0 con 0x564262d19490 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.898+0000 7fd6bc808640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d19490 0x564262d19870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bc808640 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d19490 0x564262d19870 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81d86e9b49535a1 server_cookie=680efc8dbf931c7 in_seq=0 out_seq=0 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd693fff640 10 client.6099 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd693fff640 1 -- 172.21.15.43:0/2026556157 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3813 (crc 0 0 0) 0x564262b26fe0 con 0x564262d19490 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd693fff640 10 client.6099 handle_command_reply: tid=0 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 client.6099 shutdown 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 msgr2=0x564262d86980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262d86980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd6b401b4d0 tx=0x7fd6b40a7070 comp rx=0 tx=0).stop 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 shutdown_connections 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x564262d19490 0x564262d19870 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x564262d15690 0x564262d18ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x564262d14d70 0x564262d86ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd693fff640 0 client.6099 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 --2- 172.21.15.43:0/2026556157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564262ccb830 0x564262d86980 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 >> 172.21.15.43:0/2026556157 conn(0x564262cc77e0 msgr2=0x564262cca460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 shutdown_connections 2024-03-30T13:13:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 1 -- 172.21.15.43:0/2026556157 wait complete. 2024-03-30T13:13:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6c23ef000 20 client.6099 trim_cache size 0 max 16384 2024-03-30T13:13:49.913 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/Kconfig", 2024-03-30T13:13:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628328, 2024-03-30T13:13:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.975224+0000", 2024-03-30T13:13:49.913 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.890446+0000", 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:49.914 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:49.915 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 228, 2024-03-30T13:13:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:49.916 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.890446+0000", 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:49.917 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:49.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 228, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.975224+0000" 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 228, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:49.919 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.975224+0000" 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 88, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 2, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:49.920 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:49.921 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:49.922 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:49.923 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:49.924 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:49.925 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:49.926 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:49.927 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:49.928 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:49.928 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:49.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd69c067fc0 msgr2=0x7fd69c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd69c067fc0 0x7fd69c06a480 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40ab000 tx=0x7fd6b40b1040 comp rx=0 tx=0).stop 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8156db0 msgr2=0x7fd6b8157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8156db0 0x7fd6b8157230 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8068120 tx=0x7fd6b000dc00 comp rx=0 tx=0).stop 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 shutdown_connections 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd69c067fc0 0x7fd69c06a480 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b8156db0 0x7fd6b8157230 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd6b8151f70 0x7fd6b8146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 --2- 172.21.15.43:0/1055644881 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd6a40039f0 0x7fd6b80b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 >> 172.21.15.43:0/1055644881 conn(0x7fd6b80cd1d0 msgr2=0x7fd6b8154b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 shutdown_connections 2024-03-30T13:13:49.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:49.902+0000 7fd6bf26d640 1 -- 172.21.15.43:0/1055644881 wait complete. 2024-03-30T13:13:49.931 DEBUG:tasks.cephfs.test_uninlining:getting inode info #15 of 23:1099511628341 2024-03-30T13:13:49.931 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628341 2024-03-30T13:13:49.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628341 2024-03-30T13:13:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06ca366640 1 Processor -- start 2024-03-30T13:13:50.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06ca366640 1 -- start start 2024-03-30T13:13:50.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06ca366640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 0x7f06c40b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06ca366640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06c4070640 0x7f06c40ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06ca366640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06c40ce4d0 con 0x7f06c415f880 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06ca366640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f06c4070ab0 con 0x7f06c4070640 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06ca366640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f06c40cc7f0 con 0x7f06c406f090 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f06c415f880 0x7f06c406e980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47272/0 (socket says 172.21.15.43:47272) 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 -- 172.21.15.43:0/2494413052 learned_addr learned my addr 172.21.15.43:0/2494413052 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c8903640 1 --2- 172.21.15.43:0/2494413052 >> v2:172.21.15.113:3300/0 conn(0x7f06c4070640 0x7f06c40ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c37fe640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 0x7f06c40b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2338699794 0 0) 0x7f06c40ce4d0 con 0x7f06c415f880 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06a4003530 con 0x7f06c415f880 2024-03-30T13:13:50.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1832663254 0 0) 0x7f06a4003530 con 0x7f06c415f880 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 msgr2=0x7f06c40b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 0x7f06c40b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 >> v2:172.21.15.113:3300/0 conn(0x7f06c4070640 msgr2=0x7f06c40ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 --2- 172.21.15.43:0/2494413052 >> v2:172.21.15.113:3300/0 conn(0x7f06c4070640 0x7f06c40ce9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c40ccad0 con 0x7f06c415f880 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f06bc001a00 con 0x7f06c415f880 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 >> v1:172.21.15.43:6789/0 conn(0x7f06c415f880 legacy=0x7f06c406e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a40039c0 0x7f06a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a4004810 0x7f06a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 0x7f06a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4070ab0 con 0x7f06a40039c0 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06c40cc7f0 con 0x7f06a4004810 2024-03-30T13:13:50.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f06a400ba50 con 0x7f06c406f090 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a40039c0 0x7f06a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c37fe640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a4004810 0x7f06a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c8903640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 0x7f06a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 -- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 msgr2=0x7f06a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c406f090 0x7f06a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 -- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a4004810 msgr2=0x7f06a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a4004810 0x7f06a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.050+0000 7f06c3fff640 1 -- 172.21.15.43:0/2494413052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a4003530 con 0x7f06a40039c0 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a40039c0 0x7f06a4003de0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00a910 tx=0x7f06bc01d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b678df2a9942812 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00a4e0 con 0x7f06a40039c0 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06bc0ad630 con 0x7f06a40039c0 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06c2ffd640 1 -- 172.21.15.43:0/2494413052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0ada20 con 0x7f06a40039c0 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 -- 172.21.15.43:0/2494413052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a40039c0 msgr2=0x7f06a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 --2- 172.21.15.43:0/2494413052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06a40039c0 0x7f06a4003de0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00a910 tx=0x7f06bc01d440 comp rx=0 tx=0).stop 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06c8903640 1 -- 172.21.15.43:0/2494413052 reap_dead start 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 -- 172.21.15.43:0/2494413052 shutdown_connections 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 -- 172.21.15.43:0/2494413052 >> 172.21.15.43:0/2494413052 conn(0x7f06c40b3cf0 msgr2=0x7f06c40b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 -- 172.21.15.43:0/2494413052 shutdown_connections 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 -- 172.21.15.43:0/2494413052 wait complete. 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 Processor -- start 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.054+0000 7f06ca366640 1 -- start start 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a40039c0 0x7f06c415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c415c8b0 0x7f06c4155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c40d0000 0x7f06c40d0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41609d0 con 0x7f06c415c8b0 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4160850 con 0x7f06a40039c0 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f06c41671d0 con 0x7f06c40d0000 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c415c8b0 0x7f06c4155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c415c8b0 0x7f06c4155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59896/0 (socket says 172.21.15.43:59896) 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 -- 172.21.15.43:0/3242434750 learned_addr learned my addr 172.21.15.43:0/3242434750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c3fff640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a40039c0 0x7f06c415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c8903640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c40d0000 0x7f06c40d0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 -- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c40d0000 msgr2=0x7f06c40d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c40d0000 0x7f06c40d0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 -- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a40039c0 msgr2=0x7f06c415d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a40039c0 0x7f06c415d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 -- 172.21.15.43:0/3242434750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c415a1e0 con 0x7f06c415c8b0 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c37fe640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c415c8b0 0x7f06c4155ff0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f06ac011560 tx=0x7f06ac011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c0ff9640 1 -- 172.21.15.43:0/3242434750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac015930 con 0x7f06c415c8b0 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c0ff9640 1 -- 172.21.15.43:0/3242434750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac015ad0 con 0x7f06c415c8b0 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c4159c20 con 0x7f06c415c8b0 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c0ff9640 1 -- 172.21.15.43:0/3242434750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00f460 con 0x7f06c415c8b0 2024-03-30T13:13:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c4156ac0 con 0x7f06c415c8b0 2024-03-30T13:13:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.058+0000 7f06c0ff9640 1 -- 172.21.15.43:0/3242434750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f06ac00f600 con 0x7f06c415c8b0 2024-03-30T13:13:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c0ff9640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06a8067ff0 0x7f06a806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c0ff9640 1 -- 172.21.15.43:0/3242434750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f06ac086fd0 con 0x7f06c415c8b0 2024-03-30T13:13:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c3fff640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06a8067ff0 0x7f06a806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 Processor -- start 2024-03-30T13:13:50.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- start start 2024-03-30T13:13:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c3fff640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06a8067ff0 0x7f06a806a4b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0ad000 tx=0x7f06bc001370 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d135150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d0eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56218cf39aa0 con 0x56218d135690 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56218cec0af0 con 0x56218d134d70 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56218cf35a50 con 0x56218d0eb830 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d0eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d0eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59900/0 (socket says 172.21.15.43:59900) 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 -- 172.21.15.43:0/4136214616 learned_addr learned my addr 172.21.15.43:0/4136214616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c3fff640 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d135150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c8903640 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 -- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 msgr2=0x56218d0ebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0ebc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 -- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 msgr2=0x56218d135150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d135150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 -- 172.21.15.43:0/4136214616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218cebed00 con 0x56218d135690 2024-03-30T13:13:50.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06c37fe640 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d0eb2f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0852a0 tx=0x7f06ac0575b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1626ee8253890cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06b9ffb640 1 -- 172.21.15.43:0/4136214616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac05b260 con 0x56218d135690 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06b9ffb640 1 -- 172.21.15.43:0/4136214616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac083070 con 0x56218d135690 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06b9ffb640 1 -- 172.21.15.43:0/4136214616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac05fa90 con 0x56218d135690 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- 172.21.15.43:0/4136214616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 msgr2=0x56218d0eb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d0eb2f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f06ac0852a0 tx=0x7f06ac0575b0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- 172.21.15.43:0/4136214616 shutdown_connections 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d0eb2f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/4136214616 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d135150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- 172.21.15.43:0/4136214616 >> 172.21.15.43:0/4136214616 conn(0x56218d0e77e0 msgr2=0x56218d0e9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- 172.21.15.43:0/4136214616 shutdown_connections 2024-03-30T13:13:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- 172.21.15.43:0/4136214616 wait complete. 2024-03-30T13:13:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.062+0000 7f06cd4e8000 1 -- start start 2024-03-30T13:13:50.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 Processor -- start 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d119da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d11a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56218cf01270 con 0x56218d135690 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56218cef76a0 con 0x56218d134d70 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56218ced1880 con 0x56218d0eb830 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 10 client.0 _ll_register_callbacks cb 0x56218d0d6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d11a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d11a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59908/0 (socket says 172.21.15.43:59908) 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 -- 172.21.15.43:0/695075329 learned_addr learned my addr 172.21.15.43:0/695075329 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c37fe640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d119da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c3fff640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 -- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 msgr2=0x56218d0f0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0f0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 -- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 msgr2=0x56218d119da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d119da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 -- 172.21.15.43:0/695075329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218cefe730 con 0x56218d135690 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c3fff640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c8903640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d11a2e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f06c40d1410 tx=0x7f06b4015360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75969cd81002b49 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06b2ffd640 1 -- 172.21.15.43:0/695075329 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004030 con 0x56218d135690 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06b2ffd640 1 -- 172.21.15.43:0/695075329 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f06b40041d0 con 0x56218d135690 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06ac083070 con 0x56218d135690 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06b2ffd640 1 -- 172.21.15.43:0/695075329 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f06b4027050 con 0x56218d135690 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 10 client.6111 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56218cef1ca0 con 0x56218d135690 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06b2ffd640 1 -- 172.21.15.43:0/695075329 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac083070 con 0x56218d135690 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06b2ffd640 1 -- 172.21.15.43:0/695075329 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f06b402a020 con 0x56218d135690 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 10 client.6111 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 10 client.6111 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 20 client.6111 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d13c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 4 client.6111 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56218ced4c90 con 0x56218d13c5d0 2024-03-30T13:13:50.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c3fff640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d13c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06b2ffd640 10 client.6111 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.066+0000 7f06c3fff640 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d13c9e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54cd0722edec56b3 server_cookie=71ff56539cfd6bd0 in_seq=0 out_seq=0 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06b2ffd640 1 -- 172.21.15.43:0/695075329 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x56218ced4c90 con 0x56218d13c5d0 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06b2ffd640 10 client.6111 handle_command_reply: tid=0 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 client.6111 shutdown 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 msgr2=0x56218d11a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d11a2e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f06c40d1410 tx=0x7f06b4015360 comp rx=0 tx=0).stop 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 shutdown_connections 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d13c9e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d11a2e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06b2ffd640 0 client.6111 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d134d70 0x56218d119da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/695075329 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d0eb830 0x56218d0f0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 >> 172.21.15.43:0/695075329 conn(0x56218d0e77e0 msgr2=0x56218d0ea460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 shutdown_connections 2024-03-30T13:13:50.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 1 -- 172.21.15.43:0/695075329 wait complete. 2024-03-30T13:13:50.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.070+0000 7f06cd4e8000 20 client.6111 trim_cache size 0 max 16384 2024-03-30T13:13:50.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 Processor -- start 2024-03-30T13:13:50.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- start start 2024-03-30T13:13:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d118d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d134d70 0x56218d119290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d135690 0x56218d119ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06c42028d0 con 0x56218d134d70 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4201930 con 0x56218d0eb830 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f06c4200cc0 con 0x56218d135690 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d134d70 0x56218d119290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d134d70 0x56218d119290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59916/0 (socket says 172.21.15.43:59916) 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 -- 172.21.15.43:0/2449929052 learned_addr learned my addr 172.21.15.43:0/2449929052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d118d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c8903640 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d135690 0x56218d119ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 -- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d135690 msgr2=0x56218d119ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d135690 0x56218d119ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 -- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 msgr2=0x56218d118d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d118d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 -- 172.21.15.43:0/2449929052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218cec0d00 con 0x56218d134d70 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c8903640 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d135690 0x56218d119ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d118d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06c37fe640 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d134d70 0x56218d119290 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009fd0 tx=0x7f06ac06e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce1f89f7b8bdbe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06b97fa640 1 -- 172.21.15.43:0/2449929052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac075070 con 0x56218d134d70 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06b97fa640 1 -- 172.21.15.43:0/2449929052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac07a070 con 0x56218d134d70 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06b97fa640 1 -- 172.21.15.43:0/2449929052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac05b260 con 0x56218d134d70 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2449929052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d134d70 msgr2=0x56218d119290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d134d70 0x56218d119290 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009fd0 tx=0x7f06ac06e000 comp rx=0 tx=0).stop 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2449929052 shutdown_connections 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d135690 0x56218d119ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d134d70 0x56218d119290 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2449929052 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d118d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2449929052 >> 172.21.15.43:0/2449929052 conn(0x56218d0e77e0 msgr2=0x56218d14b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2449929052 shutdown_connections 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2449929052 wait complete. 2024-03-30T13:13:50.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 -- start start 2024-03-30T13:13:50.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 Processor -- start 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d1ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d134d70 0x56218d1adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.086+0000 7f06cd4e8000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d1b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56218d14cfb0 con 0x56218d135690 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06c42012e0 con 0x56218d0eb830 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56218d1187a0 con 0x56218d134d70 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 10 client.0 _ll_register_callbacks cb 0x7f06b40095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d1b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d1b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59922/0 (socket says 172.21.15.43:59922) 2024-03-30T13:13:50.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 -- 172.21.15.43:0/2210652014 learned_addr learned my addr 172.21.15.43:0/2210652014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c37fe640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d134d70 0x56218d1adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d1ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 -- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d134d70 msgr2=0x56218d1adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d134d70 0x56218d1adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 -- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 msgr2=0x56218d1ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d1ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 -- 172.21.15.43:0/2210652014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218cfbaf70 con 0x56218d135690 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d1ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c8903640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d1b00b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005f00 tx=0x7f06b4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ed4f39fe24defd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 1 -- 172.21.15.43:0/2210652014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4024070 con 0x56218d135690 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 1 -- 172.21.15.43:0/2210652014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f06b4006250 con 0x56218d135690 2024-03-30T13:13:50.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06ac07a070 con 0x56218d135690 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 1 -- 172.21.15.43:0/2210652014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f06b403b050 con 0x56218d135690 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 10 client.6117 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56218cee0760 con 0x56218d135690 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 1 -- 172.21.15.43:0/2210652014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac07a070 con 0x56218d135690 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 1 -- 172.21.15.43:0/2210652014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f06b403e020 con 0x56218d135690 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 10 client.6117 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 10 client.6117 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 20 client.6117 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d1b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 4 client.6117 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628341}] 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628341}) -- 0x56218cf46fe0 con 0x56218d13c5d0 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d1b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06c3fff640 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d1b2600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1e5f57bf94052c server_cookie=3d0f0ca359d213b1 in_seq=0 out_seq=0 2024-03-30T13:13:50.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 10 client.6117 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 1 -- 172.21.15.43:0/2210652014 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3826 (crc 0 0 0) 0x56218cf46fe0 con 0x56218d13c5d0 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 10 client.6117 handle_command_reply: tid=0 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 client.6117 shutdown 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 msgr2=0x56218d1b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d1b00b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005f00 tx=0x7f06b4004740 comp rx=0 tx=0).stop 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 shutdown_connections 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56218d13c5d0 0x56218d1b2600 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56218d135690 0x56218d1b00b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56218d134d70 0x56218d1adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 --2- 172.21.15.43:0/2210652014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56218d0eb830 0x56218d1ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 >> 172.21.15.43:0/2210652014 conn(0x56218d0e77e0 msgr2=0x56218d14bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06b2ffd640 0 client.6117 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 shutdown_connections 2024-03-30T13:13:50.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.090+0000 7f06cd4e8000 1 -- 172.21.15.43:0/2210652014 wait complete. 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06cd4e8000 20 client.6117 trim_cache size 0 max 16384 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debug.h", 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628341, 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.976283+0000", 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.995809+0000", 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:50.101 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.102 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 2579, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.995809+0000", 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 4, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:50.103 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2579, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.976283+0000" 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.104 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2579, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.976283+0000" 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 94, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 28, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:50.105 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:50.106 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:50.107 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.108 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.109 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:50.110 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:50.111 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:50.112 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06a8067ff0 msgr2=0x7f06a806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06a8067ff0 0x7f06a806a4b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0ad000 tx=0x7f06bc001370 comp rx=0 tx=0).stop 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c415c8b0 msgr2=0x7f06c4155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c415c8b0 0x7f06c4155ff0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f06ac011560 tx=0x7f06ac011a20 comp rx=0 tx=0).stop 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 shutdown_connections 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06a8067ff0 0x7f06a806a4b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06c40d0000 0x7f06c40d0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c415c8b0 0x7f06c4155ff0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 --2- 172.21.15.43:0/3242434750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06a40039c0 0x7f06c415d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 >> 172.21.15.43:0/3242434750 conn(0x7f06c40b3cf0 msgr2=0x7f06c415ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 shutdown_connections 2024-03-30T13:13:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.094+0000 7f06ca366640 1 -- 172.21.15.43:0/3242434750 wait complete. 2024-03-30T13:13:50.115 DEBUG:tasks.cephfs.test_uninlining:getting inode info #16 of 23:1099511628348 2024-03-30T13:13:50.115 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628348 2024-03-30T13:13:50.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628348 2024-03-30T13:13:50.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f1960d640 1 Processor -- start 2024-03-30T13:13:50.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f1960d640 1 -- start start 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f1960d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7f14147d50 0x7f7f1406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f1960d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f140b4110 con 0x7f7f1406bba0 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f1960d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f140b43f0 con 0x7f7f1406c780 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f1960d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7f14148130 con 0x7f7f14147d50 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f127fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7f1406bba0 0x7f7f1406c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47284/0 (socket says 172.21.15.43:47284) 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f127fc640 1 -- 172.21.15.43:0/840322796 learned_addr learned my addr 172.21.15.43:0/840322796 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/840322796 >> v2:172.21.15.113:3300/0 conn(0x7f7f14147d50 0x7f7f1406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745178597 0 0) 0x7f7f140b4110 con 0x7f7f1406bba0 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7efc003530 con 0x7f7f1406bba0 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3479571077 0 0) 0x7f7efc003530 con 0x7f7f1406bba0 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 >> v2:172.21.15.113:3300/0 conn(0x7f7f14147d50 msgr2=0x7f7f1406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 --2- 172.21.15.43:0/840322796 >> v2:172.21.15.113:3300/0 conn(0x7f7f14147d50 0x7f7f1406b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 >> v1:172.21.15.113:6789/0 conn(0x7f7f1406c780 legacy=0x7f7f140b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f140b46d0 con 0x7f7f1406bba0 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f7f040016a0 con 0x7f7f1406bba0 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 >> v1:172.21.15.43:6789/0 conn(0x7f7f1406bba0 legacy=0x7f7f1406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc0039c0 0x7f7efc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc004810 0x7f7efc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7efc0051d0 0x7f7efc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc00c140 con 0x7f7efc0039c0 2024-03-30T13:13:50.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7efc00c2f0 con 0x7f7efc004810 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7efc00c4f0 con 0x7f7efc0051d0 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc0039c0 0x7f7efc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f137fe640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7efc0051d0 0x7f7efc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f127fc640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc004810 0x7f7efc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f12ffd640 1 -- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7efc0051d0 msgr2=0x7f7efc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7efc0051d0 0x7f7efc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f12ffd640 1 -- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc004810 msgr2=0x7f7efc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc004810 0x7f7efc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f12ffd640 1 -- 172.21.15.43:0/840322796 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7efc003530 con 0x7f7efc0039c0 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f137fe640 1 -- 172.21.15.43:0/840322796 reap_dead start 2024-03-30T13:13:50.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.230+0000 7f7f127fc640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc004810 0x7f7efc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc0039c0 0x7f7efc003de0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009ae0 tx=0x7f7f0801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54e13851c921ebb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080096b0 con 0x7f7efc0039c0 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f080a6470 con 0x7f7efc0039c0 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f11ffb640 1 -- 172.21.15.43:0/840322796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080a6860 con 0x7f7efc0039c0 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f1960d640 1 -- 172.21.15.43:0/840322796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc0039c0 msgr2=0x7f7efc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f1960d640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc0039c0 0x7f7efc003de0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009ae0 tx=0x7f7f0801ca90 comp rx=0 tx=0).stop 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f1960d640 1 -- 172.21.15.43:0/840322796 shutdown_connections 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f1960d640 1 --2- 172.21.15.43:0/840322796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7efc0039c0 0x7f7efc003de0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f1960d640 1 -- 172.21.15.43:0/840322796 >> 172.21.15.43:0/840322796 conn(0x7f7f140d1070 msgr2=0x7f7f140d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f1960d640 1 -- 172.21.15.43:0/840322796 shutdown_connections 2024-03-30T13:13:50.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.234+0000 7f7f1960d640 1 -- 172.21.15.43:0/840322796 wait complete. 2024-03-30T13:13:50.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 Processor -- start 2024-03-30T13:13:50.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 -- start start 2024-03-30T13:13:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc0039c0 0x7f7f14139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f1413f170 0x7f7f1413a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f1413fb90 0x7f7f14140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1406c600 con 0x7f7f1413fb90 2024-03-30T13:13:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1406c480 con 0x7f7efc0039c0 2024-03-30T13:13:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7f14162800 con 0x7f7f1413f170 2024-03-30T13:13:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f1413fb90 0x7f7f14140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f1413fb90 0x7f7f14140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59942/0 (socket says 172.21.15.43:59942) 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 -- 172.21.15.43:0/1923186847 learned_addr learned my addr 172.21.15.43:0/1923186847 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f127fc640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f1413f170 0x7f7f1413a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc0039c0 0x7f7f14139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 -- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f1413f170 msgr2=0x7f7f1413a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f1413f170 0x7f7f1413a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 -- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc0039c0 msgr2=0x7f7f14139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc0039c0 0x7f7f14139d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 -- 172.21.15.43:0/1923186847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1413a260 con 0x7f7f1413fb90 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f137fe640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f1413fb90 0x7f7f14140010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7f14073770 tx=0x7f7f0c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7efb7fe640 1 -- 172.21.15.43:0/1923186847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c003980 con 0x7f7f1413fb90 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7efb7fe640 1 -- 172.21.15.43:0/1923186847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0c018070 con 0x7f7f1413fb90 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7efb7fe640 1 -- 172.21.15.43:0/1923186847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c003d80 con 0x7f7f1413fb90 2024-03-30T13:13:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1413b110 con 0x7f7f1413fb90 2024-03-30T13:13:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.238+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1413b6d0 con 0x7f7f1413fb90 2024-03-30T13:13:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7efb7fe640 1 -- 172.21.15.43:0/1923186847 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7f0c0050d0 con 0x7f7f1413fb90 2024-03-30T13:13:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7efb7fe640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7f00067ff0 0x7f7f0006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7efb7fe640 1 -- 172.21.15.43:0/1923186847 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f7f0c086320 con 0x7f7f1413fb90 2024-03-30T13:13:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7f00067ff0 0x7f7f0006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 Processor -- start 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7f00067ff0 0x7f7f0006a4b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f080ad000 tx=0x7f7f080b2040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 -- start start 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x563408667c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5634084b5aa0 con 0x5634086b1690 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56340843caf0 con 0x563408667830 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5634084b1a50 con 0x5634086b0d70 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59958/0 (socket says 172.21.15.43:59958) 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 -- 172.21.15.43:0/612876871 learned_addr learned my addr 172.21.15.43:0/612876871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f137fe640 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x563408667c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 -- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 msgr2=0x5634086b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 -- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 msgr2=0x563408667c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x563408667c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 -- 172.21.15.43:0/612876871 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340843ad00 con 0x5634086b1690 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.242+0000 7f7f127fc640 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086672f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7f1406bfc0 tx=0x7f7f04010240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff6249e11e5cee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef8ff9640 1 -- 172.21.15.43:0/612876871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0401a070 con 0x5634086b1690 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef8ff9640 1 -- 172.21.15.43:0/612876871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f040133c0 con 0x5634086b1690 2024-03-30T13:13:50.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef8ff9640 1 -- 172.21.15.43:0/612876871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04013770 con 0x5634086b1690 2024-03-30T13:13:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/612876871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 msgr2=0x5634086672f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086672f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7f1406bfc0 tx=0x7f7f04010240 comp rx=0 tx=0).stop 2024-03-30T13:13:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/612876871 shutdown_connections 2024-03-30T13:13:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x563408667c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086672f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/612876871 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/612876871 >> 172.21.15.43:0/612876871 conn(0x5634086637e0 msgr2=0x563408665bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/612876871 shutdown_connections 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/612876871 wait complete. 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- start start 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 Processor -- start 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x56340866ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x563408695da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56340847d270 con 0x5634086b1690 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5634084736a0 con 0x563408667830 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56340844d880 con 0x5634086b0d70 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 10 client.0 _ll_register_callbacks cb 0x563408652490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59968/0 (socket says 172.21.15.43:59968) 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 -- 172.21.15.43:0/955578371 learned_addr learned my addr 172.21.15.43:0/955578371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x56340866ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f127fc640 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x563408695da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 -- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 msgr2=0x563408695da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x563408695da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 -- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 msgr2=0x56340866ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x56340866ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 -- 172.21.15.43:0/955578371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56340847a730 con 0x5634086b1690 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f137fe640 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086962e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0845f0 tx=0x7f7f0c0568a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650fdb7f53e91f9e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/955578371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c082060 con 0x5634086b1690 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/955578371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f7f0c05a160 con 0x5634086b1690 2024-03-30T13:13:50.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f040133c0 con 0x5634086b1690 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/955578371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7f0c073050 con 0x5634086b1690 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 10 client.6129 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56340846dca0 con 0x5634086b1690 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/955578371 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f040133c0 con 0x5634086b1690 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/955578371 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f7f0c052750 con 0x5634086b1690 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 10 client.6129 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 10 client.6129 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 20 client.6129 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b85d0 0x5634086b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 4 client.6129 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.246+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563408450c90 con 0x5634086b85d0 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b85d0 0x5634086b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b85d0 0x5634086b89b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d21ec4a47b9d32a server_cookie=5faa4e37cd45c0c in_seq=0 out_seq=0 2024-03-30T13:13:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7ef1ffb640 10 client.6129 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/955578371 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x563408450c90 con 0x5634086b85d0 2024-03-30T13:13:50.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7ef1ffb640 10 client.6129 handle_command_reply: tid=0 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 client.6129 shutdown 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 msgr2=0x5634086962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086962e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c0845f0 tx=0x7f7f0c0568a0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 shutdown_connections 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b85d0 0x5634086b89b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b1690 0x5634086962e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x563408695da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/955578371 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563408667830 0x56340866ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 >> 172.21.15.43:0/955578371 conn(0x5634086637e0 msgr2=0x563408666460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7ef1ffb640 0 client.6129 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 shutdown_connections 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 1 -- 172.21.15.43:0/955578371 wait complete. 2024-03-30T13:13:50.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.250+0000 7f7f1c78f000 20 client.6129 trim_cache size 0 max 16384 2024-03-30T13:13:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 Processor -- start 2024-03-30T13:13:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- start start 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563408667830 0x563408665bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b0d70 0x563408666110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x563408666650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14204cd0 con 0x5634086b0d70 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f14203cf0 con 0x5634086b1690 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7f14203050 con 0x563408667830 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b0d70 0x563408666110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b0d70 0x563408666110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59974/0 (socket says 172.21.15.43:59974) 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 -- 172.21.15.43:0/836980896 learned_addr learned my addr 172.21.15.43:0/836980896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563408667830 0x563408665bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 -- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563408667830 msgr2=0x563408665bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563408667830 0x563408665bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f137fe640 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x563408666650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 -- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 msgr2=0x563408666650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x563408666650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 -- 172.21.15.43:0/836980896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56340843cd00 con 0x5634086b0d70 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563408667830 0x563408665bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f137fe640 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x563408666650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f127fc640 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b0d70 0x563408666110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7f04010280 tx=0x7f7f04013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ccb2c7aaa8c360 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7ef3fff640 1 -- 172.21.15.43:0/836980896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0401a040 con 0x5634086b0d70 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7ef3fff640 1 -- 172.21.15.43:0/836980896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0401b040 con 0x5634086b0d70 2024-03-30T13:13:50.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7ef3fff640 1 -- 172.21.15.43:0/836980896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04014660 con 0x5634086b0d70 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- 172.21.15.43:0/836980896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b0d70 msgr2=0x563408666110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b0d70 0x563408666110 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7f04010280 tx=0x7f7f04013000 comp rx=0 tx=0).stop 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- 172.21.15.43:0/836980896 shutdown_connections 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x563408666650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5634086b0d70 0x563408666110 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/836980896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563408667830 0x563408665bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- 172.21.15.43:0/836980896 >> 172.21.15.43:0/836980896 conn(0x5634086637e0 msgr2=0x5634086c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- 172.21.15.43:0/836980896 shutdown_connections 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- 172.21.15.43:0/836980896 wait complete. 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 -- start start 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.266+0000 7f7f1c78f000 1 Processor -- start 2024-03-30T13:13:50.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563408667830 0x563408721bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x5634086b5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56340847d270 con 0x563408667830 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7f142036a0 con 0x5634086b1690 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5634086c8fb0 con 0x5634086b0d70 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 10 client.0 _ll_register_callbacks cb 0x7f7f0c007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563408667830 0x563408721bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563408667830 0x563408721bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59990/0 (socket says 172.21.15.43:59990) 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 -- 172.21.15.43:0/3801686893 learned_addr learned my addr 172.21.15.43:0/3801686893 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f127fc640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f137fe640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x5634086b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 -- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 msgr2=0x5634086b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 -- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 msgr2=0x5634086b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x5634086b5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 -- 172.21.15.43:0/3801686893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563408536f70 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f127fc640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f12ffd640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563408667830 0x563408721bf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7f08098280 tx=0x7f7f0801b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9f151d83cbe9aa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/3801686893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080d0060 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/3801686893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f7f080ca3b0 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f0401b040 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/3801686893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7f080d6050 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 10 client.6135 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56340845c760 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/3801686893 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0401b040 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/3801686893 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f7f080d9020 con 0x563408667830 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 10 client.6135 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 10 client.6135 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 20 client.6135 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b5550 0x5634086b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 4 client.6135 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628348}] 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628348}) -- 0x5634084c2fe0 con 0x5634086b5550 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f127fc640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b5550 0x5634086b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 10 client.6135 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f127fc640 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b5550 0x5634086b5960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863c9fced34f326d server_cookie=1bbf8559069974d7 in_seq=0 out_seq=0 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 1 -- 172.21.15.43:0/3801686893 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3820 (crc 0 0 0) 0x5634084c2fe0 con 0x5634086b5550 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 10 client.6135 handle_command_reply: tid=0 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 client.6135 shutdown 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563408667830 msgr2=0x563408721bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563408667830 0x563408721bf0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f7f08098280 tx=0x7f7f0801b2d0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 shutdown_connections 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5634086b5550 0x5634086b5960 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5634086b1690 0x5634086b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5634086b0d70 0x5634086b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 --2- 172.21.15.43:0/3801686893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563408667830 0x563408721bf0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 >> 172.21.15.43:0/3801686893 conn(0x5634086637e0 msgr2=0x5634086c7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7ef1ffb640 0 client.6135 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.270+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 shutdown_connections 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1c78f000 1 -- 172.21.15.43:0/3801686893 wait complete. 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1c78f000 20 client.6135 trim_cache size 0 max 16384 2024-03-30T13:13:50.282 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-sysfs.h", 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628348, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.978208+0000", 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:13.040011+0000", 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:50.283 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 77, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:50.284 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:13.040011+0000", 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:50.285 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 77, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.978208+0000" 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 77, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.286 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.978208+0000" 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 92, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 42, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:50.287 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:50.288 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:50.289 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.290 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.291 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.292 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:50.293 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:50.294 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:50.294 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:50.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7f00067ff0 msgr2=0x7f7f0006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7f00067ff0 0x7f7f0006a4b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f080ad000 tx=0x7f7f080b2040 comp rx=0 tx=0).stop 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f1413fb90 msgr2=0x7f7f14140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f1413fb90 0x7f7f14140010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7f14073770 tx=0x7f7f0c00ec10 comp rx=0 tx=0).stop 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 shutdown_connections 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7f00067ff0 0x7f7f0006a4b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f1413fb90 0x7f7f14140010 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7f1413f170 0x7f7f1413a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 --2- 172.21.15.43:0/1923186847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7efc0039c0 0x7f7f14139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 >> 172.21.15.43:0/1923186847 conn(0x7f7f140d1070 msgr2=0x7f7f14146790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 shutdown_connections 2024-03-30T13:13:50.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.274+0000 7f7f1960d640 1 -- 172.21.15.43:0/1923186847 wait complete. 2024-03-30T13:13:50.297 DEBUG:tasks.cephfs.test_uninlining:getting inode info #17 of 23:1099511628331 2024-03-30T13:13:50.297 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628331 2024-03-30T13:13:50.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628331 2024-03-30T13:13:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45be3e5640 1 Processor -- start 2024-03-30T13:13:50.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45be3e5640 1 -- start start 2024-03-30T13:13:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45be3e5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f45b815f880 0x7f45b80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45be3e5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45b8070640 0x7f45b806e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45be3e5640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b80ce4d0 con 0x7f45b806eec0 2024-03-30T13:13:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45be3e5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f45b8070ad0 con 0x7f45b8070640 2024-03-30T13:13:50.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45be3e5640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f45b806f310 con 0x7f45b815f880 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45b8070640 0x7f45b806e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45b8070640 0x7f45b806e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59998/0 (socket says 172.21.15.43:59998) 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b77fe640 1 -- 172.21.15.43:0/431056300 learned_addr learned my addr 172.21.15.43:0/431056300 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b7fff640 1 --2- 172.21.15.43:0/431056300 >> v2:172.21.15.113:3300/0 conn(0x7f45b815f880 0x7f45b80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b77fe640 1 -- 172.21.15.43:0/431056300 >> v2:172.21.15.113:3300/0 conn(0x7f45b815f880 msgr2=0x7f45b80b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b77fe640 1 --2- 172.21.15.43:0/431056300 >> v2:172.21.15.113:3300/0 conn(0x7f45b815f880 0x7f45b80b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b77fe640 1 -- 172.21.15.43:0/431056300 >> v1:172.21.15.113:6789/0 conn(0x7f45b806eec0 legacy=0x7f45b80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b77fe640 1 -- 172.21.15.43:0/431056300 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b80cc7f0 con 0x7f45b8070640 2024-03-30T13:13:50.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.410+0000 7f45b7fff640 1 --2- 172.21.15.43:0/431056300 >> v2:172.21.15.113:3300/0 conn(0x7f45b815f880 0x7f45b80b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45b77fe640 1 --2- 172.21.15.43:0/431056300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8070640 0x7f45b806e980 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f45b80b4860 tx=0x7f45ac01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26638df40aa47d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45b6ffd640 1 -- 172.21.15.43:0/431056300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0b0070 con 0x7f45b8070640 2024-03-30T13:13:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45b6ffd640 1 -- 172.21.15.43:0/431056300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ac0ab600 con 0x7f45b8070640 2024-03-30T13:13:50.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45b6ffd640 1 -- 172.21.15.43:0/431056300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0ab9d0 con 0x7f45b8070640 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- 172.21.15.43:0/431056300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8070640 msgr2=0x7f45b806e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 --2- 172.21.15.43:0/431056300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8070640 0x7f45b806e980 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f45b80b4860 tx=0x7f45ac01c400 comp rx=0 tx=0).stop 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- 172.21.15.43:0/431056300 shutdown_connections 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 --2- 172.21.15.43:0/431056300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b8070640 0x7f45b806e980 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 --2- 172.21.15.43:0/431056300 >> v2:172.21.15.113:3300/0 conn(0x7f45b815f880 0x7f45b80b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- 172.21.15.43:0/431056300 >> 172.21.15.43:0/431056300 conn(0x7f45b80b3cf0 msgr2=0x7f45b80b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- 172.21.15.43:0/431056300 shutdown_connections 2024-03-30T13:13:50.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- 172.21.15.43:0/431056300 wait complete. 2024-03-30T13:13:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 Processor -- start 2024-03-30T13:13:50.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- start start 2024-03-30T13:13:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b806eec0 0x7f45b80b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b8070640 0x7f45b814a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b815f880 0x7f45b814a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45b80ccf30 con 0x7f45b815f880 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45b80ccdb0 con 0x7f45b8070640 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45be3e5640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f45b80cf4d0 con 0x7f45b806eec0 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b815f880 0x7f45b814a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b815f880 0x7f45b814a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60010/0 (socket says 172.21.15.43:60010) 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 -- 172.21.15.43:0/4154281908 learned_addr learned my addr 172.21.15.43:0/4154281908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45b77fe640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b8070640 0x7f45b814a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45b7fff640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b806eec0 0x7f45b80b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 -- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b806eec0 msgr2=0x7f45b80b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b806eec0 0x7f45b80b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 -- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b8070640 msgr2=0x7f45b814a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b8070640 0x7f45b814a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 -- 172.21.15.43:0/4154281908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b80c5970 con 0x7f45b815f880 2024-03-30T13:13:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45b77fe640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b8070640 0x7f45b814a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.414+0000 7f45bc982640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b815f880 0x7f45b814a6a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c730 tx=0x7f45b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b4ff9640 1 -- 172.21.15.43:0/4154281908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0003b80 con 0x7f45b815f880 2024-03-30T13:13:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b4ff9640 1 -- 172.21.15.43:0/4154281908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b0003d20 con 0x7f45b815f880 2024-03-30T13:13:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b80c5c50 con 0x7f45b815f880 2024-03-30T13:13:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b4ff9640 1 -- 172.21.15.43:0/4154281908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b000aa30 con 0x7f45b815f880 2024-03-30T13:13:50.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b80b6690 con 0x7f45b815f880 2024-03-30T13:13:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b4ff9640 1 -- 172.21.15.43:0/4154281908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f45b000abd0 con 0x7f45b815f880 2024-03-30T13:13:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b4ff9640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45a4067fc0 0x7f45a406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b4ff9640 1 -- 172.21.15.43:0/4154281908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f45b0086590 con 0x7f45b815f880 2024-03-30T13:13:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b7fff640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45a4067fc0 0x7f45a406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 Processor -- start 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 -- start start 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45b7fff640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45a4067fc0 0x7f45a406a480 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45b80ca2d0 tx=0x7f45a8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e51d70 0x560450e52150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e52690 0x560450e082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e08830 0x560450e08c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560450c56aa0 con 0x560450e08830 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560450bddaf0 con 0x560450e51d70 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.418+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560450c52a50 con 0x560450e52690 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e08830 0x560450e08c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e08830 0x560450e08c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60016/0 (socket says 172.21.15.43:60016) 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 -- 172.21.15.43:0/3131856111 learned_addr learned my addr 172.21.15.43:0/3131856111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b7fff640 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e51d70 0x560450e52150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e52690 0x560450e082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 -- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e52690 msgr2=0x560450e082f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e52690 0x560450e082f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 -- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e51d70 msgr2=0x560450e52150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e51d70 0x560450e52150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 -- 172.21.15.43:0/3131856111 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560450bdbd00 con 0x560450e08830 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b7fff640 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e51d70 0x560450e52150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e52690 0x560450e082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e08830 0x560450e08c10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f45b005eb40 tx=0x7f45b000ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474ad32fe5b3eb9a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f459dffb640 1 -- 172.21.15.43:0/3131856111 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0056490 con 0x560450e08830 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f459dffb640 1 -- 172.21.15.43:0/3131856111 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b005a840 con 0x560450e08830 2024-03-30T13:13:50.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f459dffb640 1 -- 172.21.15.43:0/3131856111 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b005f070 con 0x560450e08830 2024-03-30T13:13:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- 172.21.15.43:0/3131856111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e08830 msgr2=0x560450e08c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e08830 0x560450e08c10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f45b005eb40 tx=0x7f45b000ce40 comp rx=0 tx=0).stop 2024-03-30T13:13:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- 172.21.15.43:0/3131856111 shutdown_connections 2024-03-30T13:13:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e08830 0x560450e08c10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e52690 0x560450e082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 --2- 172.21.15.43:0/3131856111 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e51d70 0x560450e52150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- 172.21.15.43:0/3131856111 >> 172.21.15.43:0/3131856111 conn(0x560450e047e0 msgr2=0x560450e06bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- 172.21.15.43:0/3131856111 shutdown_connections 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- 172.21.15.43:0/3131856111 wait complete. 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- start start 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 Processor -- start 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e08830 0x560450e0dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e51d70 0x560450e36da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e52690 0x560450e372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560450c1e270 con 0x560450e51d70 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560450c146a0 con 0x560450e52690 2024-03-30T13:13:50.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560450bee880 con 0x560450e08830 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 10 client.0 _ll_register_callbacks cb 0x560450df3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e51d70 0x560450e36da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e51d70 0x560450e36da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60022/0 (socket says 172.21.15.43:60022) 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 -- 172.21.15.43:0/1093026663 learned_addr learned my addr 172.21.15.43:0/1093026663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e08830 0x560450e0dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45bc982640 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e52690 0x560450e372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 -- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e08830 msgr2=0x560450e0dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e08830 0x560450e0dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 -- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e52690 msgr2=0x560450e372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e52690 0x560450e372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 -- 172.21.15.43:0/1093026663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560450c1b730 con 0x560450e51d70 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45b77fe640 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e51d70 0x560450e36da0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f45ac097d20 tx=0x7f45ac01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af21d22f6e57d110 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f4596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f4596ffd640 1 -- 172.21.15.43:0/1093026663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0b0040 con 0x560450e51d70 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f4596ffd640 1 -- 172.21.15.43:0/1093026663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f45ac0abcd0 con 0x560450e51d70 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.422+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b005a840 con 0x560450e51d70 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f4596ffd640 1 -- 172.21.15.43:0/1093026663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f45ac0c3050 con 0x560450e51d70 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 10 client.6147 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560450c0eca0 con 0x560450e51d70 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f4596ffd640 1 -- 172.21.15.43:0/1093026663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b005a840 con 0x560450e51d70 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f4596ffd640 1 -- 172.21.15.43:0/1093026663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f45ac0c6020 con 0x560450e51d70 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 10 client.6147 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 10 client.6147 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 20 client.6147 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e595d0 0x560450e599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 4 client.6147 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560450bf1c90 con 0x560450e595d0 2024-03-30T13:13:50.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e595d0 0x560450e599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e595d0 0x560450e599e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e38d72fd28c2487 server_cookie=6ae220269bd2b482 in_seq=0 out_seq=0 2024-03-30T13:13:50.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f4596ffd640 10 client.6147 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f4596ffd640 1 -- 172.21.15.43:0/1093026663 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x560450bf1c90 con 0x560450e595d0 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f4596ffd640 10 client.6147 handle_command_reply: tid=0 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 client.6147 shutdown 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e51d70 msgr2=0x560450e36da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e51d70 0x560450e36da0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f45ac097d20 tx=0x7f45ac01ac40 comp rx=0 tx=0).stop 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 shutdown_connections 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e595d0 0x560450e599e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e52690 0x560450e372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e51d70 0x560450e36da0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f4596ffd640 0 client.6147 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 --2- 172.21.15.43:0/1093026663 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e08830 0x560450e0dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 >> 172.21.15.43:0/1093026663 conn(0x560450e047e0 msgr2=0x560450e07460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 shutdown_connections 2024-03-30T13:13:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 1 -- 172.21.15.43:0/1093026663 wait complete. 2024-03-30T13:13:50.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.426+0000 7f45c1567000 20 client.6147 trim_cache size 0 max 16384 2024-03-30T13:13:50.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 Processor -- start 2024-03-30T13:13:50.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- start start 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450e37180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450e06bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e07110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f45b82021f0 con 0x560450e52690 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45b82012e0 con 0x560450e08830 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f45b80c9ed0 con 0x560450e51d70 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e07110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e07110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60028/0 (socket says 172.21.15.43:60028) 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 -- 172.21.15.43:0/2904828901 learned_addr learned my addr 172.21.15.43:0/2904828901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45b77fe640 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450e06bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 -- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 msgr2=0x560450e06bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45b7fff640 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450e37180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450e06bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 -- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 msgr2=0x560450e37180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450e37180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 -- 172.21.15.43:0/2904828901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560450bddd00 con 0x560450e52690 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45bc982640 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e07110 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f45b0056420 tx=0x7f45b0086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2515c29edae206a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f459d7fa640 1 -- 172.21.15.43:0/2904828901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0056490 con 0x560450e52690 2024-03-30T13:13:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f459d7fa640 1 -- 172.21.15.43:0/2904828901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b005f070 con 0x560450e52690 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f459d7fa640 1 -- 172.21.15.43:0/2904828901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0052b00 con 0x560450e52690 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- 172.21.15.43:0/2904828901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 msgr2=0x560450e07110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e07110 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f45b0056420 tx=0x7f45b0086050 comp rx=0 tx=0).stop 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- 172.21.15.43:0/2904828901 shutdown_connections 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e07110 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450e06bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 --2- 172.21.15.43:0/2904828901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450e37180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- 172.21.15.43:0/2904828901 >> 172.21.15.43:0/2904828901 conn(0x560450e047e0 msgr2=0x560450e68f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- 172.21.15.43:0/2904828901 shutdown_connections 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.442+0000 7f45c1567000 1 -- 172.21.15.43:0/2904828901 wait complete. 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- start start 2024-03-30T13:13:50.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 Processor -- start 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450ec4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450ec4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e55ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560450c1e270 con 0x560450e52690 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f45b81f6590 con 0x560450e08830 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560450c55db0 con 0x560450e51d70 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 10 client.0 _ll_register_callbacks cb 0x7f45ac0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e55ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e55ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60038/0 (socket says 172.21.15.43:60038) 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 -- 172.21.15.43:0/1609085966 learned_addr learned my addr 172.21.15.43:0/1609085966 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450ec4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45b77fe640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450ec4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 -- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 msgr2=0x560450ec4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450ec4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 -- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 msgr2=0x560450ec4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450ec4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 -- 172.21.15.43:0/1609085966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560450cd7f70 con 0x560450e52690 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450ec4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45bc982640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e55ad0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f45b00868b0 tx=0x7f45b001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a979e94a5fb2789 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 1 -- 172.21.15.43:0/1609085966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b0084040 con 0x560450e52690 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45b005f070 con 0x560450e52690 2024-03-30T13:13:50.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 1 -- 172.21.15.43:0/1609085966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f45b005ede0 con 0x560450e52690 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 1 -- 172.21.15.43:0/1609085966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f45b007e0d0 con 0x560450e52690 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 10 client.6153 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560450bfd760 con 0x560450e52690 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 1 -- 172.21.15.43:0/1609085966 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b005f070 con 0x560450e52690 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 1 -- 172.21.15.43:0/1609085966 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f45b007f070 con 0x560450e52690 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 10 client.6153 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 10 client.6153 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 20 client.6153 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e56490 0x560450e568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 4 client.6153 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628331}] 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628331}) -- 0x560450c63fe0 con 0x560450e56490 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e56490 0x560450e568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 10 client.6153 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e56490 0x560450e568a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29447d92fa0f331f server_cookie=b55ac679dfd0947b in_seq=0 out_seq=0 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 1 -- 172.21.15.43:0/1609085966 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3817 (crc 0 0 0) 0x560450c63fe0 con 0x560450e56490 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f4596ffd640 10 client.6153 handle_command_reply: tid=0 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 client.6153 shutdown 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 msgr2=0x560450e55ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.446+0000 7f45c1567000 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e55ad0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f45b00868b0 tx=0x7f45b001d040 comp rx=0 tx=0).stop 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 shutdown_connections 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560450e56490 0x560450e568a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f4596ffd640 0 client.6153 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560450e52690 0x560450e55ad0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560450e51d70 0x560450ec4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 --2- 172.21.15.43:0/1609085966 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560450e08830 0x560450ec4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 >> 172.21.15.43:0/1609085966 conn(0x560450e047e0 msgr2=0x560450e69620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 shutdown_connections 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 1 -- 172.21.15.43:0/1609085966 wait complete. 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45c1567000 20 client.6153 trim_cache size 0 max 16384 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/dcache.c", 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628331, 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.980858+0000", 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.907021+0000", 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:50.459 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:50.460 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3670, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.907021+0000", 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 4, 2024-03-30T13:13:50.461 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:50.462 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3670, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.980858+0000" 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3670, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.980858+0000" 2024-03-30T13:13:50.463 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 98, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 8, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:50.464 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:50.465 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:50.466 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.467 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.468 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:50.469 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.470 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:50.471 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:50.471 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45a4067fc0 msgr2=0x7f45a406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45a4067fc0 0x7f45a406a480 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f45b80ca2d0 tx=0x7f45a8008040 comp rx=0 tx=0).stop 2024-03-30T13:13:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b815f880 msgr2=0x7f45b814a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b815f880 0x7f45b814a6a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f45b000c730 tx=0x7f45b000cbf0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 shutdown_connections 2024-03-30T13:13:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f45a4067fc0 0x7f45a406a480 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b815f880 0x7f45b814a6a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f45b8070640 0x7f45b814a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 --2- 172.21.15.43:0/4154281908 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f45b806eec0 0x7f45b80b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 >> 172.21.15.43:0/4154281908 conn(0x7f45b80b3cf0 msgr2=0x7f45b80b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 shutdown_connections 2024-03-30T13:13:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.450+0000 7f45be3e5640 1 -- 172.21.15.43:0/4154281908 wait complete. 2024-03-30T13:13:50.473 DEBUG:tasks.cephfs.test_uninlining:getting inode info #18 of 23:1099511628339 2024-03-30T13:13:50.473 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628339 2024-03-30T13:13:50.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628339 2024-03-30T13:13:50.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.582+0000 7fe62cd62640 1 Processor -- start 2024-03-30T13:13:50.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.582+0000 7fe62cd62640 1 -- start start 2024-03-30T13:13:50.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe62815f200 0x7fe62815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6280cced0 0x7fe628154d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6280cfe50 con 0x7fe6280b4fc0 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe6280d0a60 con 0x7fe6280cced0 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe6280d0130 con 0x7fe62815f200 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6280cced0 0x7fe628154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6280cced0 0x7fe628154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60046/0 (socket says 172.21.15.43:60046) 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 -- 172.21.15.43:0/1840459895 learned_addr learned my addr 172.21.15.43:0/1840459895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62659c640 1 --2- 172.21.15.43:0/1840459895 >> v2:172.21.15.113:3300/0 conn(0x7fe62815f200 0x7fe62815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 -- 172.21.15.43:0/1840459895 >> v2:172.21.15.113:3300/0 conn(0x7fe62815f200 msgr2=0x7fe62815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 --2- 172.21.15.43:0/1840459895 >> v2:172.21.15.113:3300/0 conn(0x7fe62815f200 0x7fe62815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 -- 172.21.15.43:0/1840459895 >> v1:172.21.15.113:6789/0 conn(0x7fe6280b4fc0 legacy=0x7fe6280cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 -- 172.21.15.43:0/1840459895 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6280d0410 con 0x7fe6280cced0 2024-03-30T13:13:50.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe626d9d640 1 --2- 172.21.15.43:0/1840459895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280cced0 0x7fe628154d50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe6280cfcf0 tx=0x7fe62001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ed88fc11c331eb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe625d9b640 1 -- 172.21.15.43:0/1840459895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a57d0 con 0x7fe6280cced0 2024-03-30T13:13:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe625d9b640 1 -- 172.21.15.43:0/1840459895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6200a5970 con 0x7fe6280cced0 2024-03-30T13:13:50.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe625d9b640 1 -- 172.21.15.43:0/1840459895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a5d40 con 0x7fe6280cced0 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- 172.21.15.43:0/1840459895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280cced0 msgr2=0x7fe628154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 --2- 172.21.15.43:0/1840459895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280cced0 0x7fe628154d50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fe6280cfcf0 tx=0x7fe62001c9f0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- 172.21.15.43:0/1840459895 shutdown_connections 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 --2- 172.21.15.43:0/1840459895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280cced0 0x7fe628154d50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 --2- 172.21.15.43:0/1840459895 >> v2:172.21.15.113:3300/0 conn(0x7fe62815f200 0x7fe62815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- 172.21.15.43:0/1840459895 >> 172.21.15.43:0/1840459895 conn(0x7fe6280b3c80 msgr2=0x7fe6280b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- 172.21.15.43:0/1840459895 shutdown_connections 2024-03-30T13:13:50.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.586+0000 7fe62cd62640 1 -- 172.21.15.43:0/1840459895 wait complete. 2024-03-30T13:13:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 Processor -- start 2024-03-30T13:13:50.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 -- start start 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280b4fc0 0x7fe62814b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6280cced0 0x7fe6281546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe62815f200 0x7fe6280b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe6280b6ed0 con 0x7fe6280b4fc0 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe6280b6d50 con 0x7fe6280cced0 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe6280b7050 con 0x7fe62815f200 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280b4fc0 0x7fe62814b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280b4fc0 0x7fe62814b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60054/0 (socket says 172.21.15.43:60054) 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 -- 172.21.15.43:0/3466966739 learned_addr learned my addr 172.21.15.43:0/3466966739 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe626d9d640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe62815f200 0x7fe6280b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe61ffff640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6280cced0 0x7fe6281546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 -- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe62815f200 msgr2=0x7fe6280b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe62815f200 0x7fe6280b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 -- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6280cced0 msgr2=0x7fe6281546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6280cced0 0x7fe6281546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 -- 172.21.15.43:0/3466966739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe628156dd0 con 0x7fe6280b4fc0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe626d9d640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe62815f200 0x7fe6280b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe61ffff640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6280cced0 0x7fe6281546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62659c640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280b4fc0 0x7fe62814b9b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe61800d620 tx=0x7fe61800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe61effd640 1 -- 172.21.15.43:0/3466966739 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61800b620 con 0x7fe6280b4fc0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe61effd640 1 -- 172.21.15.43:0/3466966739 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe618019070 con 0x7fe6280b4fc0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe61effd640 1 -- 172.21.15.43:0/3466966739 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe61800b9e0 con 0x7fe6280b4fc0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe628155d30 con 0x7fe6280b4fc0 2024-03-30T13:13:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.590+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6281562f0 con 0x7fe6280b4fc0 2024-03-30T13:13:50.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe61effd640 1 -- 172.21.15.43:0/3466966739 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe61800bb80 con 0x7fe6280b4fc0 2024-03-30T13:13:50.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe61effd640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe608067fc0 0x7fe60806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe61effd640 1 -- 172.21.15.43:0/3466966739 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe618086a50 con 0x7fe6280b4fc0 2024-03-30T13:13:50.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe61ffff640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe608067fc0 0x7fe60806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 Processor -- start 2024-03-30T13:13:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 -- start start 2024-03-30T13:13:50.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe61ffff640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe608067fc0 0x7fe60806a480 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6280c55c0 tx=0x7fe610009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x558505327150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x5585052ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55850512baa0 con 0x5585052dd830 2024-03-30T13:13:50.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5585050b2af0 con 0x558505327690 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x558505127a50 con 0x558505326d70 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x5585052ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x5585052ddc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60060/0 (socket says 172.21.15.43:60060) 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 -- 172.21.15.43:0/2056144627 learned_addr learned my addr 172.21.15.43:0/2056144627 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe62659c640 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x558505327150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe61ffff640 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 -- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 msgr2=0x558505327150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x558505327150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 -- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 msgr2=0x5585052dd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe626d9d640 1 -- 172.21.15.43:0/2056144627 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585050b0d00 con 0x5585052dd830 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.594+0000 7fe61ffff640 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x5585052ddc10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe62001ca30 tx=0x7fe620004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295e3108a85233b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe607fff640 1 -- 172.21.15.43:0/2056144627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a57d0 con 0x5585052dd830 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe607fff640 1 -- 172.21.15.43:0/2056144627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6200a5970 con 0x5585052dd830 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe607fff640 1 -- 172.21.15.43:0/2056144627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a5d60 con 0x5585052dd830 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/2056144627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 msgr2=0x5585052ddc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x5585052ddc10 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe62001ca30 tx=0x7fe620004ac0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/2056144627 shutdown_connections 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x5585052ddc10 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2056144627 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x558505327150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/2056144627 >> 172.21.15.43:0/2056144627 conn(0x5585052d97e0 msgr2=0x5585052dbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/2056144627 shutdown_connections 2024-03-30T13:13:50.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/2056144627 wait complete. 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- start start 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 Processor -- start 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5585052dd830 0x5585052e2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850530bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505327690 0x55850530c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5585050f3270 con 0x558505327690 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5585050e96a0 con 0x5585052dd830 2024-03-30T13:13:50.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5585050c3880 con 0x558505326d70 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 10 client.0 _ll_register_callbacks cb 0x5585052c8490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505327690 0x55850530c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505327690 0x55850530c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60066/0 (socket says 172.21.15.43:60066) 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 -- 172.21.15.43:0/833753155 learned_addr learned my addr 172.21.15.43:0/833753155 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62659c640 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5585052dd830 0x5585052e2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe61ffff640 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850530bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 -- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 msgr2=0x55850530bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850530bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 -- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5585052dd830 msgr2=0x5585052e2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5585052dd830 0x5585052e2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 -- 172.21.15.43:0/833753155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585050f0730 con 0x558505327690 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe626d9d640 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505327690 0x55850530c2e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe620000f90 tx=0x7fe62001b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a038432b1bd541 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe6057fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe6057fa640 1 -- 172.21.15.43:0/833753155 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200c7020 con 0x558505327690 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe6057fa640 1 -- 172.21.15.43:0/833753155 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe6200039a0 con 0x558505327690 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6200a5970 con 0x558505327690 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe6057fa640 1 -- 172.21.15.43:0/833753155 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe6200d1050 con 0x558505327690 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 10 client.6165 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5585050e3ca0 con 0x558505327690 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe6057fa640 1 -- 172.21.15.43:0/833753155 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6200a5970 con 0x558505327690 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe6057fa640 1 -- 172.21.15.43:0/833753155 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fe6200d4020 con 0x558505327690 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 10 client.6165 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 10 client.6165 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 20 client.6165 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532e5d0 0x55850532e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 4 client.6165 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.598+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585050c6c90 con 0x55850532e5d0 2024-03-30T13:13:50.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62659c640 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532e5d0 0x55850532e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62659c640 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532e5d0 0x55850532e9e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e646c41cc6dc3f7 server_cookie=e4693a53df475310 in_seq=0 out_seq=0 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe6057fa640 10 client.6165 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe6057fa640 1 -- 172.21.15.43:0/833753155 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5585050c6c90 con 0x55850532e5d0 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe6057fa640 10 client.6165 handle_command_reply: tid=0 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 client.6165 shutdown 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505327690 msgr2=0x55850530c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505327690 0x55850530c2e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe620000f90 tx=0x7fe62001b230 comp rx=0 tx=0).stop 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 shutdown_connections 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532e5d0 0x55850532e9e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505327690 0x55850530c2e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850530bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe6057fa640 0 client.6165 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 --2- 172.21.15.43:0/833753155 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5585052dd830 0x5585052e2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 >> 172.21.15.43:0/833753155 conn(0x5585052d97e0 msgr2=0x5585052dc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 shutdown_connections 2024-03-30T13:13:50.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 1 -- 172.21.15.43:0/833753155 wait complete. 2024-03-30T13:13:50.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.602+0000 7fe62fee4000 20 client.6165 trim_cache size 0 max 16384 2024-03-30T13:13:50.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 Processor -- start 2024-03-30T13:13:50.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- start start 2024-03-30T13:13:50.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5585052dd830 0x55850530c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505326d70 0x5585052dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe628204110 con 0x558505326d70 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe628203150 con 0x558505327690 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe628151800 con 0x5585052dd830 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505326d70 0x5585052dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505326d70 0x5585052dbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60082/0 (socket says 172.21.15.43:60082) 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 -- 172.21.15.43:0/3300411505 learned_addr learned my addr 172.21.15.43:0/3300411505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62659c640 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5585052dd830 0x55850530c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe626d9d640 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 -- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5585052dd830 msgr2=0x55850530c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5585052dd830 0x55850530c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 -- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 msgr2=0x5585052dc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 -- 172.21.15.43:0/3300411505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585050b2d00 con 0x558505326d70 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe61ffff640 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505326d70 0x5585052dbbd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe61000da60 tx=0x7fe610013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd7e611fb250da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe6077fe640 1 -- 172.21.15.43:0/3300411505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610025070 con 0x558505326d70 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe6077fe640 1 -- 172.21.15.43:0/3300411505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6100149a0 con 0x558505326d70 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe6077fe640 1 -- 172.21.15.43:0/3300411505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610014d70 con 0x558505326d70 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- 172.21.15.43:0/3300411505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505326d70 msgr2=0x5585052dbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505326d70 0x5585052dbbd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe61000da60 tx=0x7fe610013000 comp rx=0 tx=0).stop 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- 172.21.15.43:0/3300411505 shutdown_connections 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x5585052dc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558505326d70 0x5585052dbbd0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 --2- 172.21.15.43:0/3300411505 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5585052dd830 0x55850530c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- 172.21.15.43:0/3300411505 >> 172.21.15.43:0/3300411505 conn(0x5585052d97e0 msgr2=0x55850533df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- 172.21.15.43:0/3300411505 shutdown_connections 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- 172.21.15.43:0/3300411505 wait complete. 2024-03-30T13:13:50.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 -- start start 2024-03-30T13:13:50.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.618+0000 7fe62fee4000 1 Processor -- start 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x558505394a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850532aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x55850532b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55850512adb0 con 0x5585052dd830 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe628202cc0 con 0x558505327690 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55850512baa0 con 0x558505326d70 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 10 client.0 _ll_register_callbacks cb 0x7fe620009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x558505394a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x558505394a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60098/0 (socket says 172.21.15.43:60098) 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 -- 172.21.15.43:0/2389760745 learned_addr learned my addr 172.21.15.43:0/2389760745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe61ffff640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850532aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe626d9d640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x55850532b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 -- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 msgr2=0x55850532aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850532aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 -- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 msgr2=0x55850532b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x55850532b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 -- 172.21.15.43:0/2389760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585051acf70 con 0x5585052dd830 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe626d9d640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x55850532b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62659c640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x558505394a50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe61805a010 tx=0x7fe6180563c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df417fcf0d781b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 1 -- 172.21.15.43:0/2389760745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe618074070 con 0x5585052dd830 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 1 -- 172.21.15.43:0/2389760745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe618054cd0 con 0x5585052dd830 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6100149a0 con 0x5585052dd830 2024-03-30T13:13:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 1 -- 172.21.15.43:0/2389760745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe618075270 con 0x5585052dd830 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 10 client.6171 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x5585050d2760 con 0x5585052dd830 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 1 -- 172.21.15.43:0/2389760745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6100149a0 con 0x5585052dd830 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 1 -- 172.21.15.43:0/2389760745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fe618013070 con 0x5585052dd830 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 10 client.6171 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 10 client.6171 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 20 client.6171 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532b550 0x55850532b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 4 client.6171 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628339}] 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628339}) -- 0x558505138fe0 con 0x55850532b550 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe61ffff640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532b550 0x55850532b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe61ffff640 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532b550 0x55850532b960 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db517490144df4a server_cookie=529bf014acef3ec2 in_seq=0 out_seq=0 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 10 client.6171 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 1 -- 172.21.15.43:0/2389760745 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3825 (crc 0 0 0) 0x558505138fe0 con 0x55850532b550 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 10 client.6171 handle_command_reply: tid=0 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 client.6171 shutdown 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 msgr2=0x558505394a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x558505394a50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe61805a010 tx=0x7fe6180563c0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 shutdown_connections 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55850532b550 0x55850532b960 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x558505327690 0x55850532b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x558505326d70 0x55850532aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 --2- 172.21.15.43:0/2389760745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5585052dd830 0x558505394a50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe6057fa640 0 client.6171 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 >> 172.21.15.43:0/2389760745 conn(0x5585052d97e0 msgr2=0x55850533e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 shutdown_connections 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 1 -- 172.21.15.43:0/2389760745 wait complete. 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62fee4000 20 client.6171 trim_cache size 0 max 16384 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-bufmap.h", 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628339, 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:15.981971+0000", 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.985944+0000", 2024-03-30T13:13:50.633 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:50.634 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 925, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.985944+0000", 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:50.635 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 925, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.636 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.981971+0000" 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 925, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:15.981971+0000" 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 102, 2024-03-30T13:13:50.637 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 24, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:50.638 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:50.639 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:50.640 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:50.641 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.642 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:50.643 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:50.644 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:50.645 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:50.645 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:50.645 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe608067fc0 msgr2=0x7fe60806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe608067fc0 0x7fe60806a480 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6280c55c0 tx=0x7fe610009040 comp rx=0 tx=0).stop 2024-03-30T13:13:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280b4fc0 msgr2=0x7fe62814b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280b4fc0 0x7fe62814b9b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe61800d620 tx=0x7fe61800dae0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 shutdown_connections 2024-03-30T13:13:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe608067fc0 0x7fe60806a480 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe62815f200 0x7fe6280b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe6280cced0 0x7fe6281546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 --2- 172.21.15.43:0/3466966739 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6280b4fc0 0x7fe62814b9b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.622+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 >> 172.21.15.43:0/3466966739 conn(0x7fe6280b3c80 msgr2=0x7fe6280c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.626+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 shutdown_connections 2024-03-30T13:13:50.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.626+0000 7fe62cd62640 1 -- 172.21.15.43:0/3466966739 wait complete. 2024-03-30T13:13:50.647 DEBUG:tasks.cephfs.test_uninlining:getting inode info #19 of 23:1099511628343 2024-03-30T13:13:50.647 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628343 2024-03-30T13:13:50.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628343 2024-03-30T13:13:50.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.754+0000 7fb7679f2640 1 Processor -- start 2024-03-30T13:13:50.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.754+0000 7fb7679f2640 1 -- start start 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb7679f2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb760144930 0x7fb7600b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb7679f2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb76006cd80 0x7fb7600ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb7679f2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7600b33c0 con 0x7fb7600b4870 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb7679f2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb7600cdb90 con 0x7fb760144930 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb7679f2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb7600b3660 con 0x7fb76006cd80 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb76578e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb760144930 0x7fb7600b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb764f8d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb7600b4870 0x7fb76006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47292/0 (socket says 172.21.15.43:47292) 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb764f8d640 1 -- 172.21.15.43:0/2651050849 learned_addr learned my addr 172.21.15.43:0/2651050849 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb765f8f640 1 --2- 172.21.15.43:0/2651050849 >> v2:172.21.15.113:3300/0 conn(0x7fb76006cd80 0x7fb7600ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb757fff640 1 -- 172.21.15.43:0/2651050849 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2133112655 0 0) 0x7fb7600b33c0 con 0x7fb7600b4870 2024-03-30T13:13:50.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb757fff640 1 -- 172.21.15.43:0/2651050849 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb74c003530 con 0x7fb7600b4870 2024-03-30T13:13:50.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb76578e640 1 -- 172.21.15.43:0/2651050849 >> v2:172.21.15.113:3300/0 conn(0x7fb76006cd80 msgr2=0x7fb7600ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb76578e640 1 --2- 172.21.15.43:0/2651050849 >> v2:172.21.15.113:3300/0 conn(0x7fb76006cd80 0x7fb7600ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb76578e640 1 -- 172.21.15.43:0/2651050849 >> v1:172.21.15.43:6789/0 conn(0x7fb7600b4870 legacy=0x7fb76006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb76578e640 1 -- 172.21.15.43:0/2651050849 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7600b3940 con 0x7fb760144930 2024-03-30T13:13:50.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb765f8f640 1 --2- 172.21.15.43:0/2651050849 >> v2:172.21.15.113:3300/0 conn(0x7fb76006cd80 0x7fb7600ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb76578e640 1 --2- 172.21.15.43:0/2651050849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb7600b2e80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb7600cdd10 tx=0x7fb75c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37ee72df2feb651 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb757fff640 1 -- 172.21.15.43:0/2651050849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0b2070 con 0x7fb760144930 2024-03-30T13:13:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb757fff640 1 -- 172.21.15.43:0/2651050849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0a9070 con 0x7fb760144930 2024-03-30T13:13:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.758+0000 7fb757fff640 1 -- 172.21.15.43:0/2651050849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0ad860 con 0x7fb760144930 2024-03-30T13:13:50.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- 172.21.15.43:0/2651050849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 msgr2=0x7fb7600b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 --2- 172.21.15.43:0/2651050849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb7600b2e80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb7600cdd10 tx=0x7fb75c01c3a0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- 172.21.15.43:0/2651050849 shutdown_connections 2024-03-30T13:13:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 --2- 172.21.15.43:0/2651050849 >> v2:172.21.15.113:3300/0 conn(0x7fb76006cd80 0x7fb7600ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 --2- 172.21.15.43:0/2651050849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb7600b2e80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- 172.21.15.43:0/2651050849 >> 172.21.15.43:0/2651050849 conn(0x7fb7600d00a0 msgr2=0x7fb7600d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- 172.21.15.43:0/2651050849 shutdown_connections 2024-03-30T13:13:50.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- 172.21.15.43:0/2651050849 wait complete. 2024-03-30T13:13:50.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 Processor -- start 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- start start 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76006cd80 0x7fb76013d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7600b4870 0x7fb7600727f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb760072d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600ceef0 con 0x7fb760144930 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7600ced70 con 0x7fb76006cd80 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600cf070 con 0x7fb7600b4870 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb760072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb760072d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60112/0 (socket says 172.21.15.43:60112) 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 -- 172.21.15.43:0/3012599021 learned_addr learned my addr 172.21.15.43:0/3012599021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb764f8d640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7600b4870 0x7fb7600727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 -- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7600b4870 msgr2=0x7fb7600727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb76578e640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76006cd80 0x7fb76013d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7600b4870 0x7fb7600727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 -- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76006cd80 msgr2=0x7fb76013d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76006cd80 0x7fb76013d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 -- 172.21.15.43:0/3012599021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76015bbc0 con 0x7fb760144930 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb765f8f640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb760072d30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb758007ef0 tx=0x7fb75800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb755ffb640 1 -- 172.21.15.43:0/3012599021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75801a070 con 0x7fb760144930 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb755ffb640 1 -- 172.21.15.43:0/3012599021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758003ba0 con 0x7fb760144930 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76015be60 con 0x7fb760144930 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb755ffb640 1 -- 172.21.15.43:0/3012599021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75800de80 con 0x7fb760144930 2024-03-30T13:13:50.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.762+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7600a9940 con 0x7fb760144930 2024-03-30T13:13:50.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb755ffb640 1 -- 172.21.15.43:0/3012599021 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb75801d070 con 0x7fb760144930 2024-03-30T13:13:50.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb755ffb640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb744067fc0 0x7fb74406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb755ffb640 1 -- 172.21.15.43:0/3012599021 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fb758086960 con 0x7fb760144930 2024-03-30T13:13:50.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76578e640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb744067fc0 0x7fb74406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 Processor -- start 2024-03-30T13:13:50.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 -- start start 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76578e640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb744067fc0 0x7fb74406a480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0af000 tx=0x7fb75c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c342d70 0x560e4c343150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c2f92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560e4c147aa0 con 0x560e4c2f9830 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560e4c0ceaf0 con 0x560e4c342d70 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560e4c143a50 con 0x560e4c343690 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60124/0 (socket says 172.21.15.43:60124) 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 -- 172.21.15.43:0/1101519940 learned_addr learned my addr 172.21.15.43:0/1101519940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76578e640 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c342d70 0x560e4c343150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 -- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 msgr2=0x560e4c2f92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb764f8d640 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c2f92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c2f92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 -- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c342d70 msgr2=0x560e4c343150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c342d70 0x560e4c343150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 -- 172.21.15.43:0/1101519940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4c0ccd00 con 0x560e4c2f9830 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb76578e640 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c342d70 0x560e4c343150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb765f8f640 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2f9c10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb758004b00 tx=0x7fb75805f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c17834f8704192 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb74affd640 1 -- 172.21.15.43:0/1101519940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75805d070 con 0x560e4c2f9830 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb74affd640 1 -- 172.21.15.43:0/1101519940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758052ed0 con 0x560e4c2f9830 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.766+0000 7fb74affd640 1 -- 172.21.15.43:0/1101519940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758056d00 con 0x560e4c2f9830 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1101519940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 msgr2=0x560e4c2f9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2f9c10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fb758004b00 tx=0x7fb75805f390 comp rx=0 tx=0).stop 2024-03-30T13:13:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1101519940 shutdown_connections 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2f9c10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c2f92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1101519940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c342d70 0x560e4c343150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1101519940 >> 172.21.15.43:0/1101519940 conn(0x560e4c2f57e0 msgr2=0x560e4c2f7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1101519940 shutdown_connections 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1101519940 wait complete. 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- start start 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 Processor -- start 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c327da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c343690 0x560e4c3282e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560e4c10f270 con 0x560e4c2f9830 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560e4c1056a0 con 0x560e4c343690 2024-03-30T13:13:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560e4c0df880 con 0x560e4c342d70 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 10 client.0 _ll_register_callbacks cb 0x560e4c2e4490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60138/0 (socket says 172.21.15.43:60138) 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 -- 172.21.15.43:0/1314046217 learned_addr learned my addr 172.21.15.43:0/1314046217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb764f8d640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c327da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb765f8f640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c343690 0x560e4c3282e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 -- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 msgr2=0x560e4c327da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c327da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 -- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c343690 msgr2=0x560e4c3282e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c343690 0x560e4c3282e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 -- 172.21.15.43:0/1314046217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e4c10c730 con 0x560e4c2f9830 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb764f8d640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c327da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb765f8f640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c343690 0x560e4c3282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76578e640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2fec40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009580 tx=0x7fb75c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f606487c54ec77fc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb73ffff640 1 -- 172.21.15.43:0/1314046217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0b2040 con 0x560e4c2f9830 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb73ffff640 1 -- 172.21.15.43:0/1314046217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fb75c0b8bc0 con 0x560e4c2f9830 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb758052ed0 con 0x560e4c2f9830 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb73ffff640 1 -- 172.21.15.43:0/1314046217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb75c0d1050 con 0x560e4c2f9830 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 10 client.6186 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560e4c0ffca0 con 0x560e4c2f9830 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb73ffff640 1 -- 172.21.15.43:0/1314046217 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758052ed0 con 0x560e4c2f9830 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb73ffff640 1 -- 172.21.15.43:0/1314046217 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fb75c0d4020 con 0x560e4c2f9830 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 10 client.6186 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 10 client.6186 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 20 client.6186 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c34a5d0 0x560e4c34a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 4 client.6186 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e4c0e2c90 con 0x560e4c34a5d0 2024-03-30T13:13:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb764f8d640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c34a5d0 0x560e4c34a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb73ffff640 10 client.6186 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.770+0000 7fb764f8d640 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c34a5d0 0x560e4c34a9e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b43aa9041d7ff5c server_cookie=86216e95e142ccd7 in_seq=0 out_seq=0 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb73ffff640 1 -- 172.21.15.43:0/1314046217 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x560e4c0e2c90 con 0x560e4c34a5d0 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb73ffff640 10 client.6186 handle_command_reply: tid=0 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 client.6186 shutdown 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 msgr2=0x560e4c2fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2fec40 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009580 tx=0x7fb75c0046d0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 shutdown_connections 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c34a5d0 0x560e4c34a9e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c343690 0x560e4c3282e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb73ffff640 0 client.6186 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c327da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 --2- 172.21.15.43:0/1314046217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c2f9830 0x560e4c2fec40 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 >> 172.21.15.43:0/1314046217 conn(0x560e4c2f57e0 msgr2=0x560e4c2f8460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 shutdown_connections 2024-03-30T13:13:50.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 1 -- 172.21.15.43:0/1314046217 wait complete. 2024-03-30T13:13:50.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.774+0000 7fb76ab74000 20 client.6186 trim_cache size 0 max 16384 2024-03-30T13:13:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 Processor -- start 2024-03-30T13:13:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 -- start start 2024-03-30T13:13:50.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c2f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c2f8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c343690 0x560e4c2f8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb760201a40 con 0x560e4c343690 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb760200a80 con 0x560e4c2f9830 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.786+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb76015d300 con 0x560e4c342d70 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c343690 0x560e4c2f8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c343690 0x560e4c2f8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60150/0 (socket says 172.21.15.43:60150) 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 -- 172.21.15.43:0/3429477780 learned_addr learned my addr 172.21.15.43:0/3429477780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c2f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76578e640 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c2f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 -- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 msgr2=0x560e4c2f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c2f8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 -- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 msgr2=0x560e4c2f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c2f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 -- 172.21.15.43:0/3429477780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4c0ced00 con 0x560e4c343690 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c343690 0x560e4c2f8650 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb758084f20 tx=0x7fb758021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64474005725d0132 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb74a7fc640 1 -- 172.21.15.43:0/3429477780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758052ed0 con 0x560e4c343690 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb74a7fc640 1 -- 172.21.15.43:0/3429477780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758056ac0 con 0x560e4c343690 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb74a7fc640 1 -- 172.21.15.43:0/3429477780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75805ef00 con 0x560e4c343690 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/3429477780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c343690 msgr2=0x560e4c2f8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c343690 0x560e4c2f8650 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb758084f20 tx=0x7fb758021000 comp rx=0 tx=0).stop 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/3429477780 shutdown_connections 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c343690 0x560e4c2f8650 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c342d70 0x560e4c2f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- 172.21.15.43:0/3429477780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c2f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/3429477780 >> 172.21.15.43:0/3429477780 conn(0x560e4c2f57e0 msgr2=0x560e4c3262b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/3429477780 shutdown_connections 2024-03-30T13:13:50.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/3429477780 wait complete. 2024-03-30T13:13:50.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- start start 2024-03-30T13:13:50.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 Processor -- start 2024-03-30T13:13:50.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c328050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c342d70 0x560e4c373d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c374240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560e4c203b00 con 0x560e4c342d70 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb76015d950 con 0x560e4c2f9830 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560e4c3269c0 con 0x560e4c343690 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 10 client.0 _ll_register_callbacks cb 0x7fb75c0b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c342d70 0x560e4c373d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c342d70 0x560e4c373d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60164/0 (socket says 172.21.15.43:60164) 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 -- 172.21.15.43:0/872730134 learned_addr learned my addr 172.21.15.43:0/872730134 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76578e640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c328050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c374240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 -- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 msgr2=0x560e4c374240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c374240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 -- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 msgr2=0x560e4c328050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c328050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 -- 172.21.15.43:0/872730134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e4c1c8f70 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb765f8f640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c374240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb764f8d640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c342d70 0x560e4c373d00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb750001640 tx=0x7fb750001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a280226e8aab3351 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb73ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb73ffff640 1 -- 172.21.15.43:0/872730134 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500072b0 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb73ffff640 1 -- 172.21.15.43:0/872730134 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fb750007450 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb758056ac0 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb73ffff640 1 -- 172.21.15.43:0/872730134 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb750026050 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 10 client.6192 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x560e4c0ee760 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb73ffff640 1 -- 172.21.15.43:0/872730134 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb758056ac0 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb73ffff640 1 -- 172.21.15.43:0/872730134 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fb750029020 con 0x560e4c342d70 2024-03-30T13:13:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 10 client.6192 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 10 client.6192 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 20 client.6192 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c374780 0x560e4c3b8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 4 client.6192 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628343}] 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.790+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628343}) -- 0x560e4c154fe0 con 0x560e4c374780 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76578e640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c374780 0x560e4c3b8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76578e640 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c374780 0x560e4c3b8640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92de8dc556e36dc9 server_cookie=fd5e178963420046 in_seq=0 out_seq=0 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb73ffff640 10 client.6192 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb73ffff640 1 -- 172.21.15.43:0/872730134 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3826 (crc 0 0 0) 0x560e4c154fe0 con 0x560e4c374780 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb73ffff640 10 client.6192 handle_command_reply: tid=0 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 client.6192 shutdown 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c342d70 msgr2=0x560e4c373d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c342d70 0x560e4c373d00 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb750001640 tx=0x7fb750001b30 comp rx=0 tx=0).stop 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 shutdown_connections 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560e4c374780 0x560e4c3b8640 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560e4c343690 0x560e4c374240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560e4c342d70 0x560e4c373d00 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb73ffff640 0 client.6192 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 --2- 172.21.15.43:0/872730134 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560e4c2f9830 0x560e4c328050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 >> 172.21.15.43:0/872730134 conn(0x560e4c2f57e0 msgr2=0x560e4c359680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 shutdown_connections 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 1 -- 172.21.15.43:0/872730134 wait complete. 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb76ab74000 20 client.6192 trim_cache size 0 max 16384 2024-03-30T13:13:50.804 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debugfs.h", 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628343, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:16.007527+0000", 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:13.006186+0000", 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:50.805 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 311, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:50.806 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:13.006186+0000", 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:50.807 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 311, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.007527+0000" 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 311, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.808 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.007527+0000" 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 124, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 32, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:50.809 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.810 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:50.811 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.812 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.813 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.814 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:50.815 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:50.816 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:50.816 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb744067fc0 msgr2=0x7fb74406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb744067fc0 0x7fb74406a480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0af000 tx=0x7fb75c008040 comp rx=0 tx=0).stop 2024-03-30T13:13:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 msgr2=0x7fb760072d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb760072d30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb758007ef0 tx=0x7fb75800d4a0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 shutdown_connections 2024-03-30T13:13:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb744067fc0 0x7fb74406a480 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb760144930 0x7fb760072d30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb7600b4870 0x7fb7600727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 --2- 172.21.15.43:0/3012599021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb76006cd80 0x7fb76013d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 >> 172.21.15.43:0/3012599021 conn(0x7fb7600d00a0 msgr2=0x7fb7600b1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 shutdown_connections 2024-03-30T13:13:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.794+0000 7fb7679f2640 1 -- 172.21.15.43:0/3012599021 wait complete. 2024-03-30T13:13:50.818 DEBUG:tasks.cephfs.test_uninlining:getting inode info #20 of 23:1099511628344 2024-03-30T13:13:50.818 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628344 2024-03-30T13:13:50.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628344 2024-03-30T13:13:50.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec99962640 1 Processor -- start 2024-03-30T13:13:50.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec99962640 1 -- start start 2024-03-30T13:13:50.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec99962640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fec94144930 0x7fec940b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec99962640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec940b4870 0x7fec9406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec99962640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec940b33c0 con 0x7fec9406cbb0 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec99962640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fec9406d000 con 0x7fec940b4870 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec99962640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fec940cdb90 con 0x7fec94144930 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec8bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec940b4870 0x7fec9406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec937fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fec9406cbb0 0x7fec940713f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47300/0 (socket says 172.21.15.43:47300) 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec937fe640 1 -- 172.21.15.43:0/1199385060 learned_addr learned my addr 172.21.15.43:0/1199385060 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec92ffd640 1 --2- 172.21.15.43:0/1199385060 >> v2:172.21.15.113:3300/0 conn(0x7fec94144930 0x7fec940b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec8bfff640 1 -- 172.21.15.43:0/1199385060 >> v2:172.21.15.113:3300/0 conn(0x7fec94144930 msgr2=0x7fec940b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1199385060 >> v2:172.21.15.113:3300/0 conn(0x7fec94144930 0x7fec940b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec8bfff640 1 -- 172.21.15.43:0/1199385060 >> v1:172.21.15.43:6789/0 conn(0x7fec9406cbb0 legacy=0x7fec940713f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec8bfff640 1 -- 172.21.15.43:0/1199385060 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec940b3660 con 0x7fec940b4870 2024-03-30T13:13:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.926+0000 7fec927fc640 1 -- 172.21.15.43:0/1199385060 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084703326 0 0) 0x7fec940b33c0 con 0x7fec9406cbb0 2024-03-30T13:13:50.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1199385060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec9406c670 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fec9406f900 tx=0x7fec7c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db8f2fe5bd031dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec927fc640 1 -- 172.21.15.43:0/1199385060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b1070 con 0x7fec940b4870 2024-03-30T13:13:50.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec927fc640 1 -- 172.21.15.43:0/1199385060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c0ac620 con 0x7fec940b4870 2024-03-30T13:13:50.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec927fc640 1 -- 172.21.15.43:0/1199385060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0ac9f0 con 0x7fec940b4870 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- 172.21.15.43:0/1199385060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 msgr2=0x7fec9406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 --2- 172.21.15.43:0/1199385060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec9406c670 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fec9406f900 tx=0x7fec7c01c3c0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- 172.21.15.43:0/1199385060 shutdown_connections 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 --2- 172.21.15.43:0/1199385060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec9406c670 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 --2- 172.21.15.43:0/1199385060 >> v2:172.21.15.113:3300/0 conn(0x7fec94144930 0x7fec940b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- 172.21.15.43:0/1199385060 >> 172.21.15.43:0/1199385060 conn(0x7fec940d00a0 msgr2=0x7fec940d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- 172.21.15.43:0/1199385060 shutdown_connections 2024-03-30T13:13:50.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- 172.21.15.43:0/1199385060 wait complete. 2024-03-30T13:13:50.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 Processor -- start 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- start start 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9406cbb0 0x7fec9413d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec940a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec94144930 0x7fec9413f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fec94167df0 con 0x7fec940b4870 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fec940cd7b0 con 0x7fec9406cbb0 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec99962640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fec94167f70 con 0x7fec94144930 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec940a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec940a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60180/0 (socket says 172.21.15.43:60180) 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.930+0000 7fec8bfff640 1 -- 172.21.15.43:0/2027427186 learned_addr learned my addr 172.21.15.43:0/2027427186 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec92ffd640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9406cbb0 0x7fec9413d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec937fe640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec94144930 0x7fec9413f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec8bfff640 1 -- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec94144930 msgr2=0x7fec9413f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec94144930 0x7fec9413f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec8bfff640 1 -- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9406cbb0 msgr2=0x7fec9413d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9406cbb0 0x7fec9413d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec8bfff640 1 -- 172.21.15.43:0/2027427186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec941f8960 con 0x7fec940b4870 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec92ffd640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9406cbb0 0x7fec9413d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec937fe640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec94144930 0x7fec9413f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec940a9be0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0af040 tx=0x7fec7c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec90ff9640 1 -- 172.21.15.43:0/2027427186 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b9020 con 0x7fec940b4870 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec90ff9640 1 -- 172.21.15.43:0/2027427186 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c0b5630 con 0x7fec940b4870 2024-03-30T13:13:50.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec941f8c40 con 0x7fec940b4870 2024-03-30T13:13:50.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec90ff9640 1 -- 172.21.15.43:0/2027427186 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0b1040 con 0x7fec940b4870 2024-03-30T13:13:50.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec941f9200 con 0x7fec940b4870 2024-03-30T13:13:50.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec90ff9640 1 -- 172.21.15.43:0/2027427186 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fec7c0bc020 con 0x7fec940b4870 2024-03-30T13:13:50.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec90ff9640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fec74067fc0 0x7fec7406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec90ff9640 1 -- 172.21.15.43:0/2027427186 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fec7c120440 con 0x7fec940b4870 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec92ffd640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fec74067fc0 0x7fec7406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec9cae4000 1 Processor -- start 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec9cae4000 1 -- start start 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.934+0000 7fec92ffd640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fec74067fc0 0x7fec7406a480 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec840097a0 tx=0x7fec84008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55cceba42150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceb9f8830 0x55cceb9f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cceb846aa0 con 0x55cceba42690 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cceb7cdaf0 con 0x55cceb9f8830 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55cceb842a50 con 0x55cceba41d70 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60196/0 (socket says 172.21.15.43:60196) 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 -- 172.21.15.43:0/2099742943 learned_addr learned my addr 172.21.15.43:0/2099742943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55cceba42150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec937fe640 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceb9f8830 0x55cceb9f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 -- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 msgr2=0x55cceba42150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55cceba42150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 -- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceb9f8830 msgr2=0x55cceb9f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceb9f8830 0x55cceb9f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 -- 172.21.15.43:0/2099742943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cceb7cbd00 con 0x55cceba42690 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec937fe640 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceb9f8830 0x55cceb9f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f82f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0ec790 tx=0x7fec7c0acf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dd89369b41e9e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec897fa640 1 -- 172.21.15.43:0/2099742943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0f46e0 con 0x55cceba42690 2024-03-30T13:13:50.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec897fa640 1 -- 172.21.15.43:0/2099742943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c0f8f10 con 0x55cceba42690 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec897fa640 1 -- 172.21.15.43:0/2099742943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0ee630 con 0x55cceba42690 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- 172.21.15.43:0/2099742943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 msgr2=0x55cceb9f82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f82f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0ec790 tx=0x7fec7c0acf90 comp rx=0 tx=0).stop 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- 172.21.15.43:0/2099742943 shutdown_connections 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceb9f8830 0x55cceb9f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f82f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2099742943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55cceba42150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- 172.21.15.43:0/2099742943 >> 172.21.15.43:0/2099742943 conn(0x55cceb9f47e0 msgr2=0x55cceb9f6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- 172.21.15.43:0/2099742943 shutdown_connections 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- 172.21.15.43:0/2099742943 wait complete. 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- start start 2024-03-30T13:13:50.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 Processor -- start 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55cceb9fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceba26da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba42690 0x55cceba272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cceb80e270 con 0x55cceb9f8830 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cceb8046a0 con 0x55cceba41d70 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55cceb7de880 con 0x55cceba42690 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 10 client.0 _ll_register_callbacks cb 0x55cceb9e3490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55cceb9fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55cceb9fdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60198/0 (socket says 172.21.15.43:60198) 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 -- 172.21.15.43:0/1418635752 learned_addr learned my addr 172.21.15.43:0/1418635752 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceba26da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec937fe640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba42690 0x55cceba272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 -- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba42690 msgr2=0x55cceba272e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba42690 0x55cceba272e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 -- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 msgr2=0x55cceba26da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceba26da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 -- 172.21.15.43:0/1418635752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cceb80b730 con 0x55cceb9f8830 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec937fe640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba42690 0x55cceba272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceba26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec92ffd640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55cceb9fdc80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fec8400eaa0 tx=0x7fec840046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184dec3de66a5e81 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec727fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec727fc640 1 -- 172.21.15.43:0/1418635752 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84027060 con 0x55cceb9f8830 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec727fc640 1 -- 172.21.15.43:0/1418635752 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fec84013430 con 0x55cceb9f8830 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.938+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec7c0f8f10 con 0x55cceb9f8830 2024-03-30T13:13:50.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec727fc640 1 -- 172.21.15.43:0/1418635752 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fec8402d050 con 0x55cceb9f8830 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 10 client.6207 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55cceb7feca0 con 0x55cceb9f8830 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec727fc640 1 -- 172.21.15.43:0/1418635752 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec7c0f8f10 con 0x55cceb9f8830 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec727fc640 1 -- 172.21.15.43:0/1418635752 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fec84030020 con 0x55cceb9f8830 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 10 client.6207 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 10 client.6207 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 20 client.6207 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba495d0 0x55cceba499b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 4 client.6207 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cceb7e1c90 con 0x55cceba495d0 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba495d0 0x55cceba499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba495d0 0x55cceba499b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5bd45c1e36c3e48 server_cookie=b34130882cb0645f in_seq=0 out_seq=0 2024-03-30T13:13:50.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec727fc640 10 client.6207 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec727fc640 1 -- 172.21.15.43:0/1418635752 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55cceb7e1c90 con 0x55cceba495d0 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec727fc640 10 client.6207 handle_command_reply: tid=0 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 client.6207 shutdown 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 msgr2=0x55cceb9fdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55cceb9fdc80 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fec8400eaa0 tx=0x7fec840046d0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 shutdown_connections 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba495d0 0x55cceba499b0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba42690 0x55cceba272e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceba26da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec727fc640 0 client.6207 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1418635752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55cceb9fdc80 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 >> 172.21.15.43:0/1418635752 conn(0x55cceb9f47e0 msgr2=0x55cceb9f7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 shutdown_connections 2024-03-30T13:13:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 1 -- 172.21.15.43:0/1418635752 wait complete. 2024-03-30T13:13:50.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.942+0000 7fec9cae4000 20 client.6207 trim_cache size 0 max 16384 2024-03-30T13:13:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 Processor -- start 2024-03-30T13:13:50.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- start start 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceb9f8830 0x55cceb9f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceb9f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fec94202050 con 0x55cceba42690 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fec94201050 con 0x55cceba41d70 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fec9406e360 con 0x55cceb9f8830 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60208/0 (socket says 172.21.15.43:60208) 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 -- 172.21.15.43:0/2208544981 learned_addr learned my addr 172.21.15.43:0/2208544981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceb9f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec92ffd640 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceb9f8830 0x55cceb9f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 -- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceb9f8830 msgr2=0x55cceb9f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceb9f8830 0x55cceb9f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 -- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 msgr2=0x55cceb9f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceb9f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 -- 172.21.15.43:0/2208544981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cceb7cdd00 con 0x55cceba42690 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec937fe640 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f7650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fec940b3590 tx=0x7fec8c0106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8464a2e9c6ea28 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec88ff9640 1 -- 172.21.15.43:0/2208544981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c010ea0 con 0x55cceba42690 2024-03-30T13:13:50.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec88ff9640 1 -- 172.21.15.43:0/2208544981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c019070 con 0x55cceba42690 2024-03-30T13:13:50.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec88ff9640 1 -- 172.21.15.43:0/2208544981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8c007720 con 0x55cceba42690 2024-03-30T13:13:50.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- 172.21.15.43:0/2208544981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 msgr2=0x55cceb9f7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f7650 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fec940b3590 tx=0x7fec8c0106d0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- 172.21.15.43:0/2208544981 shutdown_connections 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceba42690 0x55cceb9f7650 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba41d70 0x55cceb9f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 --2- 172.21.15.43:0/2208544981 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceb9f8830 0x55cceb9f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- 172.21.15.43:0/2208544981 >> 172.21.15.43:0/2208544981 conn(0x55cceb9f47e0 msgr2=0x55cceba58f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- 172.21.15.43:0/2208544981 shutdown_connections 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- 172.21.15.43:0/2208544981 wait complete. 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.958+0000 7fec9cae4000 1 -- start start 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 Processor -- start 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55ccebab7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55ccebab8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba42690 0x55cceba72d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cceb80e270 con 0x55cceb9f8830 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fec94200a00 con 0x55cceba42690 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55cceb845980 con 0x55cceba41d70 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 10 client.0 _ll_register_callbacks cb 0x7fec84009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55ccebab7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55ccebab7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60212/0 (socket says 172.21.15.43:60212) 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 -- 172.21.15.43:0/1324243021 learned_addr learned my addr 172.21.15.43:0/1324243021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55ccebab8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec937fe640 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba42690 0x55cceba72d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 -- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 msgr2=0x55ccebab8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55ccebab8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 -- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba42690 msgr2=0x55cceba72d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba42690 0x55cceba72d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 -- 172.21.15.43:0/1324243021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cceb8c7f70 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec92ffd640 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55ccebab7c30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fec8400ea70 tx=0x7fec84013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd722abacf1df35 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 1 -- 172.21.15.43:0/1324243021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8400f040 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 1 -- 172.21.15.43:0/1324243021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fec84016420 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec8c019070 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 1 -- 172.21.15.43:0/1324243021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fec84041050 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 10 client.6213 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55cceb7ed760 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 1 -- 172.21.15.43:0/1324243021 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec8c019070 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 1 -- 172.21.15.43:0/1324243021 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fec84030040 con 0x55cceb9f8830 2024-03-30T13:13:50.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 10 client.6213 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 10 client.6213 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 20 client.6213 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba73690 0x55cceba73a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 4 client.6213 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628344}] 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628344}) -- 0x55cceb853fe0 con 0x55cceba73690 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba73690 0x55cceba73a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec8bfff640 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba73690 0x55cceba73a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a39d44de45b63730 server_cookie=fa6654513ebde70b in_seq=0 out_seq=0 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 10 client.6213 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 1 -- 172.21.15.43:0/1324243021 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3831 (crc 0 0 0) 0x55cceb853fe0 con 0x55cceba73690 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 10 client.6213 handle_command_reply: tid=0 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 client.6213 shutdown 2024-03-30T13:13:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 msgr2=0x55ccebab7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55ccebab7c30 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fec8400ea70 tx=0x7fec84013000 comp rx=0 tx=0).stop 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 shutdown_connections 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cceba73690 0x55cceba73a70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cceba42690 0x55cceba72d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cceba41d70 0x55ccebab8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 --2- 172.21.15.43:0/1324243021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cceb9f8830 0x55ccebab7c30 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 >> 172.21.15.43:0/1324243021 conn(0x55cceb9f47e0 msgr2=0x55cceba59620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec727fc640 0 client.6213 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 shutdown_connections 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 1 -- 172.21.15.43:0/1324243021 wait complete. 2024-03-30T13:13:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.962+0000 7fec9cae4000 20 client.6213 trim_cache size 0 max 16384 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-dev-proto.h", 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628344, 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:16.015445+0000", 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:13.011000+0000", 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:50.975 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:50.976 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 2111, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:13.011000+0000", 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 4, 2024-03-30T13:13:50.977 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:50.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2111, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.015445+0000" 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2111, 2024-03-30T13:13:50.979 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.015445+0000" 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 126, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 34, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:50.980 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:50.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.982 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:50.983 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.984 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:50.985 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:50.986 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:50.987 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:50.988 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fec74067fc0 msgr2=0x7fec7406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fec74067fc0 0x7fec7406a480 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fec840097a0 tx=0x7fec84008040 comp rx=0 tx=0).stop 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 msgr2=0x7fec940a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec940a9be0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0af040 tx=0x7fec7c019160 comp rx=0 tx=0).stop 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 shutdown_connections 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fec74067fc0 0x7fec7406a480 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fec94144930 0x7fec9413f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec940b4870 0x7fec940a9be0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 --2- 172.21.15.43:0/2027427186 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fec9406cbb0 0x7fec9413d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 >> 172.21.15.43:0/2027427186 conn(0x7fec940d00a0 msgr2=0x7fec940b00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 shutdown_connections 2024-03-30T13:13:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:50.966+0000 7fec99962640 1 -- 172.21.15.43:0/2027427186 wait complete. 2024-03-30T13:13:50.991 DEBUG:tasks.cephfs.test_uninlining:getting inode info #21 of 23:1099511628329 2024-03-30T13:13:50.991 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628329 2024-03-30T13:13:50.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628329 2024-03-30T13:13:51.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd24a12c640 1 Processor -- start 2024-03-30T13:13:51.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd24a12c640 1 -- start start 2024-03-30T13:13:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd24a12c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2440cf9b0 0x7fd24406b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd24a12c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fd244141890 0x7fd244141d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd24a12c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2440cfea0 con 0x7fd2441414b0 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd24a12c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440d0180 con 0x7fd2440cf9b0 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd24a12c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fd2440d0460 con 0x7fd244141890 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2440cf9b0 0x7fd24406b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2440cf9b0 0x7fd24406b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60214/0 (socket says 172.21.15.43:60214) 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 -- 172.21.15.43:0/1058033605 learned_addr learned my addr 172.21.15.43:0/1058033605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd243fff640 1 --2- 172.21.15.43:0/1058033605 >> v2:172.21.15.113:3300/0 conn(0x7fd244141890 0x7fd244141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 -- 172.21.15.43:0/1058033605 >> v2:172.21.15.113:3300/0 conn(0x7fd244141890 msgr2=0x7fd244141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1058033605 >> v2:172.21.15.113:3300/0 conn(0x7fd244141890 0x7fd244141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 -- 172.21.15.43:0/1058033605 >> v1:172.21.15.113:6789/0 conn(0x7fd2441414b0 legacy=0x7fd2440cf2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 -- 172.21.15.43:0/1058033605 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2440ccbb0 con 0x7fd2440cf9b0 2024-03-30T13:13:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.102+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1058033605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24406b660 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b5ee0 tx=0x7fd23801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154d56c0bbc02e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2427fc640 1 -- 172.21.15.43:0/1058033605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380b0070 con 0x7fd2440cf9b0 2024-03-30T13:13:51.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2427fc640 1 -- 172.21.15.43:0/1058033605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2380ab600 con 0x7fd2440cf9b0 2024-03-30T13:13:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2427fc640 1 -- 172.21.15.43:0/1058033605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380ab9d0 con 0x7fd2440cf9b0 2024-03-30T13:13:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- 172.21.15.43:0/1058033605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 msgr2=0x7fd24406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 --2- 172.21.15.43:0/1058033605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24406b660 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b5ee0 tx=0x7fd23801c520 comp rx=0 tx=0).stop 2024-03-30T13:13:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- 172.21.15.43:0/1058033605 shutdown_connections 2024-03-30T13:13:51.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 --2- 172.21.15.43:0/1058033605 >> v2:172.21.15.113:3300/0 conn(0x7fd244141890 0x7fd244141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 --2- 172.21.15.43:0/1058033605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24406b660 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- 172.21.15.43:0/1058033605 >> 172.21.15.43:0/1058033605 conn(0x7fd2440d0c90 msgr2=0x7fd2440d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- 172.21.15.43:0/1058033605 shutdown_connections 2024-03-30T13:13:51.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- 172.21.15.43:0/1058033605 wait complete. 2024-03-30T13:13:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 Processor -- start 2024-03-30T13:13:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- start start 2024-03-30T13:13:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2441414b0 0x7fd2441387e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244141890 0x7fd2441f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd2440cdc70 con 0x7fd2440cf9b0 2024-03-30T13:13:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd2440cdaf0 con 0x7fd244141890 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd2440cddf0 con 0x7fd2441414b0 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24415f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60216/0 (socket says 172.21.15.43:60216) 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 -- 172.21.15.43:0/3497144590 learned_addr learned my addr 172.21.15.43:0/3497144590 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd243fff640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244141890 0x7fd2441f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 -- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2441414b0 msgr2=0x7fd2441387e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2441414b0 0x7fd2441387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 -- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244141890 msgr2=0x7fd2441f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244141890 0x7fd2441f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 -- 172.21.15.43:0/3497144590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24413bd00 con 0x7fd2440cf9b0 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd2437fe640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24415f0a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c420 tx=0x7fd23400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd22bfff640 1 -- 172.21.15.43:0/3497144590 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2340052d0 con 0x7fd2440cf9b0 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd22bfff640 1 -- 172.21.15.43:0/3497144590 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd234005470 con 0x7fd2440cf9b0 2024-03-30T13:13:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24413bfe0 con 0x7fd2440cf9b0 2024-03-30T13:13:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd22bfff640 1 -- 172.21.15.43:0/3497144590 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234007260 con 0x7fd2440cf9b0 2024-03-30T13:13:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.106+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24413b8e0 con 0x7fd2440cf9b0 2024-03-30T13:13:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd22bfff640 1 -- 172.21.15.43:0/3497144590 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fd234007400 con 0x7fd2440cf9b0 2024-03-30T13:13:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd22bfff640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd230067fc0 0x7fd23006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd22bfff640 1 -- 172.21.15.43:0/3497144590 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fd234085d10 con 0x7fd2440cf9b0 2024-03-30T13:13:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd230067fc0 0x7fd23006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 Processor -- start 2024-03-30T13:13:51.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 -- start start 2024-03-30T13:13:51.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd230067fc0 0x7fd23006a480 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b5ba0 tx=0x7fd238004740 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb05150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb05690 0x56384fabb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384fabbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56384f909aa0 con 0x56384fb05690 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56384f890af0 con 0x56384fabb830 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56384f905a50 con 0x56384fb04d70 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb05690 0x56384fabb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb05690 0x56384fabb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60220/0 (socket says 172.21.15.43:60220) 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 -- 172.21.15.43:0/1971837780 learned_addr learned my addr 172.21.15.43:0/1971837780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd243fff640 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384fabbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd2437fe640 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb05150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 -- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 msgr2=0x56384fb05150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb05150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 -- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 msgr2=0x56384fabbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384fabbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 -- 172.21.15.43:0/1971837780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56384f88ed00 con 0x56384fb05690 2024-03-30T13:13:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb05690 0x56384fabb2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd23801a150 tx=0x7fd2380a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=855675c9c9467d0f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.110+0000 7fd2297fa640 1 -- 172.21.15.43:0/1971837780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380b0040 con 0x56384fb05690 2024-03-30T13:13:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2297fa640 1 -- 172.21.15.43:0/1971837780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2380b4bd0 con 0x56384fb05690 2024-03-30T13:13:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2297fa640 1 -- 172.21.15.43:0/1971837780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238004030 con 0x56384fb05690 2024-03-30T13:13:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1971837780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb05690 msgr2=0x56384fabb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb05690 0x56384fabb2f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd23801a150 tx=0x7fd2380a8000 comp rx=0 tx=0).stop 2024-03-30T13:13:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1971837780 shutdown_connections 2024-03-30T13:13:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384fabbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb05690 0x56384fabb2f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1971837780 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1971837780 >> 172.21.15.43:0/1971837780 conn(0x56384fab77e0 msgr2=0x56384fab9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1971837780 shutdown_connections 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1971837780 wait complete. 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- start start 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 Processor -- start 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fac0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fae9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384faea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56384f8d1270 con 0x56384fabb830 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56384f8c76a0 con 0x56384fb05690 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56384f8a1880 con 0x56384fb04d70 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 10 client.0 _ll_register_callbacks cb 0x56384faa6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fac0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fac0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60236/0 (socket says 172.21.15.43:60236) 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 -- 172.21.15.43:0/585814531 learned_addr learned my addr 172.21.15.43:0/585814531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd243fff640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384faea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd242ffd640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fae9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 -- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 msgr2=0x56384fae9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fae9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 -- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 msgr2=0x56384faea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384faea2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 -- 172.21.15.43:0/585814531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56384f8ce730 con 0x56384fabb830 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd243fff640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384faea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2437fe640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fac0c40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd23405e210 tx=0x7fd234055c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1357952f6a972e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2227fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2227fc640 1 -- 172.21.15.43:0/585814531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234082060 con 0x56384fabb830 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2227fc640 1 -- 172.21.15.43:0/585814531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fd234083a70 con 0x56384fabb830 2024-03-30T13:13:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2380b4bd0 con 0x56384fabb830 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2227fc640 1 -- 172.21.15.43:0/585814531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd234073050 con 0x56384fabb830 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 10 client.6225 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56384f8c1ca0 con 0x56384fabb830 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2227fc640 1 -- 172.21.15.43:0/585814531 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2380b4bd0 con 0x56384fabb830 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd2227fc640 1 -- 172.21.15.43:0/585814531 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fd234074240 con 0x56384fabb830 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 10 client.6225 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 10 client.6225 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 20 client.6225 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb0c5d0 0x56384fb0c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 4 client.6225 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56384f8a4c90 con 0x56384fb0c5d0 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.114+0000 7fd242ffd640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb0c5d0 0x56384fb0c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd2227fc640 10 client.6225 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd242ffd640 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb0c5d0 0x56384fb0c9b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6415eb0228f80c server_cookie=d44fa41fa004f9df in_seq=0 out_seq=0 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd2227fc640 1 -- 172.21.15.43:0/585814531 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x56384f8a4c90 con 0x56384fb0c5d0 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd2227fc640 10 client.6225 handle_command_reply: tid=0 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 client.6225 shutdown 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 msgr2=0x56384fac0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fac0c40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fd23405e210 tx=0x7fd234055c10 comp rx=0 tx=0).stop 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 shutdown_connections 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb0c5d0 0x56384fb0c9b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384faea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fae9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd2227fc640 0 client.6225 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/585814531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fac0c40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 >> 172.21.15.43:0/585814531 conn(0x56384fab77e0 msgr2=0x56384faba460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 shutdown_connections 2024-03-30T13:13:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 1 -- 172.21.15.43:0/585814531 wait complete. 2024-03-30T13:13:51.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.118+0000 7fd24d2ae000 20 client.6225 trim_cache size 0 max 16384 2024-03-30T13:13:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.130+0000 7fd24d2ae000 1 Processor -- start 2024-03-30T13:13:51.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.130+0000 7fd24d2ae000 1 -- start start 2024-03-30T13:13:51.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384faea180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb04d70 0x56384fab9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb05690 0x56384faba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fd244201bc0 con 0x56384fb04d70 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd244200ca0 con 0x56384fabb830 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fd24413a530 con 0x56384fb05690 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb04d70 0x56384fab9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb04d70 0x56384fab9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60250/0 (socket says 172.21.15.43:60250) 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 -- 172.21.15.43:0/4099879896 learned_addr learned my addr 172.21.15.43:0/4099879896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384faea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd243fff640 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb05690 0x56384faba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 -- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb05690 msgr2=0x56384faba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb05690 0x56384faba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 -- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 msgr2=0x56384faea180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384faea180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 -- 172.21.15.43:0/4099879896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56384f890d00 con 0x56384fb04d70 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb04d70 0x56384fab9bd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd23801c560 tx=0x7fd238008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594e04a53cef6a8b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd228ff9640 1 -- 172.21.15.43:0/4099879896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd238004030 con 0x56384fb04d70 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd228ff9640 1 -- 172.21.15.43:0/4099879896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2380041d0 con 0x56384fb04d70 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd228ff9640 1 -- 172.21.15.43:0/4099879896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2380ce170 con 0x56384fb04d70 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- 172.21.15.43:0/4099879896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb04d70 msgr2=0x56384fab9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb04d70 0x56384fab9bd0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd23801c560 tx=0x7fd238008070 comp rx=0 tx=0).stop 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- 172.21.15.43:0/4099879896 shutdown_connections 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb05690 0x56384faba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fb04d70 0x56384fab9bd0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/4099879896 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fabb830 0x56384faea180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- 172.21.15.43:0/4099879896 >> 172.21.15.43:0/4099879896 conn(0x56384fab77e0 msgr2=0x56384fb1bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- 172.21.15.43:0/4099879896 shutdown_connections 2024-03-30T13:13:51.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- 172.21.15.43:0/4099879896 wait complete. 2024-03-30T13:13:51.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- start start 2024-03-30T13:13:51.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 Processor -- start 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fb72f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb08ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384fb09010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56384f8a1880 con 0x56384fabb830 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fd24413ab80 con 0x56384fb05690 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56384f8d1270 con 0x56384fb04d70 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 10 client.0 _ll_register_callbacks cb 0x7fd234055920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fb72f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fb72f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60256/0 (socket says 172.21.15.43:60256) 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 -- 172.21.15.43:0/1574485072 learned_addr learned my addr 172.21.15.43:0/1574485072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd243fff640 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384fb09010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 -- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 msgr2=0x56384fb08ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb08ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 -- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 msgr2=0x56384fb09010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384fb09010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 -- 172.21.15.43:0/1574485072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56384f98af70 con 0x56384fabb830 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2437fe640 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fb72f20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd23400cde0 tx=0x7fd234083eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cafcc6440c3aa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2227fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2227fc640 1 -- 172.21.15.43:0/1574485072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234078060 con 0x56384fabb830 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd2227fc640 1 -- 172.21.15.43:0/1574485072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fd234051a30 con 0x56384fabb830 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.134+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2380041d0 con 0x56384fabb830 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd2227fc640 1 -- 172.21.15.43:0/1574485072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fd23405e740 con 0x56384fabb830 2024-03-30T13:13:51.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 10 client.6231 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x56384f8b0760 con 0x56384fabb830 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd2227fc640 1 -- 172.21.15.43:0/1574485072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2380041d0 con 0x56384fabb830 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd2227fc640 1 -- 172.21.15.43:0/1574485072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7fd23407d420 con 0x56384fabb830 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 10 client.6231 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 10 client.6231 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 20 client.6231 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb09550 0x56384fb09960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 4 client.6231 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628329}] 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628329}) -- 0x56384f916fe0 con 0x56384fb09550 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb09550 0x56384fb09960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd2227fc640 10 client.6231 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd242ffd640 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb09550 0x56384fb09960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c579a59654e8fa68 server_cookie=204e8c9b88a8dbe3 in_seq=0 out_seq=0 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd2227fc640 1 -- 172.21.15.43:0/1574485072 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3815 (crc 0 0 0) 0x56384f916fe0 con 0x56384fb09550 2024-03-30T13:13:51.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd2227fc640 10 client.6231 handle_command_reply: tid=0 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 client.6231 shutdown 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 msgr2=0x56384fb72f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fb72f20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd23400cde0 tx=0x7fd234083eb0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 shutdown_connections 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56384fb09550 0x56384fb09960 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56384fb05690 0x56384fb09010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56384fb04d70 0x56384fb08ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd2227fc640 0 client.6231 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 --2- 172.21.15.43:0/1574485072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56384fabb830 0x56384fb72f20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 >> 172.21.15.43:0/1574485072 conn(0x56384fab77e0 msgr2=0x56384fb1c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 shutdown_connections 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 1 -- 172.21.15.43:0/1574485072 wait complete. 2024-03-30T13:13:51.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24d2ae000 20 client.6231 trim_cache size 0 max 16384 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/Makefile", 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628329, 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:16.016531+0000", 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.896323+0000", 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:51.148 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 356, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:51.149 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.896323+0000", 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.150 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.016531+0000" 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.151 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.016531+0000" 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 128, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 4, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:51.152 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:51.153 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:51.154 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:51.155 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.156 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.157 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:51.158 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:51.159 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd230067fc0 msgr2=0x7fd23006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd230067fc0 0x7fd23006a480 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b5ba0 tx=0x7fd238004740 comp rx=0 tx=0).stop 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 msgr2=0x7fd24415f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24415f0a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c420 tx=0x7fd23400c8e0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 shutdown_connections 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fd230067fc0 0x7fd23006a480 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fd244141890 0x7fd2441f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fd2441414b0 0x7fd2441387e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 --2- 172.21.15.43:0/3497144590 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf9b0 0x7fd24415f0a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 >> 172.21.15.43:0/3497144590 conn(0x7fd2440d0c90 msgr2=0x7fd2441602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 shutdown_connections 2024-03-30T13:13:51.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.138+0000 7fd24a12c640 1 -- 172.21.15.43:0/3497144590 wait complete. 2024-03-30T13:13:51.161 DEBUG:tasks.cephfs.test_uninlining:getting inode info #22 of 23:1099511628352 2024-03-30T13:13:51.162 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628352 2024-03-30T13:13:51.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628352 2024-03-30T13:13:51.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.270+0000 7f1989822640 1 Processor -- start 2024-03-30T13:13:51.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.270+0000 7f1989822640 1 -- start start 2024-03-30T13:13:51.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1989822640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f198413f140 0x7f19841606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1989822640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198415ff40 0x7f198406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1989822640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198406d710 con 0x7f198406cbb0 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1989822640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f198406d000 con 0x7f198413f140 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1989822640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f198406e320 con 0x7f198415ff40 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f198413f140 0x7f19841606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f198413f140 0x7f19841606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37146/0 (socket says 172.21.15.43:37146) 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 -- 172.21.15.43:0/2236480866 learned_addr learned my addr 172.21.15.43:0/2236480866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f19827fc640 1 --2- 172.21.15.43:0/2236480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198415ff40 0x7f198406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 -- 172.21.15.43:0/2236480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198415ff40 msgr2=0x7f198406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 --2- 172.21.15.43:0/2236480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198415ff40 0x7f198406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 -- 172.21.15.43:0/2236480866 >> v1:172.21.15.113:6789/0 conn(0x7f198406cbb0 legacy=0x7f1984141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 -- 172.21.15.43:0/2236480866 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198406d9f0 con 0x7f198413f140 2024-03-30T13:13:51.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f19827fc640 1 --2- 172.21.15.43:0/2236480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198415ff40 0x7f198406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1982ffd640 1 --2- 172.21.15.43:0/2236480866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198413f140 0x7f19841606d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1974009870 tx=0x7f197401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd7e5b3f0f64084 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1981ffb640 1 -- 172.21.15.43:0/2236480866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740a95f0 con 0x7f198413f140 2024-03-30T13:13:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1981ffb640 1 -- 172.21.15.43:0/2236480866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740a9790 con 0x7f198413f140 2024-03-30T13:13:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.274+0000 7f1981ffb640 1 -- 172.21.15.43:0/2236480866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740a9b60 con 0x7f198413f140 2024-03-30T13:13:51.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- 172.21.15.43:0/2236480866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198413f140 msgr2=0x7f19841606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 --2- 172.21.15.43:0/2236480866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198413f140 0x7f19841606d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1974009870 tx=0x7f197401c530 comp rx=0 tx=0).stop 2024-03-30T13:13:51.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- 172.21.15.43:0/2236480866 shutdown_connections 2024-03-30T13:13:51.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 --2- 172.21.15.43:0/2236480866 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198415ff40 0x7f198406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 --2- 172.21.15.43:0/2236480866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198413f140 0x7f19841606d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- 172.21.15.43:0/2236480866 >> 172.21.15.43:0/2236480866 conn(0x7f19840d00a0 msgr2=0x7f19840d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- 172.21.15.43:0/2236480866 shutdown_connections 2024-03-30T13:13:51.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- 172.21.15.43:0/2236480866 wait complete. 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 Processor -- start 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- start start 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198406cbb0 0x7f198413d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198413f140 0x7f1984072550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f198415ff40 0x7f19840b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1984160d90 con 0x7f198406cbb0 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1984160c10 con 0x7f198415ff40 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1984168ff0 con 0x7f198413f140 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198406cbb0 0x7f198413d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198406cbb0 0x7f198413d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37154/0 (socket says 172.21.15.43:37154) 2024-03-30T13:13:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 -- 172.21.15.43:0/1676977959 learned_addr learned my addr 172.21.15.43:0/1676977959 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f19837fe640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f198415ff40 0x7f19840b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f19827fc640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198413f140 0x7f1984072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 -- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198413f140 msgr2=0x7f1984072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198413f140 0x7f1984072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 -- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f198415ff40 msgr2=0x7f19840b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f198415ff40 0x7f19840b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 -- 172.21.15.43:0/1676977959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198406c9e0 con 0x7f198406cbb0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f19827fc640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198413f140 0x7f1984072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1982ffd640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198406cbb0 0x7f198413d4d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f19740ab040 tx=0x7f197401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f196bfff640 1 -- 172.21.15.43:0/1676977959 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740b1880 con 0x7f198406cbb0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f196bfff640 1 -- 172.21.15.43:0/1676977959 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740b1a20 con 0x7f198406cbb0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1984156040 con 0x7f198406cbb0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f196bfff640 1 -- 172.21.15.43:0/1676977959 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740b1df0 con 0x7f198406cbb0 2024-03-30T13:13:51.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.278+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1984156600 con 0x7f198406cbb0 2024-03-30T13:13:51.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f196bfff640 1 -- 172.21.15.43:0/1676977959 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f19740bc450 con 0x7f198406cbb0 2024-03-30T13:13:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f196bfff640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1970067fc0 0x7f197006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f196bfff640 1 -- 172.21.15.43:0/1676977959 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f1974121270 con 0x7f198406cbb0 2024-03-30T13:13:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f19827fc640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1970067fc0 0x7f197006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 Processor -- start 2024-03-30T13:13:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 -- start start 2024-03-30T13:13:51.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f19827fc640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1970067fc0 0x7f197006a480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f19841f8900 tx=0x7f1978008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c286932fc0 0x55c2869333a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 0x55c2868e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c2868ea020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c286733aa0 con 0x55c286932fc0 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c2866baaf0 con 0x55c2869338e0 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c28672fa50 con 0x55c2868e9c40 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c286932fc0 0x55c2869333a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c286932fc0 0x55c2869333a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37170/0 (socket says 172.21.15.43:37170) 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 -- 172.21.15.43:0/4279517335 learned_addr learned my addr 172.21.15.43:0/4279517335 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f19827fc640 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 0x55c2868e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f19837fe640 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c2868ea020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 -- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 msgr2=0x55c2868ea020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c2868ea020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 -- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 msgr2=0x55c2868e9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 0x55c2868e9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 -- 172.21.15.43:0/4279517335 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2866b8d00 con 0x55c286932fc0 2024-03-30T13:13:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f1982ffd640 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c286932fc0 0x55c2869333a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f197411f670 tx=0x7f19740a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8aece53761fbcfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f19697fa640 1 -- 172.21.15.43:0/4279517335 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740f55a0 con 0x55c286932fc0 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f19697fa640 1 -- 172.21.15.43:0/4279517335 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740f9dd0 con 0x55c286932fc0 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.282+0000 7f19697fa640 1 -- 172.21.15.43:0/4279517335 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740ef5a0 con 0x55c286932fc0 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/4279517335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c286932fc0 msgr2=0x55c2869333a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c286932fc0 0x55c2869333a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f197411f670 tx=0x7f19740a9000 comp rx=0 tx=0).stop 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/4279517335 shutdown_connections 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c2868ea020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 0x55c2868e9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- 172.21.15.43:0/4279517335 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c286932fc0 0x55c2869333a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/4279517335 >> 172.21.15.43:0/4279517335 conn(0x55c2868e17e0 msgr2=0x55c2868e3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/4279517335 shutdown_connections 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/4279517335 wait complete. 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- start start 2024-03-30T13:13:51.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 Processor -- start 2024-03-30T13:13:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c286932890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c286932fc0 0x55c2868ef910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c2868efe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c2866fb270 con 0x55c2869338e0 2024-03-30T13:13:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c2866f16a0 con 0x55c286932fc0 2024-03-30T13:13:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c2866cb880 con 0x55c2868e9c40 2024-03-30T13:13:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 10 client.0 _ll_register_callbacks cb 0x55c2868d0490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c2868efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c2868efe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37176/0 (socket says 172.21.15.43:37176) 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 -- 172.21.15.43:0/1920644470 learned_addr learned my addr 172.21.15.43:0/1920644470 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f1982ffd640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c286932890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19827fc640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c286932fc0 0x55c2868ef910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 -- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 msgr2=0x55c286932890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c286932890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 -- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c286932fc0 msgr2=0x55c2868ef910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c286932fc0 0x55c2868ef910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 -- 172.21.15.43:0/1920644470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2866f8730 con 0x55c2869338e0 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19827fc640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c286932fc0 0x55c2868ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19837fe640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c2868efe50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f197c005950 tx=0x7f197c001880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5875fb15f4cce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19627fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:51.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19627fc640 1 -- 172.21.15.43:0/1920644470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c014400 con 0x55c2869338e0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19627fc640 1 -- 172.21.15.43:0/1920644470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f197c0145a0 con 0x55c2869338e0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19740f9dd0 con 0x55c2869338e0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19627fc640 1 -- 172.21.15.43:0/1920644470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f197c021050 con 0x55c2869338e0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 10 client.6243 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55c2866ebca0 con 0x55c2869338e0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19627fc640 1 -- 172.21.15.43:0/1920644470 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740f9dd0 con 0x55c2869338e0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19627fc640 1 -- 172.21.15.43:0/1920644470 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f197c024020 con 0x55c2869338e0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 10 client.6243 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 10 client.6243 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 20 client.6243 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c2868f0bc0 0x55c286945280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 4 client.6243 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2866cec90 con 0x55c2868f0bc0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f1982ffd640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c2868f0bc0 0x55c286945280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f1982ffd640 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c2868f0bc0 0x55c286945280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b6f2ab791355d5 server_cookie=75571c07fe87f2eb in_seq=0 out_seq=0 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.286+0000 7f19627fc640 10 client.6243 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f19627fc640 1 -- 172.21.15.43:0/1920644470 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55c2866cec90 con 0x55c2868f0bc0 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f19627fc640 10 client.6243 handle_command_reply: tid=0 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 client.6243 shutdown 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 msgr2=0x55c2868efe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c2868efe50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f197c005950 tx=0x7f197c001880 comp rx=0 tx=0).stop 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 shutdown_connections 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c2868f0bc0 0x55c286945280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c2868efe50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c286932fc0 0x55c2868ef910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f19627fc640 0 client.6243 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 --2- 172.21.15.43:0/1920644470 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c2868e9c40 0x55c286932890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 >> 172.21.15.43:0/1920644470 conn(0x55c2868e17e0 msgr2=0x55c2868ea8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 shutdown_connections 2024-03-30T13:13:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 1 -- 172.21.15.43:0/1920644470 wait complete. 2024-03-30T13:13:51.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.290+0000 7f198c9a4000 20 client.6243 trim_cache size 0 max 16384 2024-03-30T13:13:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.302+0000 7f198c9a4000 1 Processor -- start 2024-03-30T13:13:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.302+0000 7f198c9a4000 1 -- start start 2024-03-30T13:13:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2868e9c40 0x55c2869122f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 0x55c2869487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f19841ffdb0 con 0x55c2868e9c40 2024-03-30T13:13:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f19840b4590 con 0x55c2869338e0 2024-03-30T13:13:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f198414ea10 con 0x55c286932fc0 2024-03-30T13:13:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2868e9c40 0x55c2869122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2868e9c40 0x55c2869122f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37190/0 (socket says 172.21.15.43:37190) 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 -- 172.21.15.43:0/2667988589 learned_addr learned my addr 172.21.15.43:0/2667988589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 -- 172.21.15.43:0/2667988589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 msgr2=0x55c2869482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 --2- 172.21.15.43:0/2667988589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 -- 172.21.15.43:0/2667988589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 msgr2=0x55c2869487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 --2- 172.21.15.43:0/2667988589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 0x55c2869487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 -- 172.21.15.43:0/2667988589 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2866bad00 con 0x55c2868e9c40 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 --2- 172.21.15.43:0/2667988589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2868e9c40 0x55c2869122f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f197411f6c0 tx=0x7f197401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef6793d534c5559 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1968ff9640 1 -- 172.21.15.43:0/2667988589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197411d070 con 0x55c2868e9c40 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1968ff9640 1 -- 172.21.15.43:0/2667988589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197411f040 con 0x55c2868e9c40 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1968ff9640 1 -- 172.21.15.43:0/2667988589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19740ef5a0 con 0x55c2868e9c40 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- 172.21.15.43:0/2667988589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2868e9c40 msgr2=0x55c2869122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- 172.21.15.43:0/2667988589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2868e9c40 0x55c2869122f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f197411f6c0 tx=0x7f197401c530 comp rx=0 tx=0).stop 2024-03-30T13:13:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- 172.21.15.43:0/2667988589 shutdown_connections 2024-03-30T13:13:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- 172.21.15.43:0/2667988589 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2869338e0 0x55c2869487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- 172.21.15.43:0/2667988589 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- 172.21.15.43:0/2667988589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2868e9c40 0x55c2869122f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- 172.21.15.43:0/2667988589 >> 172.21.15.43:0/2667988589 conn(0x55c2868e17e0 msgr2=0x55c286912d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- 172.21.15.43:0/2667988589 shutdown_connections 2024-03-30T13:13:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- 172.21.15.43:0/2667988589 wait complete. 2024-03-30T13:13:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- start start 2024-03-30T13:13:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 Processor -- start 2024-03-30T13:13:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2868e9c40 0x55c286911f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c286946810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c2866fb270 con 0x55c2869338e0 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f198414f060 con 0x55c2868e9c40 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c2866bea30 con 0x55c286932fc0 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 10 client.0 _ll_register_callbacks cb 0x7f197c003b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c286946810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c286946810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37206/0 (socket says 172.21.15.43:37206) 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 -- 172.21.15.43:0/198338453 learned_addr learned my addr 172.21.15.43:0/198338453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f1982ffd640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2868e9c40 0x55c286911f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19827fc640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 -- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 msgr2=0x55c2869462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 -- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2868e9c40 msgr2=0x55c286911f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2868e9c40 0x55c286911f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 -- 172.21.15.43:0/198338453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2867b4f70 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19827fc640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19837fe640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c286946810 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f197c000f90 tx=0x7f197c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e354c43e1713512 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19627fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19627fc640 1 -- 172.21.15.43:0/198338453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c001d80 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f19627fc640 1 -- 172.21.15.43:0/198338453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f197c025600 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.306+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f197411f040 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f19627fc640 1 -- 172.21.15.43:0/198338453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f197c036050 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 10 client.6249 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55c2866da760 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f19627fc640 1 -- 172.21.15.43:0/198338453 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197411f040 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f19627fc640 1 -- 172.21.15.43:0/198338453 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7f197c039020 con 0x55c2869338e0 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 10 client.6249 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 10 client.6249 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 20 client.6249 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c286946d50 0x55c2869a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 4 client.6249 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628352}] 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628352}) -- 0x55c286740fe0 con 0x55c286946d50 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1982ffd640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c286946d50 0x55c2869a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1982ffd640 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c286946d50 0x55c2869a4580 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6822bb17dffd1eb server_cookie=91d908e8b84dedda in_seq=0 out_seq=0 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f19627fc640 10 client.6249 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f19627fc640 1 -- 172.21.15.43:0/198338453 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3817 (crc 0 0 0) 0x55c286740fe0 con 0x55c286946d50 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f19627fc640 10 client.6249 handle_command_reply: tid=0 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 client.6249 shutdown 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 msgr2=0x55c286946810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c286946810 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f197c000f90 tx=0x7f197c001260 comp rx=0 tx=0).stop 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 shutdown_connections 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c286946d50 0x55c2869a4580 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c2869338e0 0x55c286946810 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c286932fc0 0x55c2869462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f19627fc640 0 client.6249 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 --2- 172.21.15.43:0/198338453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c2868e9c40 0x55c286911f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 >> 172.21.15.43:0/198338453 conn(0x55c2868e17e0 msgr2=0x55c2869135e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 shutdown_connections 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 1 -- 172.21.15.43:0/198338453 wait complete. 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f198c9a4000 20 client.6249 trim_cache size 0 max 16384 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/symlink.c", 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628352, 2024-03-30T13:13:51.319 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:16.017838+0000", 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:13.068364+0000", 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:51.320 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 497, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:13.068364+0000", 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:51.321 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.322 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 497, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.017838+0000" 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 497, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.017838+0000" 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 130, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:51.323 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 50, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:51.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.325 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.326 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:51.327 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:51.328 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:51.329 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:51.330 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1970067fc0 msgr2=0x7f197006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1970067fc0 0x7f197006a480 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f19841f8900 tx=0x7f1978008040 comp rx=0 tx=0).stop 2024-03-30T13:13:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198406cbb0 msgr2=0x7f198413d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198406cbb0 0x7f198413d4d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f19740ab040 tx=0x7f197401ad70 comp rx=0 tx=0).stop 2024-03-30T13:13:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 shutdown_connections 2024-03-30T13:13:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1970067fc0 0x7f197006a480 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f198415ff40 0x7f19840b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f198413f140 0x7f1984072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 --2- 172.21.15.43:0/1676977959 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f198406cbb0 0x7f198413d4d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 >> 172.21.15.43:0/1676977959 conn(0x7f19840d00a0 msgr2=0x7f19841500f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 shutdown_connections 2024-03-30T13:13:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.310+0000 7f1989822640 1 -- 172.21.15.43:0/1676977959 wait complete. 2024-03-30T13:13:51.332 DEBUG:tasks.cephfs.test_uninlining:getting inode info #23 of 23:1099511628334 2024-03-30T13:13:51.332 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628334 2024-03-30T13:13:51.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628334 2024-03-30T13:13:51.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.438+0000 7eff47259640 1 Processor -- start 2024-03-30T13:13:51.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.438+0000 7eff47259640 1 -- start start 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff47259640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff40144930 0x7eff400b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff47259640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff400ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff47259640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff400b33c0 con 0x7eff400b4870 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff47259640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7eff400cdb90 con 0x7eff40144930 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff47259640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7eff400b3660 con 0x7eff4006cd80 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff40144930 0x7eff400b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff40144930 0x7eff400b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37214/0 (socket says 172.21.15.43:37214) 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 -- 172.21.15.43:0/4016488036 learned_addr learned my addr 172.21.15.43:0/4016488036 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 -- 172.21.15.43:0/4016488036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 msgr2=0x7eff400ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff457f6640 1 --2- 172.21.15.43:0/4016488036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff400ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 --2- 172.21.15.43:0/4016488036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff400ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 -- 172.21.15.43:0/4016488036 >> v1:172.21.15.113:6789/0 conn(0x7eff400b4870 legacy=0x7eff4006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 -- 172.21.15.43:0/4016488036 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff400b3940 con 0x7eff40144930 2024-03-30T13:13:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff44ff5640 1 --2- 172.21.15.43:0/4016488036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff40144930 0x7eff400b2e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7eff34009870 tx=0x7eff3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f290516ffae450 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff3f7fe640 1 -- 172.21.15.43:0/4016488036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340ab660 con 0x7eff40144930 2024-03-30T13:13:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff3f7fe640 1 -- 172.21.15.43:0/4016488036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff340b0070 con 0x7eff40144930 2024-03-30T13:13:51.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.442+0000 7eff3f7fe640 1 -- 172.21.15.43:0/4016488036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340aba20 con 0x7eff40144930 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- 172.21.15.43:0/4016488036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff40144930 msgr2=0x7eff400b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 --2- 172.21.15.43:0/4016488036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff40144930 0x7eff400b2e80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7eff34009870 tx=0x7eff3401c4a0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- 172.21.15.43:0/4016488036 shutdown_connections 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 --2- 172.21.15.43:0/4016488036 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff400ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 --2- 172.21.15.43:0/4016488036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff40144930 0x7eff400b2e80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- 172.21.15.43:0/4016488036 >> 172.21.15.43:0/4016488036 conn(0x7eff400d00a0 msgr2=0x7eff400d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- 172.21.15.43:0/4016488036 shutdown_connections 2024-03-30T13:13:51.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- 172.21.15.43:0/4016488036 wait complete. 2024-03-30T13:13:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 Processor -- start 2024-03-30T13:13:51.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- start start 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff4015a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff400b4870 0x7eff401f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff40144930 0x7eff400a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff400ceef0 con 0x7eff400b4870 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff400ced70 con 0x7eff40144930 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7eff400cf070 con 0x7eff4006cd80 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff400b4870 0x7eff401f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff400b4870 0x7eff401f7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37216/0 (socket says 172.21.15.43:37216) 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 -- 172.21.15.43:0/3102235694 learned_addr learned my addr 172.21.15.43:0/3102235694 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff457f6640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff40144930 0x7eff400a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff4015a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 -- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 msgr2=0x7eff4015a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff4015a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 -- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff40144930 msgr2=0x7eff400a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff40144930 0x7eff400a94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 -- 172.21.15.43:0/3102235694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff401f68f0 con 0x7eff400b4870 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff4015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3ffff640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff400b4870 0x7eff401f7990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007fd0 tx=0x7eff2c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3d7fa640 1 -- 172.21.15.43:0/3102235694 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c017070 con 0x7eff400b4870 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3d7fa640 1 -- 172.21.15.43:0/3102235694 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff2c0052d0 con 0x7eff400b4870 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff401f6bd0 con 0x7eff400b4870 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff3d7fa640 1 -- 172.21.15.43:0/3102235694 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c003840 con 0x7eff400b4870 2024-03-30T13:13:51.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.446+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff401f7190 con 0x7eff400b4870 2024-03-30T13:13:51.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff3d7fa640 1 -- 172.21.15.43:0/3102235694 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7eff2c01a070 con 0x7eff400b4870 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff3d7fa640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7eff24067fc0 0x7eff2406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff3d7fa640 1 -- 172.21.15.43:0/3102235694 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7eff2c085a90 con 0x7eff400b4870 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7eff24067fc0 0x7eff2406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 Processor -- start 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- start start 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7eff24067fc0 0x7eff2406a480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7eff340af000 tx=0x7eff34008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c717150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55942c51baa0 con 0x55942c6cd830 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55942c4a2af0 con 0x55942c717690 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55942c517a50 con 0x55942c716d70 2024-03-30T13:13:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37224/0 (socket says 172.21.15.43:37224) 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 -- 172.21.15.43:0/2894189921 learned_addr learned my addr 172.21.15.43:0/2894189921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff44ff5640 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c717150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff3ffff640 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 -- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 msgr2=0x55942c717150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c717150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 -- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 msgr2=0x55942c6cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 -- 172.21.15.43:0/2894189921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942c4a0d00 con 0x55942c6cd830 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff44ff5640 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c717150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff457f6640 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cdc10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7eff38007fd0 tx=0x7eff3800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bd9e74f8e81610 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff2a7fc640 1 -- 172.21.15.43:0/2894189921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3800f6c0 con 0x55942c6cd830 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff2a7fc640 1 -- 172.21.15.43:0/2894189921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3800f860 con 0x55942c6cd830 2024-03-30T13:13:51.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff2a7fc640 1 -- 172.21.15.43:0/2894189921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3800fc30 con 0x55942c6cd830 2024-03-30T13:13:51.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- 172.21.15.43:0/2894189921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 msgr2=0x55942c6cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cdc10 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7eff38007fd0 tx=0x7eff3800e820 comp rx=0 tx=0).stop 2024-03-30T13:13:51.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- 172.21.15.43:0/2894189921 shutdown_connections 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cdc10 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 --2- 172.21.15.43:0/2894189921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c717150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- 172.21.15.43:0/2894189921 >> 172.21.15.43:0/2894189921 conn(0x55942c6c97e0 msgr2=0x55942c6cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- 172.21.15.43:0/2894189921 shutdown_connections 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.450+0000 7eff4a3db000 1 -- 172.21.15.43:0/2894189921 wait complete. 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 -- start start 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 Processor -- start 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c6fbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c6fc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55942c4e3270 con 0x55942c6cd830 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55942c4d96a0 con 0x55942c716d70 2024-03-30T13:13:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55942c4b3880 con 0x55942c717690 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 10 client.0 _ll_register_callbacks cb 0x55942c6b8490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6d2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37230/0 (socket says 172.21.15.43:37230) 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 -- 172.21.15.43:0/3138730987 learned_addr learned my addr 172.21.15.43:0/3138730987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff3ffff640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c6fbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff457f6640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c6fc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 -- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 msgr2=0x55942c6fc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c6fc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 -- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 msgr2=0x55942c6fbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c6fbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 -- 172.21.15.43:0/3138730987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942c4e0730 con 0x55942c6cd830 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff3ffff640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c6fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6d2c10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7eff3401c4e0 tx=0x7eff340046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8206e43d1eb27c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff1f7fe640 1 -- 172.21.15.43:0/3138730987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340c13f0 con 0x55942c6cd830 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff1f7fe640 1 -- 172.21.15.43:0/3138730987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7eff340c1590 con 0x55942c6cd830 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff3800f860 con 0x55942c6cd830 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff1f7fe640 1 -- 172.21.15.43:0/3138730987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7eff340ce050 con 0x55942c6cd830 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 10 client.6261 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55942c4d3ca0 con 0x55942c6cd830 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff1f7fe640 1 -- 172.21.15.43:0/3138730987 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3800f860 con 0x55942c6cd830 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff1f7fe640 1 -- 172.21.15.43:0/3138730987 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7eff340d1020 con 0x55942c6cd830 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 10 client.6261 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 10 client.6261 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 20 client.6261 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c71e5d0 0x55942c71e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 4 client.6261 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:51.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55942c4b6c90 con 0x55942c71e5d0 2024-03-30T13:13:51.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff3ffff640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c71e5d0 0x55942c71e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff1f7fe640 10 client.6261 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.454+0000 7eff3ffff640 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c71e5d0 0x55942c71e9b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73344e3d9129d4b1 server_cookie=1df2ffb40119320 in_seq=0 out_seq=0 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff1f7fe640 1 -- 172.21.15.43:0/3138730987 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55942c4b6c90 con 0x55942c71e5d0 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff1f7fe640 10 client.6261 handle_command_reply: tid=0 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 client.6261 shutdown 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 msgr2=0x55942c6d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6d2c10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7eff3401c4e0 tx=0x7eff340046d0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 shutdown_connections 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c71e5d0 0x55942c71e9b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c6fc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c6fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3138730987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6d2c10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff1f7fe640 0 client.6261 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 >> 172.21.15.43:0/3138730987 conn(0x55942c6c97e0 msgr2=0x55942c6cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 shutdown_connections 2024-03-30T13:13:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 1 -- 172.21.15.43:0/3138730987 wait complete. 2024-03-30T13:13:51.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.458+0000 7eff4a3db000 20 client.6261 trim_cache size 0 max 16384 2024-03-30T13:13:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 Processor -- start 2024-03-30T13:13:51.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 -- start start 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c6cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7eff40201c70 con 0x55942c6cd830 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff40200cc0 con 0x55942c717690 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7eff401f6210 con 0x55942c716d70 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37236/0 (socket says 172.21.15.43:37236) 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 -- 172.21.15.43:0/3539924469 learned_addr learned my addr 172.21.15.43:0/3539924469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff457f6640 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 -- 172.21.15.43:0/3539924469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 msgr2=0x55942c6cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c6cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 -- 172.21.15.43:0/3539924469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 msgr2=0x55942c6cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.470+0000 7eff44ff5640 1 -- 172.21.15.43:0/3539924469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55942c4a2d00 con 0x55942c6cd830 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cbbd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7eff34006a20 tx=0x7eff34009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeb2927d00c2f9d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff29ffb640 1 -- 172.21.15.43:0/3539924469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340d6020 con 0x55942c6cd830 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff29ffb640 1 -- 172.21.15.43:0/3539924469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff340c6070 con 0x55942c6cd830 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff29ffb640 1 -- 172.21.15.43:0/3539924469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340c13f0 con 0x55942c6cd830 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/3539924469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 msgr2=0x55942c6cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cbbd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7eff34006a20 tx=0x7eff34009240 comp rx=0 tx=0).stop 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/3539924469 shutdown_connections 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c717690 0x55942c6cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c716d70 0x55942c6cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- 172.21.15.43:0/3539924469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c6cbbd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/3539924469 >> 172.21.15.43:0/3539924469 conn(0x55942c6c97e0 msgr2=0x55942c72df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/3539924469 shutdown_connections 2024-03-30T13:13:51.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/3539924469 wait complete. 2024-03-30T13:13:51.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- start start 2024-03-30T13:13:51.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 Processor -- start 2024-03-30T13:13:51.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c7893c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c789900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c785950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55942c4d96a0 con 0x55942c6cd830 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7eff402006a0 con 0x55942c716d70 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55942c4e3270 con 0x55942c717690 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 10 client.0 _ll_register_callbacks cb 0x7eff340096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c7893c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c7893c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37246/0 (socket says 172.21.15.43:37246) 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 -- 172.21.15.43:0/834885142 learned_addr learned my addr 172.21.15.43:0/834885142 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff457f6640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c785950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff3ffff640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c789900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 -- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 msgr2=0x55942c785950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c785950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 -- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 msgr2=0x55942c789900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c789900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 -- 172.21.15.43:0/834885142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55942c59cf70 con 0x55942c6cd830 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff3ffff640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c789900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:51.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff44ff5640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c7893c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7eff3401c740 tx=0x7eff3401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df29e426f1e7927 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff1f7fe640 1 -- 172.21.15.43:0/834885142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff340df060 con 0x55942c6cd830 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff1f7fe640 1 -- 172.21.15.43:0/834885142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7eff340c38b0 con 0x55942c6cd830 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff340c6070 con 0x55942c6cd830 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff1f7fe640 1 -- 172.21.15.43:0/834885142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7eff340d6070 con 0x55942c6cd830 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 10 client.6267 fetch_fsmap learned FSMap version 20 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x55942c4c2760 con 0x55942c6cd830 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff1f7fe640 1 -- 172.21.15.43:0/834885142 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff340c6070 con 0x55942c6cd830 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff1f7fe640 1 -- 172.21.15.43:0/834885142 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 20) ==== 3494+0+0 (secure 0 0 0) 0x7eff340d1040 con 0x55942c6cd830 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 10 client.6267 fetch_fsmap finished waiting for FSMap version 20 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 10 client.6267 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 20 client.6267 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c7862b0 0x55942c786690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:51.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 4 client.6267 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628334}] 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628334}) -- 0x55942c528fe0 con 0x55942c7862b0 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.474+0000 7eff3ffff640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c7862b0 0x55942c786690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff3ffff640 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c7862b0 0x55942c786690 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b5df4979d57c97 server_cookie=e89f39c2b898fa78 in_seq=0 out_seq=0 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff1f7fe640 10 client.6267 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff1f7fe640 1 -- 172.21.15.43:0/834885142 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3821 (crc 0 0 0) 0x55942c528fe0 con 0x55942c7862b0 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff1f7fe640 10 client.6267 handle_command_reply: tid=0 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 client.6267 shutdown 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 msgr2=0x55942c7893c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c7893c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7eff3401c740 tx=0x7eff3401ace0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 shutdown_connections 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55942c7862b0 0x55942c786690 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55942c717690 0x55942c785950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55942c716d70 0x55942c789900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff1f7fe640 0 client.6267 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 --2- 172.21.15.43:0/834885142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55942c6cd830 0x55942c7893c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 >> 172.21.15.43:0/834885142 conn(0x55942c6c97e0 msgr2=0x55942c72e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 shutdown_connections 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 1 -- 172.21.15.43:0/834885142 wait complete. 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff4a3db000 20 client.6267 trim_cache size 0 max 16384 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/downcall.h", 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628334, 2024-03-30T13:13:51.487 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:16.020521+0000", 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:12.931375+0000", 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:13:51.488 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 2953, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:12.931375+0000", 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:13:51.489 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:13:51.490 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2953, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.020521+0000" 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2953, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:16.020521+0000" 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.491 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 132, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 14, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:13:51.492 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 18446744073709551614, 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.493 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:13:51.494 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:13:51.495 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:13:51.496 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:13:51.497 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:13:51.498 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:13:51.499 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7eff24067fc0 msgr2=0x7eff2406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7eff24067fc0 0x7eff2406a480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7eff340af000 tx=0x7eff34008040 comp rx=0 tx=0).stop 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff400b4870 msgr2=0x7eff401f7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff400b4870 0x7eff401f7990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7eff2c007fd0 tx=0x7eff2c00c890 comp rx=0 tx=0).stop 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 shutdown_connections 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7eff24067fc0 0x7eff2406a480 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7eff40144930 0x7eff400a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff400b4870 0x7eff401f7990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 --2- 172.21.15.43:0/3102235694 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7eff4006cd80 0x7eff4015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 >> 172.21.15.43:0/3102235694 conn(0x7eff400d00a0 msgr2=0x7eff4015c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 shutdown_connections 2024-03-30T13:13:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:51.478+0000 7eff47259640 1 -- 172.21.15.43:0/3102235694 wait complete. 2024-03-30T13:13:51.501 DEBUG:tasks.cephfs.test_uninlining:removing dir linux-5.4 2024-03-30T13:13:51.501 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && rm -rf linux-5.4/ 2024-03-30T13:13:52.068 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_length == 0 2024-03-30T13:13:52.068 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_version == CEPH_INLINE_NONE 2024-03-30T13:13:52.068 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: without clearing cache, without unmount client, without mount.open_background, without multimds, with snapshots 2024-03-30T13:13:52.068 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-30T13:13:52.068 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-30T13:13:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f773ba52640 1 Processor -- start 2024-03-30T13:13:52.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f773ba52640 1 -- start start 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f773ba52640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77340b4870 0x7f773406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f773ba52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f77340ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f773ba52640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77340b3660 con 0x7f7734144930 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f773ba52640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f77340cdb90 con 0x7f77340b4870 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f773ba52640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f77340b3940 con 0x7f773406cbb0 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7738fed640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77340b4870 0x7f773406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7738fed640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77340b4870 0x7f773406c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37258/0 (socket says 172.21.15.43:37258) 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7738fed640 1 -- 172.21.15.43:0/446053991 learned_addr learned my addr 172.21.15.43:0/446053991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7739fef640 1 --2- 172.21.15.43:0/446053991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f77340ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7738fed640 1 -- 172.21.15.43:0/446053991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 msgr2=0x7f77340ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7738fed640 1 --2- 172.21.15.43:0/446053991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f77340ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7738fed640 1 -- 172.21.15.43:0/446053991 >> v1:172.21.15.113:6789/0 conn(0x7f7734144930 legacy=0x7f77340b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.202+0000 7f7738fed640 1 -- 172.21.15.43:0/446053991 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77340b3c20 con 0x7f77340b4870 2024-03-30T13:13:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f7738fed640 1 --2- 172.21.15.43:0/446053991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f773406c670 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7724009870 tx=0x7f772401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdfb9beb4ba889c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f772bfff640 1 -- 172.21.15.43:0/446053991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77240b1070 con 0x7f77340b4870 2024-03-30T13:13:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f772bfff640 1 -- 172.21.15.43:0/446053991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77240a8050 con 0x7f77340b4870 2024-03-30T13:13:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f772bfff640 1 -- 172.21.15.43:0/446053991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77240ac860 con 0x7f77340b4870 2024-03-30T13:13:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 -- 172.21.15.43:0/446053991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 msgr2=0x7f773406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 --2- 172.21.15.43:0/446053991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f773406c670 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f7724009870 tx=0x7f772401c4d0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 -- 172.21.15.43:0/446053991 shutdown_connections 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 --2- 172.21.15.43:0/446053991 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f77340ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 --2- 172.21.15.43:0/446053991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f773406c670 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 -- 172.21.15.43:0/446053991 >> 172.21.15.43:0/446053991 conn(0x7f77340d00a0 msgr2=0x7f77340d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 -- 172.21.15.43:0/446053991 shutdown_connections 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 -- 172.21.15.43:0/446053991 wait complete. 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 Processor -- start 2024-03-30T13:13:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 -- start start 2024-03-30T13:13:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.206+0000 7f773ba52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f773413bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f7734140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7734144930 0x7f773413c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f77340cf180 con 0x7f77340b4870 2024-03-30T13:13:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f77340cf000 con 0x7f7734144930 2024-03-30T13:13:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f77340cf300 con 0x7f773406cbb0 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7738fed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f7734140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f77397ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f773413bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f77397ee640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f773413bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:37502/0 (socket says 172.21.15.43:37502) 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f77397ee640 1 -- 172.21.15.43:0/416762635 learned_addr learned my addr 172.21.15.43:0/416762635 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7739fef640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7734144930 0x7f773413c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7738fed640 1 -- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 msgr2=0x7f773413bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7738fed640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f773413bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7738fed640 1 -- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7734144930 msgr2=0x7f773413c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7738fed640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7734144930 0x7f773413c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7738fed640 1 -- 172.21.15.43:0/416762635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f773415c870 con 0x7f77340b4870 2024-03-30T13:13:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7738fed640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f7734140940 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f77240ac000 tx=0x7f772401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7729ffb640 1 -- 172.21.15.43:0/416762635 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77240b1040 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7729ffb640 1 -- 172.21.15.43:0/416762635 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77240bc070 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f773415cb50 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7729ffb640 1 -- 172.21.15.43:0/416762635 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77240b7690 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f773415d110 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773415d7d0 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7729ffb640 1 -- 172.21.15.43:0/416762635 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f77240b7830 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7729ffb640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7718067fc0 0x7f771806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f7729ffb640 1 -- 172.21.15.43:0/416762635 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f7724120790 con 0x7f77340b4870 2024-03-30T13:13:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.210+0000 7f77397ee640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7718067fc0 0x7f771806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.214+0000 7f77397ee640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7718067fc0 0x7f771806a480 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7730001f60 tx=0x7f7730009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.214+0000 7f7729ffb640 1 -- 172.21.15.43:0/416762635 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7724016150 con 0x7f77340b4870 2024-03-30T13:13:52.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.418+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7f7734068900 con 0x7f77340b4870 2024-03-30T13:13:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.966+0000 7f7729ffb640 1 -- 172.21.15.43:0/416762635 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v21) ==== 105+0+0 (secure 0 0 0) 0x7f77240f0e10 con 0x7f77340b4870 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.970+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7718067fc0 msgr2=0x7f771806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.970+0000 7f773ba52640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7718067fc0 0x7f771806a480 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7730001f60 tx=0x7f7730009040 comp rx=0 tx=0).stop 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.970+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 msgr2=0x7f7734140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.970+0000 7f773ba52640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f7734140940 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f77240ac000 tx=0x7f772401ad10 comp rx=0 tx=0).stop 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 shutdown_connections 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7718067fc0 0x7f771806a480 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7734144930 0x7f773413c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77340b4870 0x7f7734140940 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 --2- 172.21.15.43:0/416762635 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f773406cbb0 0x7f773413bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 >> 172.21.15.43:0/416762635 conn(0x7f77340d00a0 msgr2=0x7f77340727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 shutdown_connections 2024-03-30T13:13:52.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:52.974+0000 7f773ba52640 1 -- 172.21.15.43:0/416762635 wait complete. 2024-03-30T13:13:53.001 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-30T13:13:53.001 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-30T13:13:53.001 DEBUG:tasks.cephfs.test_uninlining:starting snapshotter thread 2024-03-30T13:13:53.001 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4 2024-03-30T13:13:53.021 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_0 2024-03-30T13:13:53.021 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-30T13:13:53.022 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-30T13:13:54.080 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_1 2024-03-30T13:13:55.097 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_2 2024-03-30T13:13:56.121 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_3 2024-03-30T13:13:56.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:56.903 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:13:57.140 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_4 2024-03-30T13:13:57.875 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-30T13:13:57.875 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-30T13:13:57.895 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph 2024-03-30T13:13:57.895 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2 2024-03-30T13:13:57.895 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs 2024-03-30T13:13:57.896 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-30T13:13:57.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:57.896 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-30T13:13:57.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:57.945 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-30T13:13:57.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:57.997 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-30T13:13:58.054 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-30T13:13:58.054 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-30T13:13:58.205 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_5 2024-03-30T13:13:58.218 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:13:58.218 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/Makefile 2024-03-30T13:13:58.218 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:13:58.218 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/io.h 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/Makefile 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/acl.h 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:13:58.219 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:13:58.220 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:13:58.220 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:13:58.220 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:13:58.220 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:13:58.220 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:13:58.220 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-30T13:13:58.220 DEBUG:tasks.cephfs.test_uninlining:starting scrub 2024-03-30T13:13:58.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:58.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive 2024-03-30T13:13:58.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.354+0000 7ff9ec3cd640 1 Processor -- start 2024-03-30T13:13:58.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.354+0000 7ff9ec3cd640 1 -- start start 2024-03-30T13:13:58.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.354+0000 7ff9ec3cd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 0x7ff9e40d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.354+0000 7ff9ec3cd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e4074030 con 0x7ff9e40b7cd0 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.354+0000 7ff9ec3cd640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e40d2c50 con 0x7ff9e4161af0 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.354+0000 7ff9ec3cd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e40d4150 con 0x7ff9e40d3cf0 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9ea96a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff9e40b7cd0 0x7ff9e406b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37618/0 (socket says 172.21.15.43:37618) 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9ea96a640 1 -- 172.21.15.43:0/1449368226 learned_addr learned my addr 172.21.15.43:0/1449368226 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 0x7ff9e40d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579801226 0 0) 0x7ff9e4074030 con 0x7ff9e40b7cd0 2024-03-30T13:13:58.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9d0003530 con 0x7ff9e40b7cd0 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367814722 0 0) 0x7ff9e40d2c50 con 0x7ff9e4161af0 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9e4074030 con 0x7ff9e4161af0 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 423568020 0 0) 0x7ff9d0003530 con 0x7ff9e40b7cd0 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 msgr2=0x7ff9e40d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 0x7ff9e40d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 >> v1:172.21.15.113:6789/0 conn(0x7ff9e4161af0 legacy=0x7ff9e40b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e40d2f30 con 0x7ff9e40b7cd0 2024-03-30T13:13:58.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7ff9dc001a00 con 0x7ff9e40b7cd0 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 >> v1:172.21.15.43:6789/0 conn(0x7ff9e40b7cd0 legacy=0x7ff9e406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 0x7ff9d0003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0004130 0x7ff9d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9d0004d50 0x7ff9d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d000bae0 con 0x7ff9d0004130 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d000bc90 con 0x7ff9d0004d50 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d000be90 con 0x7ff9e40d3cf0 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0004130 0x7ff9d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9ea169640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 0x7ff9d0003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9d0004d50 0x7ff9d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 -- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 msgr2=0x7ff9d0003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9e40d3cf0 0x7ff9d0003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 -- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9d0004d50 msgr2=0x7ff9d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9d0004d50 0x7ff9d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 -- 172.21.15.43:0/1449368226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d0003530 con 0x7ff9d0004130 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0004130 0x7ff9d0004810 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009ae0 tx=0x7ff9d401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd7ef713ff3c294 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40b2070 con 0x7ff9d0004130 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40a5520 con 0x7ff9d0004130 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.358+0000 7ff9e9167640 1 -- 172.21.15.43:0/1449368226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40096b0 con 0x7ff9d0004130 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/1449368226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0004130 msgr2=0x7ff9d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 --2- 172.21.15.43:0/1449368226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9d0004130 0x7ff9d0004810 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009ae0 tx=0x7ff9d401c9b0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ea96a640 1 -- 172.21.15.43:0/1449368226 reap_dead start 2024-03-30T13:13:58.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/1449368226 shutdown_connections 2024-03-30T13:13:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/1449368226 >> 172.21.15.43:0/1449368226 conn(0x7ff9e40d42e0 msgr2=0x7ff9e40d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/1449368226 shutdown_connections 2024-03-30T13:13:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/1449368226 wait complete. 2024-03-30T13:13:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 Processor -- start 2024-03-30T13:13:58.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- start start 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9d0004130 0x7ff9e41f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e40cbd50 0x7ff9e40b9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9e40c7db0 0x7ff9e41f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e40d37d0 con 0x7ff9e40cbd50 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e40d34f0 con 0x7ff9e40c7db0 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e4161ed0 con 0x7ff9d0004130 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e40cbd50 0x7ff9e40b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e40cbd50 0x7ff9e40b9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37268/0 (socket says 172.21.15.43:37268) 2024-03-30T13:13:58.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 -- 172.21.15.43:0/3977873735 learned_addr learned my addr 172.21.15.43:0/3977873735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9e40c7db0 0x7ff9e41f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ea169640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9d0004130 0x7ff9e41f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 -- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9d0004130 msgr2=0x7ff9e41f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9d0004130 0x7ff9e41f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 -- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9e40c7db0 msgr2=0x7ff9e41f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9e40c7db0 0x7ff9e41f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 -- 172.21.15.43:0/3977873735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e40cac10 con 0x7ff9e40cbd50 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ea169640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9d0004130 0x7ff9e41f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9e9968640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e40cbd50 0x7ff9e40b9470 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009ae0 tx=0x7ff9d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9daffd640 1 -- 172.21.15.43:0/3977873735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40bc030 con 0x7ff9e40cbd50 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9daffd640 1 -- 172.21.15.43:0/3977873735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d400f000 con 0x7ff9e40cbd50 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e40ca650 con 0x7ff9e40cbd50 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9daffd640 1 -- 172.21.15.43:0/3977873735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40b2040 con 0x7ff9e40cbd50 2024-03-30T13:13:58.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.362+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e4152920 con 0x7ff9e40cbd50 2024-03-30T13:13:58.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9daffd640 1 -- 172.21.15.43:0/3977873735 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff9d40bf070 con 0x7ff9e40cbd50 2024-03-30T13:13:58.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9daffd640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff9c8067fc0 0x7ff9c806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9daffd640 1 -- 172.21.15.43:0/3977873735 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7ff9d4120640 con 0x7ff9e40cbd50 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ea169640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff9c8067fc0 0x7ff9c806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ea169640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff9c8067fc0 0x7ff9c806a480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0007af0 tx=0x7ff9e0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 Processor -- start 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 -- start start 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ade73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ad9dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f96abebc60 con 0x55f96ade7920 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f96ab88b80 con 0x55f96ade7000 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55f96abe7c10 con 0x55f96ad9d850 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37272/0 (socket says 172.21.15.43:37272) 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 -- 172.21.15.43:0/1315074771 learned_addr learned my addr 172.21.15.43:0/1315074771 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ea169640 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ade73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ad9dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 -- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 msgr2=0x55f96ad9dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ad9dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 -- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 msgr2=0x55f96ade73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ade73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 -- 172.21.15.43:0/1315074771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96ac92f10 con 0x55f96ade7920 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9ea169640 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ade73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9d310 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40f0420 tx=0x7ff9d4009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ba2517b2cc1508 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9c7fff640 1 -- 172.21.15.43:0/1315074771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40f4840 con 0x55f96ade7920 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9c7fff640 1 -- 172.21.15.43:0/1315074771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40f9070 con 0x55f96ade7920 2024-03-30T13:13:58.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.366+0000 7ff9c7fff640 1 -- 172.21.15.43:0/1315074771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40ecb00 con 0x55f96ade7920 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1315074771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 msgr2=0x55f96ad9d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9d310 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40f0420 tx=0x7ff9d4009770 comp rx=0 tx=0).stop 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1315074771 shutdown_connections 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ad9dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9d310 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1315074771 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ade73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1315074771 >> 172.21.15.43:0/1315074771 conn(0x55f96ab98bd0 msgr2=0x55f96ad9bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1315074771 shutdown_connections 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1315074771 wait complete. 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- start start 2024-03-30T13:13:58.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 Processor -- start 2024-03-30T13:13:58.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ad9d850 0x55f96ada1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96adcb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ade7920 0x55f96adcb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f96abebc60 con 0x55f96ad9d850 2024-03-30T13:13:58.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f96aba92a0 con 0x55f96ade7000 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55f96abe7c10 con 0x55f96ade7920 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 10 client.0 _ll_register_callbacks cb 0x55f96ad88af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ad9d850 0x55f96ada1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ad9d850 0x55f96ada1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37274/0 (socket says 172.21.15.43:37274) 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 -- 172.21.15.43:0/1065068819 learned_addr learned my addr 172.21.15.43:0/1065068819 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96adcb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ade7920 0x55f96adcb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 -- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ade7920 msgr2=0x55f96adcb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ade7920 0x55f96adcb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 -- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 msgr2=0x55f96adcb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96adcb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 -- 172.21.15.43:0/1065068819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96abc58d0 con 0x55f96ad9d850 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96adcb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:58.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ea169640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ad9d850 0x55f96ada1db0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0014700 tx=0x7ff9e0007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6344bf50dfc72cd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9c57fa640 1 -- 172.21.15.43:0/1065068819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0013700 con 0x55f96ad9d850 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9c57fa640 1 -- 172.21.15.43:0/1065068819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7ff9e00138a0 con 0x55f96ad9d850 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d40f9070 con 0x55f96ad9d850 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9c57fa640 1 -- 172.21.15.43:0/1065068819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff9e002e050 con 0x55f96ad9d850 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 10 client.6285 fetch_fsmap learned FSMap version 21 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f96ac310d0 con 0x55f96ad9d850 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9c57fa640 1 -- 172.21.15.43:0/1065068819 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40f9070 con 0x55f96ad9d850 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9c57fa640 1 -- 172.21.15.43:0/1065068819 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7ff9e0031020 con 0x55f96ad9d850 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 10 client.6285 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 10 client.6285 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 20 client.6285 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96aded640 0x55f96adeda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 4 client.6285 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f96ab86f70 con 0x55f96aded640 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.370+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96aded640 0x55f96adeda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9e9968640 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96aded640 0x55f96adeda20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10d187c7150c676d server_cookie=343f414578b0277b in_seq=0 out_seq=0 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9c57fa640 10 client.6285 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9c57fa640 1 -- 172.21.15.43:0/1065068819 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55f96ab86f70 con 0x55f96aded640 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9c57fa640 10 client.6285 handle_command_reply: tid=0 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 client.6285 shutdown 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ad9d850 msgr2=0x55f96ada1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ad9d850 0x55f96ada1db0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0014700 tx=0x7ff9e0007a80 comp rx=0 tx=0).stop 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 shutdown_connections 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96aded640 0x55f96adeda20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ade7920 0x55f96adcb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96adcb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9c57fa640 0 client.6285 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/1065068819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ad9d850 0x55f96ada1db0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 >> 172.21.15.43:0/1065068819 conn(0x55f96ab98bd0 msgr2=0x55f96ad9bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 shutdown_connections 2024-03-30T13:13:58.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 1 -- 172.21.15.43:0/1065068819 wait complete. 2024-03-30T13:13:58.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.374+0000 7ff9ef54f000 20 client.6285 trim_cache size 0 max 16384 2024-03-30T13:13:58.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 Processor -- start 2024-03-30T13:13:58.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- start start 2024-03-30T13:13:58.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96adf3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7000 0x55f96adfd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7920 0x55f96adfdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4201660 con 0x55f96ade7000 2024-03-30T13:13:58.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4200680 con 0x55f96ade7920 2024-03-30T13:13:58.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e41ff9e0 con 0x55f96ad9d850 2024-03-30T13:13:58.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7000 0x55f96adfd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7000 0x55f96adfd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37290/0 (socket says 172.21.15.43:37290) 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 -- 172.21.15.43:0/2763676626 learned_addr learned my addr 172.21.15.43:0/2763676626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ea169640 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96adf3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7920 0x55f96adfdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 -- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 msgr2=0x55f96adf3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96adf3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 -- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7920 msgr2=0x55f96adfdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7920 0x55f96adfdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 -- 172.21.15.43:0/2763676626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f96abb9410 con 0x55f96ade7000 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ea169640 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96adf3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7920 0x55f96adfdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7000 0x55f96adfd700 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40b0040 tx=0x7ff9d40b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55adb31ea6dfc0cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9c77fe640 1 -- 172.21.15.43:0/2763676626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40f9070 con 0x55f96ade7000 2024-03-30T13:13:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9c77fe640 1 -- 172.21.15.43:0/2763676626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40ee790 con 0x55f96ade7000 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9c77fe640 1 -- 172.21.15.43:0/2763676626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40f8be0 con 0x55f96ade7000 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2763676626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7000 msgr2=0x55f96adfd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7000 0x55f96adfd700 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40b0040 tx=0x7ff9d40b9000 comp rx=0 tx=0).stop 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2763676626 shutdown_connections 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7920 0x55f96adfdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7000 0x55f96adfd700 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2763676626 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96adf3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2763676626 >> 172.21.15.43:0/2763676626 conn(0x55f96ab98bd0 msgr2=0x55f96adebd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2763676626 shutdown_connections 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2763676626 wait complete. 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- start start 2024-03-30T13:13:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 Processor -- start 2024-03-30T13:13:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ae58970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ae58eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f96ab732c0 con 0x55f96ade7920 2024-03-30T13:13:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e4200030 con 0x55f96ade7000 2024-03-30T13:13:58.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55f96abb1370 con 0x55f96ad9d850 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ef54f000 10 client.0 _ll_register_callbacks cb 0x7ff9e0017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ea96a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ea96a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37296/0 (socket says 172.21.15.43:37296) 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9ea96a640 1 -- 172.21.15.43:0/2014342636 learned_addr learned my addr 172.21.15.43:0/2014342636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.394+0000 7ff9e9968640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ae58eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea169640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ae58970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea96a640 1 -- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 msgr2=0x55f96ae58970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ae58970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea96a640 1 -- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 msgr2=0x55f96ae58eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ae58eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea96a640 1 -- 172.21.15.43:0/2014342636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f96ac6d1e0 con 0x55f96ade7920 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9e9968640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ae58eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea96a640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9b360 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc001f30 tx=0x7ff9dc018a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e88e76981fe9407 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9c57fa640 1 -- 172.21.15.43:0/2014342636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc010040 con 0x55f96ade7920 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9c57fa640 1 -- 172.21.15.43:0/2014342636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7ff9dc01faf0 con 0x55f96ade7920 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d40ee790 con 0x55f96ade7920 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9c57fa640 1 -- 172.21.15.43:0/2014342636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff9dc01fcf0 con 0x55f96ade7920 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 10 client.6291 fetch_fsmap learned FSMap version 21 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f96a8a73b0 con 0x55f96ade7920 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9c57fa640 1 -- 172.21.15.43:0/2014342636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40ee790 con 0x55f96ade7920 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9c57fa640 1 -- 172.21.15.43:0/2014342636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7ff9dc02c070 con 0x55f96ade7920 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 10 client.6291 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 10 client.6291 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 20 client.6291 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96ad9c270 0x55f96ad9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 4 client.6291 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-30T13:13:58.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55f96abf91a0 con 0x55f96ad9c270 2024-03-30T13:13:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea169640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96ad9c270 0x55f96ad9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9ea169640 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96ad9c270 0x55f96ad9c680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d73916bc6d2bd7e server_cookie=a599cb3d917b44d in_seq=0 out_seq=0 2024-03-30T13:13:58.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.398+0000 7ff9c57fa640 10 client.6291 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9c57fa640 1 -- 172.21.15.43:0/2014342636 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f96abf91a0 con 0x55f96ad9c270 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9c57fa640 10 client.6291 handle_command_reply: tid=0 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 client.6291 shutdown 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 msgr2=0x55f96ad9b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9b360 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc001f30 tx=0x7ff9dc018a40 comp rx=0 tx=0).stop 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 shutdown_connections 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f96ad9c270 0x55f96ad9c680 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f96ade7920 0x55f96ad9b360 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f96ade7000 0x55f96ae58eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 --2- 172.21.15.43:0/2014342636 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f96ad9d850 0x55f96ae58970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 >> 172.21.15.43:0/2014342636 conn(0x55f96ab98bd0 msgr2=0x55f96ada2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9c57fa640 0 client.6291 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 shutdown_connections 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 1 -- 172.21.15.43:0/2014342636 wait complete. 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ef54f000 20 client.6291 trim_cache size 0 max 16384 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "b8e79244-1d55-4029-b4a0-9f0f9d72a948", 2024-03-30T13:13:58.409 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-03-30T13:13:58.410 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:58.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff9c8067fc0 msgr2=0x7ff9c806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff9c8067fc0 0x7ff9c806a480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0007af0 tx=0x7ff9e0009040 comp rx=0 tx=0).stop 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e40cbd50 msgr2=0x7ff9e40b9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e40cbd50 0x7ff9e40b9470 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009ae0 tx=0x7ff9d0005840 comp rx=0 tx=0).stop 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 shutdown_connections 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff9c8067fc0 0x7ff9c806a480 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff9e40c7db0 0x7ff9e41f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9e40cbd50 0x7ff9e40b9470 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 --2- 172.21.15.43:0/3977873735 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff9d0004130 0x7ff9e41f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 >> 172.21.15.43:0/3977873735 conn(0x7ff9e40d42e0 msgr2=0x7ff9e40d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 shutdown_connections 2024-03-30T13:13:58.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.402+0000 7ff9ec3cd640 1 -- 172.21.15.43:0/3977873735 wait complete. 2024-03-30T13:13:58.424 DEBUG:tasks.cephfs.test_uninlining:scrub start response: {'return_code': 0, 'scrub_tag': 'b8e79244-1d55-4029-b4a0-9f0f9d72a948', 'mode': 'asynchronous'} 2024-03-30T13:13:58.424 DEBUG:tasks.cephfs.test_uninlining:waiting for scrub to complete 2024-03-30T13:13:58.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:13:58.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:13:58.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4522f3b640 1 Processor -- start 2024-03-30T13:13:58.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4522f3b640 1 -- start start 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4522f3b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f451c15e4c0 0x7f451c0b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4522f3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f451c06c070 0x7f451c0cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4522f3b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451c06c850 con 0x7f451c0cf4f0 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4522f3b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f451c15e8a0 con 0x7f451c06c070 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4522f3b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f451c0cf920 con 0x7f451c15e4c0 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4513fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f451c06c070 0x7f451c0cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4513fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f451c06c070 0x7f451c0cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37304/0 (socket says 172.21.15.43:37304) 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4513fff640 1 -- 172.21.15.43:0/4005291387 learned_addr learned my addr 172.21.15.43:0/4005291387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4520cd7640 1 --2- 172.21.15.43:0/4005291387 >> v2:172.21.15.113:3300/0 conn(0x7f451c15e4c0 0x7f451c0b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4513fff640 1 -- 172.21.15.43:0/4005291387 >> v2:172.21.15.113:3300/0 conn(0x7f451c15e4c0 msgr2=0x7f451c0b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4513fff640 1 --2- 172.21.15.43:0/4005291387 >> v2:172.21.15.113:3300/0 conn(0x7f451c15e4c0 0x7f451c0b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4513fff640 1 -- 172.21.15.43:0/4005291387 >> v1:172.21.15.113:6789/0 conn(0x7f451c0cf4f0 legacy=0x7f451c161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.538+0000 7f4513fff640 1 -- 172.21.15.43:0/4005291387 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c06cb30 con 0x7f451c06c070 2024-03-30T13:13:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4513fff640 1 --2- 172.21.15.43:0/4005291387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c06c070 0x7f451c0cefb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f451c0d0cc0 tx=0x7f450c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f31f3fad44abbad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f45137fe640 1 -- 172.21.15.43:0/4005291387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c0b0070 con 0x7f451c06c070 2024-03-30T13:13:58.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f45137fe640 1 -- 172.21.15.43:0/4005291387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c0ab600 con 0x7f451c06c070 2024-03-30T13:13:58.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f45137fe640 1 -- 172.21.15.43:0/4005291387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c0ab9d0 con 0x7f451c06c070 2024-03-30T13:13:58.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 -- 172.21.15.43:0/4005291387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c06c070 msgr2=0x7f451c0cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 --2- 172.21.15.43:0/4005291387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c06c070 0x7f451c0cefb0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f451c0d0cc0 tx=0x7f450c01c520 comp rx=0 tx=0).stop 2024-03-30T13:13:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 -- 172.21.15.43:0/4005291387 shutdown_connections 2024-03-30T13:13:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 --2- 172.21.15.43:0/4005291387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c06c070 0x7f451c0cefb0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 --2- 172.21.15.43:0/4005291387 >> v2:172.21.15.113:3300/0 conn(0x7f451c15e4c0 0x7f451c0b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 -- 172.21.15.43:0/4005291387 >> 172.21.15.43:0/4005291387 conn(0x7f451c0b6410 msgr2=0x7f451c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 -- 172.21.15.43:0/4005291387 shutdown_connections 2024-03-30T13:13:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.542+0000 7f4522f3b640 1 -- 172.21.15.43:0/4005291387 wait complete. 2024-03-30T13:13:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 Processor -- start 2024-03-30T13:13:58.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 -- start start 2024-03-30T13:13:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451c06c070 0x7f451c0c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c0cf4f0 0x7f451c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451c15e4c0 0x7f451c15c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f451c16b3d0 con 0x7f451c0cf4f0 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f451c16b250 con 0x7f451c06c070 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f451c16b550 con 0x7f451c15e4c0 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c0cf4f0 0x7f451c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c0cf4f0 0x7f451c0ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37312/0 (socket says 172.21.15.43:37312) 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 -- 172.21.15.43:0/3634209243 learned_addr learned my addr 172.21.15.43:0/3634209243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451c06c070 0x7f451c0c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f45214d8640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451c15e4c0 0x7f451c15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 -- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451c15e4c0 msgr2=0x7f451c15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451c15e4c0 0x7f451c15c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 -- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451c06c070 msgr2=0x7f451c0c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451c06c070 0x7f451c0c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 -- 172.21.15.43:0/3634209243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c0cac40 con 0x7f451c0cf4f0 2024-03-30T13:13:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4513fff640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c0cf4f0 0x7f451c0ca700 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f450c0ae040 tx=0x7f450c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f45117fa640 1 -- 172.21.15.43:0/3634209243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c0b8020 con 0x7f451c0cf4f0 2024-03-30T13:13:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f45117fa640 1 -- 172.21.15.43:0/3634209243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c0b4410 con 0x7f451c0cf4f0 2024-03-30T13:13:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c0c9660 con 0x7f451c0cf4f0 2024-03-30T13:13:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f45117fa640 1 -- 172.21.15.43:0/3634209243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c0b0040 con 0x7f451c0cf4f0 2024-03-30T13:13:58.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c0c9c20 con 0x7f451c0cf4f0 2024-03-30T13:13:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.546+0000 7f45117fa640 1 -- 172.21.15.43:0/3634209243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f450c0bb020 con 0x7f451c0cf4f0 2024-03-30T13:13:58.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45117fa640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4508067fc0 0x7f450806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45117fa640 1 -- 172.21.15.43:0/3634209243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f450c11fc90 con 0x7f451c0cf4f0 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4508067fc0 0x7f450806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 Processor -- start 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- start start 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4508067fc0 0x7f450806a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f451c15d070 tx=0x7f4518009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78cc7600 0x55aa78cc79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78c7ace0 0x55aa78c7b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa78b63e20 con 0x55aa78cc7600 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aa78accb00 con 0x55aa78cc7f20 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aa78a69940 con 0x55aa78c7ace0 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78cc7600 0x55aa78cc79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78cc7600 0x55aa78cc79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37328/0 (socket says 172.21.15.43:37328) 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 -- 172.21.15.43:0/2568704464 learned_addr learned my addr 172.21.15.43:0/2568704464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4513fff640 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45214d8640 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78c7ace0 0x55aa78c7b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 -- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78c7ace0 msgr2=0x55aa78c7b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78c7ace0 0x55aa78c7b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 -- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 msgr2=0x55aa78c7a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 -- 172.21.15.43:0/2568704464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa78a52380 con 0x55aa78cc7600 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45214d8640 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78c7ace0 0x55aa78c7b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f4520cd7640 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78cc7600 0x55aa78cc79e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4518011d80 tx=0x7f4518014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76ef98e0fcb3a84 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45027fc640 1 -- 172.21.15.43:0/2568704464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518014bd0 con 0x55aa78cc7600 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45027fc640 1 -- 172.21.15.43:0/2568704464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f451801b070 con 0x55aa78cc7600 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45027fc640 1 -- 172.21.15.43:0/2568704464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518004030 con 0x55aa78cc7600 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- 172.21.15.43:0/2568704464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78cc7600 msgr2=0x55aa78cc79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78cc7600 0x55aa78cc79e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4518011d80 tx=0x7f4518014000 comp rx=0 tx=0).stop 2024-03-30T13:13:58.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- 172.21.15.43:0/2568704464 shutdown_connections 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78c7ace0 0x55aa78c7b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 --2- 172.21.15.43:0/2568704464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78cc7600 0x55aa78cc79e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- 172.21.15.43:0/2568704464 >> 172.21.15.43:0/2568704464 conn(0x55aa78ca83a0 msgr2=0x55aa78caa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- 172.21.15.43:0/2568704464 shutdown_connections 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.550+0000 7f45260bd000 1 -- 172.21.15.43:0/2568704464 wait complete. 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 -- start start 2024-03-30T13:13:58.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 Processor -- start 2024-03-30T13:13:58.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa78a8a1e0 con 0x55aa78c7ace0 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aa78ac8ab0 con 0x55aa78cc7f20 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aa78ad1130 con 0x55aa78cc7600 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 10 client.0 _ll_register_callbacks cb 0x55aa78c686a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37340/0 (socket says 172.21.15.43:37340) 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 -- 172.21.15.43:0/3363100308 learned_addr learned my addr 172.21.15.43:0/3363100308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45214d8640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4513fff640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 -- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 msgr2=0x55aa78cceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 -- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 msgr2=0x55aa78c7f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 -- 172.21.15.43:0/3363100308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa78a88b70 con 0x55aa78c7ace0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45214d8640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4513fff640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cce960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x55aa78a94dc0 tx=0x7f4518004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba0d7decaf8f53c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f44fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3363100308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518037060 con 0x55aa78c7ace0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3363100308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f4518003620 con 0x55aa78c7ace0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f451801b070 con 0x55aa78c7ace0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3363100308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f451803d050 con 0x55aa78c7ace0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 10 client.6303 fetch_fsmap learned FSMap version 21 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55aa78a69b90 con 0x55aa78c7ace0 2024-03-30T13:13:58.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3363100308 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f451801b070 con 0x55aa78c7ace0 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3363100308 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f4518040020 con 0x55aa78c7ace0 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 10 client.6303 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 10 client.6303 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 20 client.6303 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78c80660 0x55aa78c80a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 4 client.6303 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa78a67dd0 con 0x55aa78c80660 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4513fff640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78c80660 0x55aa78c80a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f4513fff640 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78c80660 0x55aa78c80a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20ffbf0b68b07f53 server_cookie=7818aa9c56712a2c in_seq=0 out_seq=0 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.554+0000 7f44fb7fe640 10 client.6303 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3363100308 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55aa78a67dd0 con 0x55aa78c80660 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f44fb7fe640 10 client.6303 handle_command_reply: tid=0 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 client.6303 shutdown 2024-03-30T13:13:58.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 msgr2=0x55aa78cce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cce960 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x55aa78a94dc0 tx=0x7f4518004740 comp rx=0 tx=0).stop 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 shutdown_connections 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78c80660 0x55aa78c80a70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78c7f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f44fb7fe640 0 client.6303 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 --2- 172.21.15.43:0/3363100308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cce960 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 >> 172.21.15.43:0/3363100308 conn(0x55aa78ca83a0 msgr2=0x55aa78caa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 shutdown_connections 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 1 -- 172.21.15.43:0/3363100308 wait complete. 2024-03-30T13:13:58.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.558+0000 7f45260bd000 20 client.6303 trim_cache size 0 max 16384 2024-03-30T13:13:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 Processor -- start 2024-03-30T13:13:58.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 -- start start 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cc56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cc5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78cc6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f451c202800 con 0x55aa78c7ace0 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f451c202e50 con 0x55aa78cc7f20 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aa78b9b750 con 0x55aa78cc7600 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cc56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cc56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37348/0 (socket says 172.21.15.43:37348) 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 -- 172.21.15.43:0/621363315 learned_addr learned my addr 172.21.15.43:0/621363315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45214d8640 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78cc6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4513fff640 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cc5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 -- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 msgr2=0x55aa78cc5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cc5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 -- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 msgr2=0x55aa78cc6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78cc6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 -- 172.21.15.43:0/621363315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa78b4d470 con 0x55aa78c7ace0 2024-03-30T13:13:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4520cd7640 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cc56b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4518011690 tx=0x7f451802b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcefc443cec056a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4501ffb640 1 -- 172.21.15.43:0/621363315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518003620 con 0x55aa78c7ace0 2024-03-30T13:13:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4501ffb640 1 -- 172.21.15.43:0/621363315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4518003ae0 con 0x55aa78c7ace0 2024-03-30T13:13:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f4501ffb640 1 -- 172.21.15.43:0/621363315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518032410 con 0x55aa78c7ace0 2024-03-30T13:13:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 -- 172.21.15.43:0/621363315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 msgr2=0x55aa78cc56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.578+0000 7f45260bd000 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cc56b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4518011690 tx=0x7f451802b050 comp rx=0 tx=0).stop 2024-03-30T13:13:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- 172.21.15.43:0/621363315 shutdown_connections 2024-03-30T13:13:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78cc6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78cc5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 --2- 172.21.15.43:0/621363315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78cc56b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- 172.21.15.43:0/621363315 >> 172.21.15.43:0/621363315 conn(0x55aa78ca83a0 msgr2=0x55aa78cddaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- 172.21.15.43:0/621363315 shutdown_connections 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- 172.21.15.43:0/621363315 wait complete. 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- start start 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 Processor -- start 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78ccdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78d35270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78d357b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55aa78acbe10 con 0x55aa78c7ace0 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55aa78a83750 con 0x55aa78cc7f20 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55aa78a98150 con 0x55aa78cc7600 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 10 client.0 _ll_register_callbacks cb 0x7f450c0a4fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78ccdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78ccdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37350/0 (socket says 172.21.15.43:37350) 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 -- 172.21.15.43:0/3310371107 learned_addr learned my addr 172.21.15.43:0/3310371107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45214d8640 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78d357b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4513fff640 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78d35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 -- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 msgr2=0x55aa78d35270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78d35270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 -- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 msgr2=0x55aa78d357b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78d357b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 -- 172.21.15.43:0/3310371107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa78a5b1e0 con 0x55aa78c7ace0 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4520cd7640 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78ccdb90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4518024040 tx=0x7f4518018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb853955c0dd16ef server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f44fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3310371107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4518049440 con 0x55aa78c7ace0 2024-03-30T13:13:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4518003ae0 con 0x55aa78c7ace0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3310371107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f45180495e0 con 0x55aa78c7ace0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3310371107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f45180059a0 con 0x55aa78c7ace0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 10 client.6309 fetch_fsmap learned FSMap version 21 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55aa78a607b0 con 0x55aa78c7ace0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3310371107 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4518003ae0 con 0x55aa78c7ace0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3310371107 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f4518040040 con 0x55aa78c7ace0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 10 client.6309 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 10 client.6309 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 20 client.6309 populate_metadata read hostname 'smithi043' 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78cc65c0 0x55aa78cc69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 4 client.6309 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa78ada040 con 0x55aa78cc65c0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.582+0000 7f4513fff640 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78cc65c0 0x55aa78cc69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:13:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f4513fff640 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78cc65c0 0x55aa78cc69d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42131b5ce087889c server_cookie=11f34a2f2f49d2a6 in_seq=0 out_seq=0 2024-03-30T13:13:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f44fb7fe640 10 client.6309 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f44fb7fe640 1 -- 172.21.15.43:0/3310371107 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+857 (crc 0 0 0) 0x55aa78ada040 con 0x55aa78cc65c0 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f44fb7fe640 10 client.6309 handle_command_reply: tid=0 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 client.6309 shutdown 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 msgr2=0x55aa78ccdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78ccdb90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4518024040 tx=0x7f4518018000 comp rx=0 tx=0).stop 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 shutdown_connections 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55aa78cc65c0 0x55aa78cc69d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55aa78cc7f20 0x55aa78d357b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55aa78cc7600 0x55aa78d35270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f44fb7fe640 0 client.6309 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 --2- 172.21.15.43:0/3310371107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aa78c7ace0 0x55aa78ccdb90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 >> 172.21.15.43:0/3310371107 conn(0x55aa78ca83a0 msgr2=0x55aa78c7f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 shutdown_connections 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 1 -- 172.21.15.43:0/3310371107 wait complete. 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.586+0000 7f45260bd000 20 client.6309 trim_cache size 0 max 16384 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:13:58.595 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "scrub active (19 inodes in the stack)", 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "b8e79244-1d55-4029-b4a0-9f0f9d72a948": { 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "b8e79244-1d55-4029-b4a0-9f0f9d72a948", 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive" 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_stats": { 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "b8e79244-1d55-4029-b4a0-9f0f9d72a948": { 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-03-30T13:13:58.397094+0000", 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:13:58.596 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 0, 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 0, 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "9ec31cb9-756a-4ece-b87a-bdc758385b7c": { 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-03-30T13:13:15.761997+0000", 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 23, 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 23, 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-03-30T13:13:58.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:58.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:13:58.598 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:13:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4508067fc0 msgr2=0x7f450806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4508067fc0 0x7f450806a480 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f451c15d070 tx=0x7f4518009190 comp rx=0 tx=0).stop 2024-03-30T13:13:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c0cf4f0 msgr2=0x7f451c0ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:13:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c0cf4f0 0x7f451c0ca700 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f450c0ae040 tx=0x7f450c0192c0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 shutdown_connections 2024-03-30T13:13:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4508067fc0 0x7f450806a480 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f451c15e4c0 0x7f451c15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f451c0cf4f0 0x7f451c0ca700 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 --2- 172.21.15.43:0/3634209243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f451c06c070 0x7f451c0c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:13:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 >> 172.21.15.43:0/3634209243 conn(0x7f451c0b6410 msgr2=0x7f451c0b9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:13:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 shutdown_connections 2024-03-30T13:13:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:13:58.590+0000 7f4522f3b640 1 -- 172.21.15.43:0/3634209243 wait complete. 2024-03-30T13:13:58.610 INFO:tasks.cephfs.filesystem:scrub status for tag:b8e79244-1d55-4029-b4a0-9f0f9d72a948 - {'path': '/', 'tag': 'b8e79244-1d55-4029-b4a0-9f0f9d72a948', 'options': 'recursive'} 2024-03-30T13:13:59.223 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_6 2024-03-30T13:14:00.245 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_7 2024-03-30T13:14:01.265 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_8 2024-03-30T13:14:02.285 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && mkdir linux-5.4/.snap/snap_9 2024-03-30T13:14:27.047 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:27.051 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:28.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:28.611 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-03-30T13:14:28.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cf7bd640 1 Processor -- start 2024-03-30T13:14:28.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cf7bd640 1 -- start start 2024-03-30T13:14:28.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cf7bd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f26c815f200 0x7f26c815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cf7bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f26c80b4fc0 0x7f26c80cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cf7bd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26c80cfe50 con 0x7f26c80b5450 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cf7bd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f26c80d0a60 con 0x7f26c80b4fc0 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cf7bd640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f26c80d0130 con 0x7f26c815f200 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cdd5a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f26c80b5450 0x7f26c8154d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54190/0 (socket says 172.21.15.43:54190) 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cdd5a640 1 -- 172.21.15.43:0/2308642120 learned_addr learned my addr 172.21.15.43:0/2308642120 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26ccd58640 1 --2- 172.21.15.43:0/2308642120 >> v2:172.21.15.43:3300/0 conn(0x7f26c80b4fc0 0x7f26c80cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26cd559640 1 --2- 172.21.15.43:0/2308642120 >> v2:172.21.15.113:3300/0 conn(0x7f26c815f200 0x7f26c815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26bffff640 1 -- 172.21.15.43:0/2308642120 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840662539 0 0) 0x7f26c80cfe50 con 0x7f26c80b5450 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26bffff640 1 -- 172.21.15.43:0/2308642120 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26b4003530 con 0x7f26c80b5450 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26ccd58640 1 -- 172.21.15.43:0/2308642120 >> v2:172.21.15.113:3300/0 conn(0x7f26c815f200 msgr2=0x7f26c815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26ccd58640 1 --2- 172.21.15.43:0/2308642120 >> v2:172.21.15.113:3300/0 conn(0x7f26c815f200 0x7f26c815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26ccd58640 1 -- 172.21.15.43:0/2308642120 >> v1:172.21.15.43:6789/0 conn(0x7f26c80b5450 legacy=0x7f26c8154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.746+0000 7f26ccd58640 1 -- 172.21.15.43:0/2308642120 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c80d0410 con 0x7f26c80b4fc0 2024-03-30T13:14:28.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26ccd58640 1 --2- 172.21.15.43:0/2308642120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c80b4fc0 0x7f26c80cc7c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f26c80cfcf0 tx=0x7f26b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c50babb5ee54194 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:28.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26bffff640 1 -- 172.21.15.43:0/2308642120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80b0070 con 0x7f26c80b4fc0 2024-03-30T13:14:28.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26bffff640 1 -- 172.21.15.43:0/2308642120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b80ab600 con 0x7f26c80b4fc0 2024-03-30T13:14:28.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26bffff640 1 -- 172.21.15.43:0/2308642120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80ab9d0 con 0x7f26c80b4fc0 2024-03-30T13:14:28.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 -- 172.21.15.43:0/2308642120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c80b4fc0 msgr2=0x7f26c80cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/2308642120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c80b4fc0 0x7f26c80cc7c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f26c80cfcf0 tx=0x7f26b801c480 comp rx=0 tx=0).stop 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 -- 172.21.15.43:0/2308642120 shutdown_connections 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/2308642120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c80b4fc0 0x7f26c80cc7c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/2308642120 >> v2:172.21.15.113:3300/0 conn(0x7f26c815f200 0x7f26c815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 -- 172.21.15.43:0/2308642120 >> 172.21.15.43:0/2308642120 conn(0x7f26c80b3c80 msgr2=0x7f26c80b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 -- 172.21.15.43:0/2308642120 shutdown_connections 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 -- 172.21.15.43:0/2308642120 wait complete. 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.750+0000 7f26cf7bd640 1 Processor -- start 2024-03-30T13:14:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 -- start start 2024-03-30T13:14:28.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c80b4fc0 0x7f26c8151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26c80b5450 0x7f26c80cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c815f200 0x7f26c8154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b76e0 con 0x7f26c815f200 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80b7560 con 0x7f26c80b4fc0 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80b7860 con 0x7f26c80b5450 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c815f200 0x7f26c8154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c815f200 0x7f26c8154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33638/0 (socket says 172.21.15.43:33638) 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 -- 172.21.15.43:0/3492640158 learned_addr learned my addr 172.21.15.43:0/3492640158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cd559640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c80b4fc0 0x7f26c8151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26ccd58640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26c80b5450 0x7f26c80cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 -- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26c80b5450 msgr2=0x7f26c80cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26c80b5450 0x7f26c80cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 -- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c80b4fc0 msgr2=0x7f26c8151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c80b4fc0 0x7f26c8151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 -- 172.21.15.43:0/3492640158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c8160960 con 0x7f26c815f200 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26ccd58640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26c80b5450 0x7f26c80cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cdd5a640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c815f200 0x7f26c8154060 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c450 tx=0x7f26c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26bdffb640 1 -- 172.21.15.43:0/3492640158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c0017070 con 0x7f26c815f200 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26bdffb640 1 -- 172.21.15.43:0/3492640158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c0005cf0 con 0x7f26c815f200 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c80cabe0 con 0x7f26c815f200 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26bdffb640 1 -- 172.21.15.43:0/3492640158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c00072b0 con 0x7f26c815f200 2024-03-30T13:14:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c80cb1a0 con 0x7f26c815f200 2024-03-30T13:14:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26bdffb640 1 -- 172.21.15.43:0/3492640158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f26c001a070 con 0x7f26c815f200 2024-03-30T13:14:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.754+0000 7f26bdffb640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f26ac067fc0 0x7f26ac06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26bdffb640 1 -- 172.21.15.43:0/3492640158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f26c00858b0 con 0x7f26c815f200 2024-03-30T13:14:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26cd559640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f26ac067fc0 0x7f26ac06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 Processor -- start 2024-03-30T13:14:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- start start 2024-03-30T13:14:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26cd559640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f26ac067fc0 0x7f26ac06a480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f26c4011340 tx=0x7f26c400b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:28.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c6b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6bf20 0x560108c1e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c1f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560108b07e20 con 0x560108c6bf20 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560108a70b00 con 0x560108c1ece0 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560108a0d940 con 0x560108c6b600 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6bf20 0x560108c1e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6bf20 0x560108c1e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33640/0 (socket says 172.21.15.43:33640) 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 -- 172.21.15.43:0/336817605 learned_addr learned my addr 172.21.15.43:0/336817605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26cd559640 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c6b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26cdd5a640 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c1f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 -- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 msgr2=0x560108c6b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c6b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 -- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 msgr2=0x560108c1f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c1f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 -- 172.21.15.43:0/336817605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601089f6380 con 0x560108c6bf20 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26ccd58640 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6bf20 0x560108c1e7a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f26b8019090 tx=0x7f26b80a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fc3d5e7851d484 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26b2ffd640 1 -- 172.21.15.43:0/336817605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80b0050 con 0x560108c6bf20 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26b2ffd640 1 -- 172.21.15.43:0/336817605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b80b1040 con 0x560108c6bf20 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26b2ffd640 1 -- 172.21.15.43:0/336817605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80aba00 con 0x560108c6bf20 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- 172.21.15.43:0/336817605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6bf20 msgr2=0x560108c1e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6bf20 0x560108c1e7a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f26b8019090 tx=0x7f26b80a6000 comp rx=0 tx=0).stop 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- 172.21.15.43:0/336817605 shutdown_connections 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c1f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6bf20 0x560108c1e7a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 --2- 172.21.15.43:0/336817605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c6b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- 172.21.15.43:0/336817605 >> 172.21.15.43:0/336817605 conn(0x560108c4c3a0 msgr2=0x560108c4e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- 172.21.15.43:0/336817605 shutdown_connections 2024-03-30T13:14:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- 172.21.15.43:0/336817605 wait complete. 2024-03-30T13:14:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 -- start start 2024-03-30T13:14:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.758+0000 7f26d293f000 1 Processor -- start 2024-03-30T13:14:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c72750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c72c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6bf20 0x560108c24430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560108a2e1e0 con 0x560108c1ece0 2024-03-30T13:14:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560108a6cab0 con 0x560108c6bf20 2024-03-30T13:14:28.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560108b3f750 con 0x560108c6b600 2024-03-30T13:14:28.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 10 client.0 _ll_register_callbacks cb 0x560108c0c6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c72750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c72750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33652/0 (socket says 172.21.15.43:33652) 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 -- 172.21.15.43:0/2414019118 learned_addr learned my addr 172.21.15.43:0/2414019118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26ccd58640 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c72c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cdd5a640 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6bf20 0x560108c24430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 -- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 msgr2=0x560108c72c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c72c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 -- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6bf20 msgr2=0x560108c24430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6bf20 0x560108c24430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 -- 172.21.15.43:0/2414019118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560108a2cb70 con 0x560108c1ece0 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26cd559640 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c72750 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f26c40045f0 tx=0x7f26c4004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db25efe3c2bbf1f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26a7fff640 1 -- 172.21.15.43:0/2414019118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4017e60 con 0x560108c1ece0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26a7fff640 1 -- 172.21.15.43:0/2414019118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f26c4023450 con 0x560108c1ece0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b80b1040 con 0x560108c1ece0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26a7fff640 1 -- 172.21.15.43:0/2414019118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f26c4030050 con 0x560108c1ece0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 10 client.6324 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x560108a0db90 con 0x560108c1ece0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26a7fff640 1 -- 172.21.15.43:0/2414019118 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b80b1040 con 0x560108c1ece0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26a7fff640 1 -- 172.21.15.43:0/2414019118 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f26c4033020 con 0x560108c1ece0 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 10 client.6324 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 10 client.6324 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 20 client.6324 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c251a0 0x560108c255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 4 client.6324 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560108a0bdd0 con 0x560108c251a0 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26ccd58640 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c251a0 0x560108c255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26a7fff640 10 client.6324 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.762+0000 7f26ccd58640 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c251a0 0x560108c255b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c15277a515135b7c server_cookie=129685f7e610f668 in_seq=0 out_seq=0 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26a7fff640 1 -- 172.21.15.43:0/2414019118 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x560108a0bdd0 con 0x560108c251a0 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26a7fff640 10 client.6324 handle_command_reply: tid=0 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 client.6324 shutdown 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 msgr2=0x560108c72750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c72750 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f26c40045f0 tx=0x7f26c4004a50 comp rx=0 tx=0).stop 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 shutdown_connections 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c251a0 0x560108c255b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6bf20 0x560108c24430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6b600 0x560108c72c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 --2- 172.21.15.43:0/2414019118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c72750 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26a7fff640 0 client.6324 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 >> 172.21.15.43:0/2414019118 conn(0x560108c4c3a0 msgr2=0x560108c1dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 shutdown_connections 2024-03-30T13:14:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 1 -- 172.21.15.43:0/2414019118 wait complete. 2024-03-30T13:14:28.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.766+0000 7f26d293f000 20 client.6324 trim_cache size 0 max 16384 2024-03-30T13:14:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 Processor -- start 2024-03-30T13:14:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 -- start start 2024-03-30T13:14:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6b600 0x560108c26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c25580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8204670 con 0x560108c6b600 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8203670 con 0x560108c1ece0 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f26c80c9620 con 0x560108c6bf20 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6b600 0x560108c26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6b600 0x560108c26a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33660/0 (socket says 172.21.15.43:33660) 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 -- 172.21.15.43:0/4048665344 learned_addr learned my addr 172.21.15.43:0/4048665344 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26cd559640 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26cdd5a640 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c25580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 -- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 msgr2=0x560108c25580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c25580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 -- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 msgr2=0x560108c264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c264d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 -- 172.21.15.43:0/4048665344 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560108c6bd50 con 0x560108c6b600 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.782+0000 7f26ccd58640 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6b600 0x560108c26a10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f26b8006cf0 tx=0x7f26b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a4a547242c76e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26b27fc640 1 -- 172.21.15.43:0/4048665344 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80b0040 con 0x560108c6b600 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26b27fc640 1 -- 172.21.15.43:0/4048665344 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b80c60d0 con 0x560108c6b600 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26b27fc640 1 -- 172.21.15.43:0/4048665344 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b80c6480 con 0x560108c6b600 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/4048665344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6b600 msgr2=0x560108c26a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6b600 0x560108c26a10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f26b8006cf0 tx=0x7f26b8004740 comp rx=0 tx=0).stop 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/4048665344 shutdown_connections 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c25580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c6b600 0x560108c26a10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- 172.21.15.43:0/4048665344 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c1ece0 0x560108c264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/4048665344 >> 172.21.15.43:0/4048665344 conn(0x560108c4c3a0 msgr2=0x560108c78e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/4048665344 shutdown_connections 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/4048665344 wait complete. 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- start start 2024-03-30T13:14:28.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 Processor -- start 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c242a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6b600 0x560108c6e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c6e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560108c2a330 con 0x560108c1ece0 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f26c80c9c70 con 0x560108c6b600 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560108a27750 con 0x560108c6bf20 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 10 client.0 _ll_register_callbacks cb 0x7f26c4009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c242a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c242a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33664/0 (socket says 172.21.15.43:33664) 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 -- 172.21.15.43:0/3582308653 learned_addr learned my addr 172.21.15.43:0/3582308653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26ccd58640 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6b600 0x560108c6e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cdd5a640 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c6e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 -- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 msgr2=0x560108c6e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c6e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 -- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6b600 msgr2=0x560108c6e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6b600 0x560108c6e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 -- 172.21.15.43:0/3582308653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560108af1470 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26cd559640 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c242a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f26c4004310 tx=0x7f26c4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b35b05ff6c6cb5f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26a7fff640 1 -- 172.21.15.43:0/3582308653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c4023450 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26a7fff640 1 -- 172.21.15.43:0/3582308653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f26c403fc90 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b80c60d0 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26a7fff640 1 -- 172.21.15.43:0/3582308653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f26c4015620 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 10 client.6330 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5601089ff1e0 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26a7fff640 1 -- 172.21.15.43:0/3582308653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26b80c60d0 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26a7fff640 1 -- 172.21.15.43:0/3582308653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f26c4046050 con 0x560108c1ece0 2024-03-30T13:14:28.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 10 client.6330 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 10 client.6330 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 20 client.6330 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c6ed90 0x560108cdd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 4 client.6330 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.786+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560108a7e040 con 0x560108c6ed90 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26ccd58640 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c6ed90 0x560108cdd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26ccd58640 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c6ed90 0x560108cdd610 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b23e39d05d952bcb server_cookie=a6499667393ee688 in_seq=0 out_seq=0 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26a7fff640 10 client.6330 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26a7fff640 1 -- 172.21.15.43:0/3582308653 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+662 (crc 0 0 0) 0x560108a7e040 con 0x560108c6ed90 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26a7fff640 10 client.6330 handle_command_reply: tid=0 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 client.6330 shutdown 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 msgr2=0x560108c242a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c242a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f26c4004310 tx=0x7f26c4016000 comp rx=0 tx=0).stop 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 shutdown_connections 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560108c6ed90 0x560108cdd610 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560108c6bf20 0x560108c6e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560108c6b600 0x560108c6e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26a7fff640 0 client.6330 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 --2- 172.21.15.43:0/3582308653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560108c1ece0 0x560108c242a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 >> 172.21.15.43:0/3582308653 conn(0x560108c4c3a0 msgr2=0x560108c25580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 shutdown_connections 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 1 -- 172.21.15.43:0/3582308653 wait complete. 2024-03-30T13:14:28.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26d293f000 20 client.6330 trim_cache size 0 max 16384 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {}, 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_stats": { 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "b8e79244-1d55-4029-b4a0-9f0f9d72a948": { 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-03-30T13:13:58.397094+0000", 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 23, 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 23, 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:28.867 INFO:teuthology.orchestra.run.smithi043.stdout: "9ec31cb9-756a-4ece-b87a-bdc758385b7c": { 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: "start_time": "2024-03-30T13:13:15.761997+0000", 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_started": 23, 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_passed": 23, 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_failed": 0, 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: "uninline_skipped": 0 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:28.868 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f26ac067fc0 msgr2=0x7f26ac06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f26ac067fc0 0x7f26ac06a480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f26c4011340 tx=0x7f26c400b040 comp rx=0 tx=0).stop 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c815f200 msgr2=0x7f26c8154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c815f200 0x7f26c8154060 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c450 tx=0x7f26c000c910 comp rx=0 tx=0).stop 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 shutdown_connections 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f26ac067fc0 0x7f26ac06a480 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26c815f200 0x7f26c8154060 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f26c80b5450 0x7f26c80cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 --2- 172.21.15.43:0/3492640158 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f26c80b4fc0 0x7f26c8151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:28.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 >> 172.21.15.43:0/3492640158 conn(0x7f26c80b3c80 msgr2=0x7f26c815c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 shutdown_connections 2024-03-30T13:14:28.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:28.790+0000 7f26cf7bd640 1 -- 172.21.15.43:0/3492640158 wait complete. 2024-03-30T13:14:28.871 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-30T13:14:28.871 DEBUG:tasks.cephfs.test_uninlining:joining snapshotter thread 2024-03-30T13:14:28.871 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_0 2024-03-30T13:14:28.889 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_1 2024-03-30T13:14:28.950 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_2 2024-03-30T13:14:29.015 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_3 2024-03-30T13:14:29.078 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_4 2024-03-30T13:14:29.138 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_5 2024-03-30T13:14:29.198 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_6 2024-03-30T13:14:29.259 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_7 2024-03-30T13:14:29.321 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_8 2024-03-30T13:14:29.389 DEBUG:teuthology.orchestra.run.smithi113:> cd /home/ubuntu/cephtest/mnt.1 && rmdir linux-5.4/.snap/snap_9 2024-03-30T13:14:29.454 DEBUG:tasks.cephfs.test_uninlining:extracting inodes 2024-03-30T13:14:29.454 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:14:29.454 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Kconfig 2024-03-30T13:14:29.465 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628358 linux-5.4/fs/ceph/Kconfig 2024-03-30T13:14:29.465 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/Makefile 2024-03-30T13:14:29.465 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/Makefile 2024-03-30T13:14:29.518 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628359 linux-5.4/fs/ceph/Makefile 2024-03-30T13:14:29.518 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:14:29.518 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:14:29.571 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628365 linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:14:29.571 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/io.h 2024-03-30T13:14:29.571 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/io.h 2024-03-30T13:14:29.622 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628372 linux-5.4/fs/ceph/io.h 2024-03-30T13:14:29.623 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:14:29.623 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:14:29.674 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628374 linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:14:29.675 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:14:29.675 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Kconfig 2024-03-30T13:14:29.726 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628386 linux-5.4/fs/ext2/Kconfig 2024-03-30T13:14:29.726 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:14:29.726 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:14:29.778 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628404 linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:14:29.779 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:14:29.779 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:14:29.831 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628402 linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:14:29.831 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/Makefile 2024-03-30T13:14:29.831 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/Makefile 2024-03-30T13:14:29.887 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628387 linux-5.4/fs/ext2/Makefile 2024-03-30T13:14:29.887 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:14:29.888 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/symlink.c 2024-03-30T13:14:29.942 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628399 linux-5.4/fs/ext2/symlink.c 2024-03-30T13:14:29.942 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/acl.h 2024-03-30T13:14:29.942 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/acl.h 2024-03-30T13:14:29.999 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628389 linux-5.4/fs/ext2/acl.h 2024-03-30T13:14:29.999 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:14:29.999 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:14:30.050 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628403 linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:14:30.051 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:14:30.051 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/ext2/xattr.h 2024-03-30T13:14:30.106 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628401 linux-5.4/fs/ext2/xattr.h 2024-03-30T13:14:30.106 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:14:30.106 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:14:30.158 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628406 linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:14:30.158 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:14:30.159 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:14:30.210 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628419 linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:14:30.211 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:14:30.211 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:14:30.267 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628426 linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:14:30.267 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:14:30.267 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:14:30.318 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628409 linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:14:30.318 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:14:30.318 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:14:30.371 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628417 linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:14:30.371 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:14:30.371 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:14:30.422 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628421 linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:14:30.422 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:14:30.422 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:14:30.473 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628422 linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:14:30.473 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:14:30.473 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/Makefile 2024-03-30T13:14:30.526 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628407 linux-5.4/fs/orangefs/Makefile 2024-03-30T13:14:30.526 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:14:30.526 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:14:30.577 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628430 linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:14:30.577 DEBUG:tasks.cephfs.test_uninlining:getting inode for:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:14:30.577 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && ls -i linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:14:30.635 INFO:teuthology.orchestra.run.smithi043.stdout:1099511628412 linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:14:30.636 DEBUG:tasks.cephfs.test_uninlining:getting inline data info 2024-03-30T13:14:30.636 DEBUG:tasks.cephfs.test_uninlining:getting inode info #1 of 23:1099511628358 2024-03-30T13:14:30.636 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628358 2024-03-30T13:14:30.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628358 2024-03-30T13:14:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe329010640 1 Processor -- start 2024-03-30T13:14:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe329010640 1 -- start start 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe329010640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe32415ff40 0x7fe32406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe329010640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe32406d710 con 0x7fe32406cbb0 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe329010640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe32406d9f0 con 0x7fe32413f140 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe329010640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe32406e320 con 0x7fe32415ff40 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe32406cbb0 0x7fe3240ce940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54198/0 (socket says 172.21.15.43:54198) 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 -- 172.21.15.43:0/1863559943 learned_addr learned my addr 172.21.15.43:0/1863559943 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32259c640 1 --2- 172.21.15.43:0/1863559943 >> v2:172.21.15.113:3300/0 conn(0x7fe32415ff40 0x7fe32406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248268618 0 0) 0x7fe32406d710 con 0x7fe32406cbb0 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe30c003530 con 0x7fe32406cbb0 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1473538345 0 0) 0x7fe32406d9f0 con 0x7fe32413f140 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe32406d710 con 0x7fe32413f140 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2053796052 0 0) 0x7fe30c003530 con 0x7fe32406cbb0 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 >> v2:172.21.15.113:3300/0 conn(0x7fe32415ff40 msgr2=0x7fe32406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 --2- 172.21.15.43:0/1863559943 >> v2:172.21.15.113:3300/0 conn(0x7fe32415ff40 0x7fe32406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 >> v1:172.21.15.113:6789/0 conn(0x7fe32413f140 legacy=0x7fe3241606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32406dcd0 con 0x7fe32406cbb0 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fe31c001a00 con 0x7fe32406cbb0 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 >> v1:172.21.15.43:6789/0 conn(0x7fe32406cbb0 legacy=0x7fe3240ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c003b50 0x7fe30c003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c004810 0x7fe30c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c0051d0 0x7fe30c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe32406e320 con 0x7fe30c0051d0 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c00c050 con 0x7fe30c004810 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe30c00c250 con 0x7fe30c003b50 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c0051d0 0x7fe30c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32259c640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c004810 0x7fe30c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe322d9d640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c003b50 0x7fe30c003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 -- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c003b50 msgr2=0x7fe30c003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c003b50 0x7fe30c003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 -- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c004810 msgr2=0x7fe30c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c004810 0x7fe30c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe32359e640 1 -- 172.21.15.43:0/1863559943 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c003530 con 0x7fe30c0051d0 2024-03-30T13:14:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.814+0000 7fe322d9d640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe30c003b50 0x7fe30c003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe32359e640 1 -- 172.21.15.43:0/1863559943 reap_dead start 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe32359e640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c0051d0 0x7fe30c00b9d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00a810 tx=0x7fe31c01fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4390cadea33e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c00a430 con 0x7fe30c0051d0 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe31c0aa6b0 con 0x7fe30c0051d0 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe321d9b640 1 -- 172.21.15.43:0/1863559943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0aaaa0 con 0x7fe30c0051d0 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 -- 172.21.15.43:0/1863559943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c0051d0 msgr2=0x7fe30c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c0051d0 0x7fe30c00b9d0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00a810 tx=0x7fe31c01fa80 comp rx=0 tx=0).stop 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 -- 172.21.15.43:0/1863559943 shutdown_connections 2024-03-30T13:14:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 --2- 172.21.15.43:0/1863559943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe30c0051d0 0x7fe30c00b9d0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 -- 172.21.15.43:0/1863559943 >> 172.21.15.43:0/1863559943 conn(0x7fe3240d00a0 msgr2=0x7fe3240d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 -- 172.21.15.43:0/1863559943 shutdown_connections 2024-03-30T13:14:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 -- 172.21.15.43:0/1863559943 wait complete. 2024-03-30T13:14:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 Processor -- start 2024-03-30T13:14:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.818+0000 7fe329010640 1 -- start start 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0051d0 0x7fe32413b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe32413c060 0x7fe3240b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3240b4040 0x7fe32415f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe32413f6a0 con 0x7fe32413c060 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe32413f520 con 0x7fe30c0051d0 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe32406c670 con 0x7fe3240b4040 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe32413c060 0x7fe3240b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe32413c060 0x7fe3240b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33684/0 (socket says 172.21.15.43:33684) 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 -- 172.21.15.43:0/3541333932 learned_addr learned my addr 172.21.15.43:0/3541333932 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32359e640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3240b4040 0x7fe32415f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe322d9d640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0051d0 0x7fe32413b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 -- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3240b4040 msgr2=0x7fe32415f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3240b4040 0x7fe32415f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 -- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0051d0 msgr2=0x7fe32413b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0051d0 0x7fe32413b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 -- 172.21.15.43:0/3541333932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32413cd90 con 0x7fe32413c060 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe322d9d640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0051d0 0x7fe32413b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe32259c640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe32413c060 0x7fe3240b3b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe3180115c0 tx=0x7fe318011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe30b7fe640 1 -- 172.21.15.43:0/3541333932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318004220 con 0x7fe32413c060 2024-03-30T13:14:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe30b7fe640 1 -- 172.21.15.43:0/3541333932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3180043c0 con 0x7fe32413c060 2024-03-30T13:14:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3240b3020 con 0x7fe32413c060 2024-03-30T13:14:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe30b7fe640 1 -- 172.21.15.43:0/3541333932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318015930 con 0x7fe32413c060 2024-03-30T13:14:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3240b35e0 con 0x7fe32413c060 2024-03-30T13:14:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe30b7fe640 1 -- 172.21.15.43:0/3541333932 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe318015ad0 con 0x7fe32413c060 2024-03-30T13:14:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe30b7fe640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe310067fc0 0x7fe31006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe30b7fe640 1 -- 172.21.15.43:0/3541333932 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe318086e30 con 0x7fe32413c060 2024-03-30T13:14:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.822+0000 7fe322d9d640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe310067fc0 0x7fe31006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 Processor -- start 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- start start 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe310067fc0 0x7fe31006a480 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe314001700 tx=0x7fe314009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b64a0690 0x5578b64562f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b6456c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578b62a4aa0 con 0x5578b649fd70 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5578b622baf0 con 0x5578b6456830 2024-03-30T13:14:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5578b62a0a50 con 0x5578b64a0690 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33698/0 (socket says 172.21.15.43:33698) 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 -- 172.21.15.43:0/3197654211 learned_addr learned my addr 172.21.15.43:0/3197654211 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32359e640 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b6456c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32259c640 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b64a0690 0x5578b64562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 -- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b64a0690 msgr2=0x5578b64562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b64a0690 0x5578b64562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 -- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 msgr2=0x5578b6456c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b6456c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 -- 172.21.15.43:0/3197654211 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b6229d00 con 0x5578b649fd70 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32259c640 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b64a0690 0x5578b64562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe322d9d640 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a0150 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe314005360 tx=0x7fe314011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc58133e835159f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe308ff9640 1 -- 172.21.15.43:0/3197654211 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314003ba0 con 0x5578b649fd70 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe308ff9640 1 -- 172.21.15.43:0/3197654211 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe314003d60 con 0x5578b649fd70 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe308ff9640 1 -- 172.21.15.43:0/3197654211 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3140136c0 con 0x5578b649fd70 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- 172.21.15.43:0/3197654211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 msgr2=0x5578b64a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a0150 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fe314005360 tx=0x7fe314011000 comp rx=0 tx=0).stop 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- 172.21.15.43:0/3197654211 shutdown_connections 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b6456c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b64a0690 0x5578b64562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 --2- 172.21.15.43:0/3197654211 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a0150 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- 172.21.15.43:0/3197654211 >> 172.21.15.43:0/3197654211 conn(0x5578b64527e0 msgr2=0x5578b6454bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- 172.21.15.43:0/3197654211 shutdown_connections 2024-03-30T13:14:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- 172.21.15.43:0/3197654211 wait complete. 2024-03-30T13:14:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 -- start start 2024-03-30T13:14:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.826+0000 7fe32c192000 1 Processor -- start 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b645bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6484da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b64a0690 0x5578b64852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5578b626c270 con 0x5578b64a0690 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5578b62626a0 con 0x5578b6456830 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5578b623c880 con 0x5578b649fd70 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 10 client.0 _ll_register_callbacks cb 0x5578b6441490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b64a0690 0x5578b64852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b64a0690 0x5578b64852e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33714/0 (socket says 172.21.15.43:33714) 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 -- 172.21.15.43:0/748480674 learned_addr learned my addr 172.21.15.43:0/748480674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe322d9d640 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b645bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32259c640 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6484da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 -- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 msgr2=0x5578b6484da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6484da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 -- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 msgr2=0x5578b645bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b645bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 -- 172.21.15.43:0/748480674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b6269730 con 0x5578b64a0690 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32359e640 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b64a0690 0x5578b64852e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe31c09c1c0 tx=0x7fe31c01e2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76b5b0bd4a2f356 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 1 -- 172.21.15.43:0/748480674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0bb060 con 0x5578b64a0690 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe314003d60 con 0x5578b64a0690 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 1 -- 172.21.15.43:0/748480674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe31c0aa4b0 con 0x5578b64a0690 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 1 -- 172.21.15.43:0/748480674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe31c0c1050 con 0x5578b64a0690 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 10 client.6342 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5578b625cca0 con 0x5578b64a0690 2024-03-30T13:14:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 1 -- 172.21.15.43:0/748480674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe314003d60 con 0x5578b64a0690 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 1 -- 172.21.15.43:0/748480674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fe31c0c4020 con 0x5578b64a0690 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 10 client.6342 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 10 client.6342 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 20 client.6342 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a75d0 0x5578b64a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 4 client.6342 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578b623fc90 con 0x5578b64a75d0 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe322d9d640 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a75d0 0x5578b64a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe322d9d640 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a75d0 0x5578b64a79b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9672d40dbfb6dbea server_cookie=9af78dcc79210d1d in_seq=0 out_seq=0 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 10 client.6342 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 1 -- 172.21.15.43:0/748480674 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5578b623fc90 con 0x5578b64a75d0 2024-03-30T13:14:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 10 client.6342 handle_command_reply: tid=0 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 client.6342 shutdown 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b64a0690 msgr2=0x5578b64852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b64a0690 0x5578b64852e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe31c09c1c0 tx=0x7fe31c01e2c0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 shutdown_connections 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a75d0 0x5578b64a79b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b64a0690 0x5578b64852e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6484da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 --2- 172.21.15.43:0/748480674 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b6456830 0x5578b645bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe301ffb640 0 client.6342 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:30.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 >> 172.21.15.43:0/748480674 conn(0x5578b64527e0 msgr2=0x5578b6455460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 shutdown_connections 2024-03-30T13:14:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.830+0000 7fe32c192000 1 -- 172.21.15.43:0/748480674 wait complete. 2024-03-30T13:14:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.834+0000 7fe32c192000 20 client.6342 trim_cache size 0 max 16384 2024-03-30T13:14:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 Processor -- start 2024-03-30T13:14:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 -- start start 2024-03-30T13:14:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b6456830 0x5578b6485180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe324201720 con 0x5578b649fd70 2024-03-30T13:14:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe3242007c0 con 0x5578b64a0690 2024-03-30T13:14:30.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe324144bf0 con 0x5578b6456830 2024-03-30T13:14:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a2a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33718/0 (socket says 172.21.15.43:33718) 2024-03-30T13:14:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 -- 172.21.15.43:0/2841797305 learned_addr learned my addr 172.21.15.43:0/2841797305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32359e640 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe322d9d640 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b6456830 0x5578b6485180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 -- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b6456830 msgr2=0x5578b6485180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b6456830 0x5578b6485180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 -- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 msgr2=0x5578b64a2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe32259c640 1 -- 172.21.15.43:0/2841797305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578b622bd00 con 0x5578b649fd70 2024-03-30T13:14:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.846+0000 7fe322d9d640 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b6456830 0x5578b6485180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:30.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32259c640 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a2a80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe318056b90 tx=0x7fe318056c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1db1981e7669f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe303fff640 1 -- 172.21.15.43:0/2841797305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318022040 con 0x5578b649fd70 2024-03-30T13:14:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe303fff640 1 -- 172.21.15.43:0/2841797305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe318054f00 con 0x5578b649fd70 2024-03-30T13:14:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe303fff640 1 -- 172.21.15.43:0/2841797305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318056050 con 0x5578b649fd70 2024-03-30T13:14:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/2841797305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 msgr2=0x5578b64a2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a2a80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe318056b90 tx=0x7fe318056c00 comp rx=0 tx=0).stop 2024-03-30T13:14:30.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/2841797305 shutdown_connections 2024-03-30T13:14:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b649fd70 0x5578b64a2a80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- 172.21.15.43:0/2841797305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b6456830 0x5578b6485180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/2841797305 >> 172.21.15.43:0/2841797305 conn(0x5578b64527e0 msgr2=0x5578b64b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/2841797305 shutdown_connections 2024-03-30T13:14:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/2841797305 wait complete. 2024-03-30T13:14:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- start start 2024-03-30T13:14:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 Processor -- start 2024-03-30T13:14:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b6456830 0x5578b6512240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6512780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe324200170 con 0x5578b6456830 2024-03-30T13:14:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5578b6360b00 con 0x5578b64a0690 2024-03-30T13:14:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5578b623ece0 con 0x5578b649fd70 2024-03-30T13:14:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 10 client.0 _ll_register_callbacks cb 0x7fe30c003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b6456830 0x5578b6512240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b6456830 0x5578b6512240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33728/0 (socket says 172.21.15.43:33728) 2024-03-30T13:14:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 -- 172.21.15.43:0/1983034241 learned_addr learned my addr 172.21.15.43:0/1983034241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32259c640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6512780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32359e640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 -- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 msgr2=0x5578b6512780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6512780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 -- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 msgr2=0x5578b64a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 -- 172.21.15.43:0/1983034241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578b6325f70 con 0x5578b6456830 2024-03-30T13:14:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32359e640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe322d9d640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b6456830 0x5578b6512240 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e590 tx=0x7fe3140012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a6c198174515cc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe301ffb640 1 -- 172.21.15.43:0/1983034241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314038060 con 0x5578b6456830 2024-03-30T13:14:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe301ffb640 1 -- 172.21.15.43:0/1983034241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7fe3140067e0 con 0x5578b6456830 2024-03-30T13:14:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe318054f00 con 0x5578b6456830 2024-03-30T13:14:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe301ffb640 1 -- 172.21.15.43:0/1983034241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe314006a20 con 0x5578b6456830 2024-03-30T13:14:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 10 client.6348 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5578b624b760 con 0x5578b6456830 2024-03-30T13:14:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe301ffb640 1 -- 172.21.15.43:0/1983034241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe318054f00 con 0x5578b6456830 2024-03-30T13:14:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe301ffb640 1 -- 172.21.15.43:0/1983034241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fe314006bc0 con 0x5578b6456830 2024-03-30T13:14:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 10 client.6348 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 10 client.6348 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 20 client.6348 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a4010 0x5578b64a4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 4 client.6348 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628358}] 2024-03-30T13:14:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.850+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628358}) -- 0x5578b62b1fe0 con 0x5578b64a4010 2024-03-30T13:14:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32259c640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a4010 0x5578b64a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32259c640 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a4010 0x5578b64a4420 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fd1f080e0dcf88 server_cookie=43e6fedad530c6e8 in_seq=0 out_seq=0 2024-03-30T13:14:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe301ffb640 10 client.6348 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe301ffb640 1 -- 172.21.15.43:0/1983034241 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3794 (crc 0 0 0) 0x5578b62b1fe0 con 0x5578b64a4010 2024-03-30T13:14:30.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe301ffb640 10 client.6348 handle_command_reply: tid=0 2024-03-30T13:14:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 client.6348 shutdown 2024-03-30T13:14:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b6456830 msgr2=0x5578b6512240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b6456830 0x5578b6512240 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e590 tx=0x7fe3140012e0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 shutdown_connections 2024-03-30T13:14:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5578b64a4010 0x5578b64a4420 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5578b64a0690 0x5578b64a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5578b649fd70 0x5578b6512780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 --2- 172.21.15.43:0/1983034241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5578b6456830 0x5578b6512240 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe301ffb640 0 client.6348 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 >> 172.21.15.43:0/1983034241 conn(0x5578b64527e0 msgr2=0x5578b64b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 shutdown_connections 2024-03-30T13:14:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 1 -- 172.21.15.43:0/1983034241 wait complete. 2024-03-30T13:14:30.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe32c192000 20 client.6348 trim_cache size 0 max 16384 2024-03-30T13:14:30.879 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:30.879 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/Kconfig", 2024-03-30T13:14:30.879 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628358, 2024-03-30T13:14:30.880 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:30.880 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.465610+0000", 2024-03-30T13:14:30.880 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.445469+0000", 2024-03-30T13:14:30.880 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:30.880 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:30.880 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:30.881 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:30.882 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:30.882 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:30.882 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:30.882 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:30.882 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.882 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:30.882 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1481, 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.445469+0000", 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:30.883 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:30.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1481, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.465610+0000" 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1481, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:30.885 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.465610+0000" 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 190, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 2, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:30.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:30.887 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:30.888 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:30.889 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:30.890 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:30.891 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:30.892 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:30.893 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:30.894 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe310067fc0 msgr2=0x7fe31006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe310067fc0 0x7fe31006a480 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe314001700 tx=0x7fe314009040 comp rx=0 tx=0).stop 2024-03-30T13:14:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe32413c060 msgr2=0x7fe3240b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:30.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe32413c060 0x7fe3240b3b00 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe3180115c0 tx=0x7fe318011a80 comp rx=0 tx=0).stop 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 shutdown_connections 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe310067fc0 0x7fe31006a480 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe3240b4040 0x7fe32415f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe32413c060 0x7fe3240b3b00 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 --2- 172.21.15.43:0/3541333932 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe30c0051d0 0x7fe32413b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 >> 172.21.15.43:0/3541333932 conn(0x7fe3240d00a0 msgr2=0x7fe3240cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 shutdown_connections 2024-03-30T13:14:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:30.854+0000 7fe329010640 1 -- 172.21.15.43:0/3541333932 wait complete. 2024-03-30T13:14:30.896 DEBUG:tasks.cephfs.test_uninlining:getting inode info #2 of 23:1099511628359 2024-03-30T13:14:30.897 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628359 2024-03-30T13:14:30.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628359 2024-03-30T13:14:31.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf2efb640 1 Processor -- start 2024-03-30T13:14:31.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf2efb640 1 -- start start 2024-03-30T13:14:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf2efb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec155160 0x7f0cec0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf2efb640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0cec06cd80 0x7f0cec155540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf2efb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cec15fa40 con 0x7f0cec15f5e0 2024-03-30T13:14:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf2efb640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0cec162830 con 0x7f0cec06cd80 2024-03-30T13:14:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf2efb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec162b10 con 0x7f0cec155160 2024-03-30T13:14:31.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cebfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0cec15f5e0 0x7f0cec06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54208/0 (socket says 172.21.15.43:54208) 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cebfff640 1 -- 172.21.15.43:0/3142717356 learned_addr learned my addr 172.21.15.43:0/3142717356 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3142717356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec155160 0x7f0cec0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf1498640 1 --2- 172.21.15.43:0/3142717356 >> v2:172.21.15.113:3300/0 conn(0x7f0cec06cd80 0x7f0cec155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0ceb7fe640 1 -- 172.21.15.43:0/3142717356 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1899599501 0 0) 0x7f0cd40014a0 con 0x7f0cec15f5e0 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0ceb7fe640 1 -- 172.21.15.43:0/3142717356 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cd8003530 con 0x7f0cec15f5e0 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf0c97640 1 -- 172.21.15.43:0/3142717356 >> v2:172.21.15.113:3300/0 conn(0x7f0cec06cd80 msgr2=0x7f0cec155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3142717356 >> v2:172.21.15.113:3300/0 conn(0x7f0cec06cd80 0x7f0cec155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0ceb7fe640 1 -- 172.21.15.43:0/3142717356 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 781811307 0 0) 0x7f0cd8003530 con 0x7f0cec15f5e0 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf0c97640 1 -- 172.21.15.43:0/3142717356 >> v1:172.21.15.43:6789/0 conn(0x7f0cec15f5e0 legacy=0x7f0cec06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.014+0000 7f0cf0c97640 1 -- 172.21.15.43:0/3142717356 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec162df0 con 0x7f0cec155160 2024-03-30T13:14:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3142717356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec155160 0x7f0cec0b2e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40097c0 tx=0x7f0ce401c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=178560c18d115636 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0ceb7fe640 1 -- 172.21.15.43:0/3142717356 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40b0070 con 0x7f0cec155160 2024-03-30T13:14:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0ceb7fe640 1 -- 172.21.15.43:0/3142717356 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce40ab400 con 0x7f0cec155160 2024-03-30T13:14:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0ceb7fe640 1 -- 172.21.15.43:0/3142717356 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4001bd0 con 0x7f0cec155160 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 -- 172.21.15.43:0/3142717356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec155160 msgr2=0x7f0cec0b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/3142717356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec155160 0x7f0cec0b2e80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40097c0 tx=0x7f0ce401c390 comp rx=0 tx=0).stop 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 -- 172.21.15.43:0/3142717356 shutdown_connections 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/3142717356 >> v2:172.21.15.113:3300/0 conn(0x7f0cec06cd80 0x7f0cec155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/3142717356 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec155160 0x7f0cec0b2e80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 -- 172.21.15.43:0/3142717356 >> 172.21.15.43:0/3142717356 conn(0x7f0cec0d00a0 msgr2=0x7f0cec0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 -- 172.21.15.43:0/3142717356 shutdown_connections 2024-03-30T13:14:31.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.018+0000 7f0cf2efb640 1 -- 172.21.15.43:0/3142717356 wait complete. 2024-03-30T13:14:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 Processor -- start 2024-03-30T13:14:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 -- start start 2024-03-30T13:14:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec06cd80 0x7f0cec06bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cec155160 0x7f0cec14bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cec15f5e0 0x7f0cec14c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec072870 con 0x7f0cec155160 2024-03-30T13:14:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec0726f0 con 0x7f0cec15f5e0 2024-03-30T13:14:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec0729f0 con 0x7f0cec06cd80 2024-03-30T13:14:31.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cec155160 0x7f0cec14bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cec155160 0x7f0cec14bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33740/0 (socket says 172.21.15.43:33740) 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 -- 172.21.15.43:0/1797642019 learned_addr learned my addr 172.21.15.43:0/1797642019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec06cd80 0x7f0cec06bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf1498640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cec15f5e0 0x7f0cec14c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 -- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec06cd80 msgr2=0x7f0cec06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec06cd80 0x7f0cec06bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 -- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cec15f5e0 msgr2=0x7f0cec14c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cec15f5e0 0x7f0cec14c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 -- 172.21.15.43:0/1797642019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cec156910 con 0x7f0cec155160 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec06cd80 0x7f0cec06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cebfff640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cec155160 0x7f0cec14bfb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400cc50 tx=0x7f0cd400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0ce97fa640 1 -- 172.21.15.43:0/1797642019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4017070 con 0x7f0cec155160 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0ce97fa640 1 -- 172.21.15.43:0/1797642019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd40070f0 con 0x7f0cec155160 2024-03-30T13:14:31.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cec156bf0 con 0x7f0cec155160 2024-03-30T13:14:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0ce97fa640 1 -- 172.21.15.43:0/1797642019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400ce60 con 0x7f0cec155160 2024-03-30T13:14:31.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cec1571b0 con 0x7f0cec155160 2024-03-30T13:14:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.022+0000 7f0ce97fa640 1 -- 172.21.15.43:0/1797642019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0cd401a070 con 0x7f0cec155160 2024-03-30T13:14:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0ce97fa640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0ccc067fc0 0x7f0ccc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0ce97fa640 1 -- 172.21.15.43:0/1797642019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0cd4086660 con 0x7f0cec155160 2024-03-30T13:14:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0ccc067fc0 0x7f0ccc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 Processor -- start 2024-03-30T13:14:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- start start 2024-03-30T13:14:31.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0ccc067fc0 0x7f0ccc06a480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40ad000 tx=0x7f0ce40a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c470999150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47094f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47094fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c47079daa0 con 0x55c470999690 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c470724af0 con 0x55c47094f830 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c470799a50 con 0x55c470998d70 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47094f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47094f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33742/0 (socket says 172.21.15.43:33742) 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 -- 172.21.15.43:0/3461357848 learned_addr learned my addr 172.21.15.43:0/3461357848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf1498640 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47094fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c470999150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 -- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 msgr2=0x55c470999150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c470999150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 -- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 msgr2=0x55c47094fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47094fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 -- 172.21.15.43:0/3461357848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c470722d00 con 0x55c470999690 2024-03-30T13:14:31.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cebfff640 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47094f2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0cd405ec90 tx=0x7f0cd4056050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355e55dbb3351fa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0ce27fc640 1 -- 172.21.15.43:0/3461357848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4054680 con 0x55c470999690 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0ce27fc640 1 -- 172.21.15.43:0/3461357848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd40548e0 con 0x55c470999690 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0ce27fc640 1 -- 172.21.15.43:0/3461357848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd405ed30 con 0x55c470999690 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- 172.21.15.43:0/3461357848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 msgr2=0x55c47094f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47094f2f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0cd405ec90 tx=0x7f0cd4056050 comp rx=0 tx=0).stop 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- 172.21.15.43:0/3461357848 shutdown_connections 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47094fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47094f2f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3461357848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c470999150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- 172.21.15.43:0/3461357848 >> 172.21.15.43:0/3461357848 conn(0x55c47094b7e0 msgr2=0x55c47094dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- 172.21.15.43:0/3461357848 shutdown_connections 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.026+0000 7f0cf607d000 1 -- 172.21.15.43:0/3461357848 wait complete. 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 -- start start 2024-03-30T13:14:31.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 Processor -- start 2024-03-30T13:14:31.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c47094f830 0x55c470954c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c47097dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c470999690 0x55c47097e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c470765270 con 0x55c470998d70 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c47075b6a0 con 0x55c470999690 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c470735880 con 0x55c47094f830 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 10 client.0 _ll_register_callbacks cb 0x55c47093a490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c47097dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c47097dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33750/0 (socket says 172.21.15.43:33750) 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 -- 172.21.15.43:0/3562006014 learned_addr learned my addr 172.21.15.43:0/3562006014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf1498640 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c470999690 0x55c47097e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c47094f830 0x55c470954c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 -- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c47094f830 msgr2=0x55c470954c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c47094f830 0x55c470954c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 -- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c470999690 msgr2=0x55c47097e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c470999690 0x55c47097e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 -- 172.21.15.43:0/3562006014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c470762730 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cebfff640 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c47097dda0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400a930 tx=0x7f0cd4001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52d5dee96320011 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0ccb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/3562006014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4073060 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/3562006014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0cd405ef40 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cd40548e0 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/3562006014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0cd406ebd0 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 10 client.6360 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55c470755ca0 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/3562006014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd40548e0 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/3562006014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f0cd4004b70 con 0x55c470998d70 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 10 client.6360 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 10 client.6360 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 20 client.6360 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c4709a05d0 0x55c4709a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 4 client.6360 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c470738c90 con 0x55c4709a05d0 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c4709a05d0 0x55c4709a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c4709a05d0 0x55c4709a09e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8985257262a8c334 server_cookie=94f85dfba7e5418c in_seq=0 out_seq=0 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.030+0000 7f0ccb7fe640 10 client.6360 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/3562006014 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55c470738c90 con 0x55c4709a05d0 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0ccb7fe640 10 client.6360 handle_command_reply: tid=0 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 client.6360 shutdown 2024-03-30T13:14:31.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 msgr2=0x55c47097dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c47097dda0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400a930 tx=0x7f0cd4001260 comp rx=0 tx=0).stop 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 shutdown_connections 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c4709a05d0 0x55c4709a09e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c470999690 0x55c47097e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c47097dda0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0ccb7fe640 0 client.6360 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3562006014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c47094f830 0x55c470954c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 >> 172.21.15.43:0/3562006014 conn(0x55c47094b7e0 msgr2=0x55c47094e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 shutdown_connections 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 1 -- 172.21.15.43:0/3562006014 wait complete. 2024-03-30T13:14:31.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.034+0000 7f0cf607d000 20 client.6360 trim_cache size 0 max 16384 2024-03-30T13:14:31.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.046+0000 7f0cf607d000 1 Processor -- start 2024-03-30T13:14:31.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.046+0000 7f0cf607d000 1 -- start start 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47097cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c47097d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47097da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec204150 con 0x55c470999690 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec203130 con 0x55c47094f830 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0cec0cde20 con 0x55c470998d70 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47097da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47097da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33764/0 (socket says 172.21.15.43:33764) 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 -- 172.21.15.43:0/3392763860 learned_addr learned my addr 172.21.15.43:0/3392763860 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47097cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c47097d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 -- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 msgr2=0x55c47097d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c47097d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 -- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 msgr2=0x55c47097cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47097cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 -- 172.21.15.43:0/3392763860 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c470724d00 con 0x55c470999690 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47097da90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009b80 tx=0x7f0cdc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d0c5f4e1a8f64c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0ce1ffb640 1 -- 172.21.15.43:0/3392763860 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc01b070 con 0x55c470999690 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0ce1ffb640 1 -- 172.21.15.43:0/3392763860 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cdc015e70 con 0x55c470999690 2024-03-30T13:14:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0ce1ffb640 1 -- 172.21.15.43:0/3392763860 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc004250 con 0x55c470999690 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- 172.21.15.43:0/3392763860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 msgr2=0x55c47097da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47097da90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009b80 tx=0x7f0cdc009190 comp rx=0 tx=0).stop 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- 172.21.15.43:0/3392763860 shutdown_connections 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470999690 0x55c47097da90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470998d70 0x55c47097d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- 172.21.15.43:0/3392763860 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c47097cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- 172.21.15.43:0/3392763860 >> 172.21.15.43:0/3392763860 conn(0x55c47094b7e0 msgr2=0x55c4709af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- 172.21.15.43:0/3392763860 shutdown_connections 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- 172.21.15.43:0/3392763860 wait complete. 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- start start 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 Processor -- start 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c470a0b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c470a0b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470999690 0x55c47099cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c470735880 con 0x55c470998d70 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0cec202ae0 con 0x55c47094f830 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c470859b00 con 0x55c470999690 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf607d000 10 client.0 _ll_register_callbacks cb 0x7f0cd406d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c470a0b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c470a0b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33780/0 (socket says 172.21.15.43:33780) 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 -- 172.21.15.43:0/4270249971 learned_addr learned my addr 172.21.15.43:0/4270249971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf1498640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470999690 0x55c47099cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c470a0b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 -- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470999690 msgr2=0x55c47099cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470999690 0x55c47099cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 -- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 msgr2=0x55c470a0b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c470a0b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cebfff640 1 -- 172.21.15.43:0/4270249971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c47081ef70 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.050+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c470a0b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cebfff640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c470a0b860 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4086980 tx=0x7f0cd4056460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8505fd8284776337 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/4270249971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd40548e0 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/4270249971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f0cd401dcb0 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cdc015e70 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/4270249971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0cd4073020 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 10 client.6366 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55c470744760 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/4270249971 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cdc015e70 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/4270249971 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f0cd4055af0 con 0x55c470998d70 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 10 client.6366 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 10 client.6366 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 20 client.6366 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c47099d010 0x55c47099d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 4 client.6366 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628359}] 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628359}) -- 0x55c4707aafe0 con 0x55c47099d010 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c47099d010 0x55c47099d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 10 client.6366 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf0c97640 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c47099d010 0x55c47099d420 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64b3b2931081ead server_cookie=a7880be28b6b1002 in_seq=0 out_seq=0 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 1 -- 172.21.15.43:0/4270249971 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3792 (crc 0 0 0) 0x55c4707aafe0 con 0x55c47099d010 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 10 client.6366 handle_command_reply: tid=0 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 client.6366 shutdown 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 msgr2=0x55c470a0b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c470a0b860 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4086980 tx=0x7f0cd4056460 comp rx=0 tx=0).stop 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 shutdown_connections 2024-03-30T13:14:31.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c47099d010 0x55c47099d420 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c470999690 0x55c47099cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0ccb7fe640 0 client.6366 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c470998d70 0x55c470a0b860 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 --2- 172.21.15.43:0/4270249971 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c47094f830 0x55c470a0b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 >> 172.21.15.43:0/4270249971 conn(0x55c47094b7e0 msgr2=0x55c4709aff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 shutdown_connections 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 1 -- 172.21.15.43:0/4270249971 wait complete. 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf607d000 20 client.6366 trim_cache size 0 max 16384 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/Makefile", 2024-03-30T13:14:31.064 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628359, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.512804+0000", 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.450173+0000", 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:31.065 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 356, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:31.066 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.450173+0000", 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:31.067 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.512804+0000" 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.068 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.512804+0000" 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 218, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 4, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:31.069 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:31.070 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:31.071 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.072 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.073 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.074 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.075 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:31.076 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:31.076 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0ccc067fc0 msgr2=0x7f0ccc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0ccc067fc0 0x7f0ccc06a480 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40ad000 tx=0x7f0ce40a6000 comp rx=0 tx=0).stop 2024-03-30T13:14:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cec155160 msgr2=0x7f0cec14bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cec155160 0x7f0cec14bfb0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400cc50 tx=0x7f0cd400a430 comp rx=0 tx=0).stop 2024-03-30T13:14:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 shutdown_connections 2024-03-30T13:14:31.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0ccc067fc0 0x7f0ccc06a480 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0cec15f5e0 0x7f0cec14c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0cec155160 0x7f0cec14bfb0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 --2- 172.21.15.43:0/1797642019 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0cec06cd80 0x7f0cec06bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 >> 172.21.15.43:0/1797642019 conn(0x7f0cec0d00a0 msgr2=0x7f0cec0d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 shutdown_connections 2024-03-30T13:14:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.054+0000 7f0cf2efb640 1 -- 172.21.15.43:0/1797642019 wait complete. 2024-03-30T13:14:31.078 DEBUG:tasks.cephfs.test_uninlining:getting inode info #3 of 23:1099511628365 2024-03-30T13:14:31.078 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628365 2024-03-30T13:14:31.078 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628365 2024-03-30T13:14:31.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.190+0000 7f5103098640 1 Processor -- start 2024-03-30T13:14:31.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.190+0000 7f5103098640 1 -- start start 2024-03-30T13:14:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5103098640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f50fc15ff40 0x7f50fc06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5103098640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc06cbb0 0x7f50fc0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5103098640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50fc06d710 con 0x7f50fc13f140 2024-03-30T13:14:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5103098640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f50fc06e320 con 0x7f50fc15ff40 2024-03-30T13:14:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5103098640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f50fc160de0 con 0x7f50fc06cbb0 2024-03-30T13:14:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f50fbfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f50fc15ff40 0x7f50fc06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5100e34640 1 --1- >> v1:172.21.15.113:6789/0 conn(0x7f50fc13f140 0x7f50fc1606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.113:6789/0 says I am v1:172.21.15.43:48986/0 (socket says 172.21.15.43:48986) 2024-03-30T13:14:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5100e34640 1 -- 172.21.15.43:0/2429474619 learned_addr learned my addr 172.21.15.43:0/2429474619 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:31.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f5101635640 1 --2- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc06cbb0 0x7f50fc0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f50fbfff640 1 -- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc06cbb0 msgr2=0x7f50fc0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f50fbfff640 1 --2- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc06cbb0 0x7f50fc0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f50fbfff640 1 -- 172.21.15.43:0/2429474619 >> v1:172.21.15.113:6789/0 conn(0x7f50fc13f140 legacy=0x7f50fc1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.194+0000 7f50fbfff640 1 -- 172.21.15.43:0/2429474619 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc06d9f0 con 0x7f50fc15ff40 2024-03-30T13:14:31.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f50fbfff640 1 --2- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc15ff40 0x7f50fc06c670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097c0 tx=0x7f50e401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb4f59705fcfa66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f50fb7fe640 1 -- 172.21.15.43:0/2429474619 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40b2070 con 0x7f50fc15ff40 2024-03-30T13:14:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f50fb7fe640 1 -- 172.21.15.43:0/2429474619 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e40ad640 con 0x7f50fc15ff40 2024-03-30T13:14:31.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f50fb7fe640 1 -- 172.21.15.43:0/2429474619 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40ada10 con 0x7f50fc15ff40 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 -- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc15ff40 msgr2=0x7f50fc06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 --2- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc15ff40 0x7f50fc06c670 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f50e40097c0 tx=0x7f50e401c4a0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 -- 172.21.15.43:0/2429474619 shutdown_connections 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 --2- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc06cbb0 0x7f50fc0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 --2- 172.21.15.43:0/2429474619 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc15ff40 0x7f50fc06c670 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 -- 172.21.15.43:0/2429474619 >> 172.21.15.43:0/2429474619 conn(0x7f50fc0d00a0 msgr2=0x7f50fc0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 -- 172.21.15.43:0/2429474619 shutdown_connections 2024-03-30T13:14:31.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 -- 172.21.15.43:0/2429474619 wait complete. 2024-03-30T13:14:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 Processor -- start 2024-03-30T13:14:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 -- start start 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.198+0000 7f5103098640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc06cbb0 0x7f50fc13dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5103098640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc13f140 0x7f50fc152af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5103098640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ff40 0x7f50fc154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5103098640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0cf810 con 0x7f50fc15ff40 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5103098640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0cf690 con 0x7f50fc06cbb0 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5103098640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f50fc0cf990 con 0x7f50fc13f140 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ff40 0x7f50fc154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ff40 0x7f50fc154230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33786/0 (socket says 172.21.15.43:33786) 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 -- 172.21.15.43:0/2280794900 learned_addr learned my addr 172.21.15.43:0/2280794900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5100e34640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc06cbb0 0x7f50fc13dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50fbfff640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc13f140 0x7f50fc152af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 -- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc13f140 msgr2=0x7f50fc152af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc13f140 0x7f50fc152af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 -- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc06cbb0 msgr2=0x7f50fc13dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc06cbb0 0x7f50fc13dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 -- 172.21.15.43:0/2280794900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc13dba0 con 0x7f50fc15ff40 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50fbfff640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc13f140 0x7f50fc152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5101635640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ff40 0x7f50fc154230 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f50f400efc0 tx=0x7f50f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50f97fa640 1 -- 172.21.15.43:0/2280794900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4018020 con 0x7f50fc15ff40 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50f97fa640 1 -- 172.21.15.43:0/2280794900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f4014480 con 0x7f50fc15ff40 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc151d30 con 0x7f50fc15ff40 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50f97fa640 1 -- 172.21.15.43:0/2280794900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4014870 con 0x7f50fc15ff40 2024-03-30T13:14:31.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1522f0 con 0x7f50fc15ff40 2024-03-30T13:14:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50f97fa640 1 -- 172.21.15.43:0/2280794900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f50f401b020 con 0x7f50fc15ff40 2024-03-30T13:14:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50f97fa640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50dc067ff0 0x7f50dc06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f50f97fa640 1 -- 172.21.15.43:0/2280794900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f50f4087010 con 0x7f50fc15ff40 2024-03-30T13:14:31.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.202+0000 7f5100e34640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50dc067ff0 0x7f50dc06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f5100e34640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50dc067ff0 0x7f50dc06a4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f50fc13c810 tx=0x7f50ec007e90 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 Processor -- start 2024-03-30T13:14:31.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- start start 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51933a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a5149700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a514a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e3a4f93aa0 con 0x55e3a51938e0 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e3a4f1aaf0 con 0x55e3a5192fc0 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e3a4f8fa50 con 0x55e3a5149c40 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a5149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a5149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33796/0 (socket says 172.21.15.43:33796) 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 -- 172.21.15.43:0/1804528573 learned_addr learned my addr 172.21.15.43:0/1804528573 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f5100e34640 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f5101635640 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a514a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 -- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 msgr2=0x55e3a514a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a514a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 -- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 msgr2=0x55e3a51933a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51933a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 -- 172.21.15.43:0/1804528573 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a4f18d00 con 0x55e3a51938e0 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f5100e34640 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50fbfff640 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a5149700 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f50e401c4e0 tx=0x7f50e40a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eafeebde62291c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50f27fc640 1 -- 172.21.15.43:0/1804528573 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40b2040 con 0x55e3a51938e0 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50f27fc640 1 -- 172.21.15.43:0/1804528573 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e40ad4c0 con 0x55e3a51938e0 2024-03-30T13:14:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f50f27fc640 1 -- 172.21.15.43:0/1804528573 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e40ad870 con 0x55e3a51938e0 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- 172.21.15.43:0/1804528573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 msgr2=0x55e3a5149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a5149700 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f50e401c4e0 tx=0x7f50e40a7070 comp rx=0 tx=0).stop 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- 172.21.15.43:0/1804528573 shutdown_connections 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a514a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a5149700 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 --2- 172.21.15.43:0/1804528573 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51933a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- 172.21.15.43:0/1804528573 >> 172.21.15.43:0/1804528573 conn(0x55e3a51417e0 msgr2=0x55e3a5143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- 172.21.15.43:0/1804528573 shutdown_connections 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- 172.21.15.43:0/1804528573 wait complete. 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 -- start start 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.206+0000 7f510621a000 1 Processor -- start 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a5192890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a514f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e3a4f5b270 con 0x55e3a5149c40 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e3a4f516a0 con 0x55e3a5192fc0 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e3a4f2b880 con 0x55e3a51938e0 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 10 client.0 _ll_register_callbacks cb 0x55e3a5130490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a5192890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a5192890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33804/0 (socket says 172.21.15.43:33804) 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 -- 172.21.15.43:0/501081012 learned_addr learned my addr 172.21.15.43:0/501081012 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5101635640 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50fbfff640 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a514f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 -- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 msgr2=0x55e3a514fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 -- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 msgr2=0x55e3a514f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a514f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 -- 172.21.15.43:0/501081012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a4f58730 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f5100e34640 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a5192890 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0134a0 tx=0x7f50ec000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bebe65589101c92 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50db7fe640 1 -- 172.21.15.43:0/501081012 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec027060 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50db7fe640 1 -- 172.21.15.43:0/501081012 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f50ec022c70 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e40ad4c0 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50db7fe640 1 -- 172.21.15.43:0/501081012 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f50ec012560 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 10 client.6375 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55e3a4f4bca0 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50db7fe640 1 -- 172.21.15.43:0/501081012 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50e40ad4c0 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50db7fe640 1 -- 172.21.15.43:0/501081012 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f50ec012700 con 0x55e3a5149c40 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 10 client.6375 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 10 client.6375 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 20 client.6375 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a5150b30 0x55e3a5150f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 4 client.6375 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:31.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3a4f2ec90 con 0x55e3a5150b30 2024-03-30T13:14:31.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50fbfff640 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a5150b30 0x55e3a5150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50fbfff640 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a5150b30 0x55e3a5150f10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2800125c996b5091 server_cookie=2df7a9220523fc47 in_seq=0 out_seq=0 2024-03-30T13:14:31.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.210+0000 7f50db7fe640 10 client.6375 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f50db7fe640 1 -- 172.21.15.43:0/501081012 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55e3a4f2ec90 con 0x55e3a5150b30 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f50db7fe640 10 client.6375 handle_command_reply: tid=0 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 client.6375 shutdown 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 msgr2=0x55e3a5192890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a5192890 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0134a0 tx=0x7f50ec000bd0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 shutdown_connections 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a5150b30 0x55e3a5150f10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a514f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f50db7fe640 0 client.6375 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 --2- 172.21.15.43:0/501081012 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a5192890 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 >> 172.21.15.43:0/501081012 conn(0x55e3a51417e0 msgr2=0x55e3a514a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 shutdown_connections 2024-03-30T13:14:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 1 -- 172.21.15.43:0/501081012 wait complete. 2024-03-30T13:14:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.214+0000 7f510621a000 20 client.6375 trim_cache size 0 max 16384 2024-03-30T13:14:31.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 Processor -- start 2024-03-30T13:14:31.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 -- start start 2024-03-30T13:14:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a51aa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51aa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1ff9a0 con 0x55e3a5149c40 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0b49d0 con 0x55e3a5192fc0 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.226+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f50fc13d400 con 0x55e3a51938e0 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a51aa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a51aa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32830/0 (socket says 172.21.15.43:32830) 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 -- 172.21.15.43:0/2114071770 learned_addr learned my addr 172.21.15.43:0/2114071770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50fbfff640 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51aa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 -- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 msgr2=0x55e3a514a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 -- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 msgr2=0x55e3a51aa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51aa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 -- 172.21.15.43:0/2114071770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3a5193710 con 0x55e3a5149c40 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50fbfff640 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a51aa2d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f50ec004b90 tx=0x7f50ec020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c1c306e8cf594d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50f1ffb640 1 -- 172.21.15.43:0/2114071770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0127c0 con 0x55e3a5149c40 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50f1ffb640 1 -- 172.21.15.43:0/2114071770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec012c50 con 0x55e3a5149c40 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50f1ffb640 1 -- 172.21.15.43:0/2114071770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec012e00 con 0x55e3a5149c40 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- 172.21.15.43:0/2114071770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 msgr2=0x55e3a51aa2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a51aa2d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f50ec004b90 tx=0x7f50ec020000 comp rx=0 tx=0).stop 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- 172.21.15.43:0/2114071770 shutdown_connections 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a51938e0 0x55e3a514a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a51aa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 --2- 172.21.15.43:0/2114071770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a5149c40 0x55e3a51aa2d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- 172.21.15.43:0/2114071770 >> 172.21.15.43:0/2114071770 conn(0x55e3a51417e0 msgr2=0x55e3a51735e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- 172.21.15.43:0/2114071770 shutdown_connections 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- 172.21.15.43:0/2114071770 wait complete. 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- start start 2024-03-30T13:14:31.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 Processor -- start 2024-03-30T13:14:31.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a52050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a5205610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a51a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e3a4f516a0 con 0x55e3a51938e0 2024-03-30T13:14:31.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc0b4380 con 0x55e3a5192fc0 2024-03-30T13:14:31.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e3a5145000 con 0x55e3a5149c40 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 10 client.0 _ll_register_callbacks cb 0x7f50ec001ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a51a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a51a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32840/0 (socket says 172.21.15.43:32840) 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 -- 172.21.15.43:0/769273659 learned_addr learned my addr 172.21.15.43:0/769273659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a52050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50fbfff640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a5205610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 -- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 msgr2=0x55e3a52050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a52050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 -- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 msgr2=0x55e3a5205610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a5205610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 -- 172.21.15.43:0/769273659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3a4f1ad00 con 0x55e3a51938e0 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5100e34640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a52050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50fbfff640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a5205610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f5101635640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a51a62d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f50f4056f20 tx=0x7f50f4057690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5ea400b74b0abd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50db7fe640 1 -- 172.21.15.43:0/769273659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f405fb70 con 0x55e3a51938e0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50db7fe640 1 -- 172.21.15.43:0/769273659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f50f40563e0 con 0x55e3a51938e0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ec012c50 con 0x55e3a51938e0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50db7fe640 1 -- 172.21.15.43:0/769273659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f50f40565e0 con 0x55e3a51938e0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 10 client.6381 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55e3a5014f70 con 0x55e3a51938e0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.230+0000 7f50db7fe640 1 -- 172.21.15.43:0/769273659 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ec012c50 con 0x55e3a51938e0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f50db7fe640 1 -- 172.21.15.43:0/769273659 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f50f4052ce0 con 0x55e3a51938e0 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 10 client.6381 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 10 client.6381 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 20 client.6381 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a51a7070 0x55e3a51a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 4 client.6381 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628365}] 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628365}) -- 0x55e3a4fa0fe0 con 0x55e3a51a7070 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5100e34640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a51a7070 0x55e3a51a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5100e34640 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a51a7070 0x55e3a51a7480 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0259e348ad850bb server_cookie=8aa0ba6a51dc7a61 in_seq=0 out_seq=0 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f50db7fe640 10 client.6381 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f50db7fe640 1 -- 172.21.15.43:0/769273659 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3796 (crc 0 0 0) 0x55e3a4fa0fe0 con 0x55e3a51a7070 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f50db7fe640 10 client.6381 handle_command_reply: tid=0 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 client.6381 shutdown 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 msgr2=0x55e3a51a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a51a62d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f50f4056f20 tx=0x7f50f4057690 comp rx=0 tx=0).stop 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 shutdown_connections 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e3a51a7070 0x55e3a51a7480 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3a51938e0 0x55e3a51a62d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e3a5192fc0 0x55e3a5205610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 --2- 172.21.15.43:0/769273659 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e3a5149c40 0x55e3a52050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f50db7fe640 0 client.6381 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 >> 172.21.15.43:0/769273659 conn(0x55e3a51417e0 msgr2=0x55e3a5143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 shutdown_connections 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 1 -- 172.21.15.43:0/769273659 wait complete. 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f510621a000 20 client.6381 trim_cache size 0 max 16384 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/ceph_frag.c", 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628365, 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.537908+0000", 2024-03-30T13:14:31.244 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.493788+0000", 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:31.245 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 401, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.493788+0000", 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:31.246 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 401, 2024-03-30T13:14:31.247 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.248 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.248 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.248 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.537908+0000" 2024-03-30T13:14:31.248 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 401, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.537908+0000" 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 222, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 16, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:31.249 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.250 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:31.251 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.252 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.253 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.254 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:31.255 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:31.256 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:31.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50dc067ff0 msgr2=0x7f50dc06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50dc067ff0 0x7f50dc06a4b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f50fc13c810 tx=0x7f50ec007e90 comp rx=0 tx=0).stop 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ff40 msgr2=0x7f50fc154230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ff40 0x7f50fc154230 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f50f400efc0 tx=0x7f50f400c490 comp rx=0 tx=0).stop 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 shutdown_connections 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f50dc067ff0 0x7f50dc06a4b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50fc15ff40 0x7f50fc154230 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f50fc13f140 0x7f50fc152af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 --2- 172.21.15.43:0/2280794900 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f50fc06cbb0 0x7f50fc13dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 >> 172.21.15.43:0/2280794900 conn(0x7f50fc0d00a0 msgr2=0x7f50fc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 shutdown_connections 2024-03-30T13:14:31.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.234+0000 7f5103098640 1 -- 172.21.15.43:0/2280794900 wait complete. 2024-03-30T13:14:31.259 DEBUG:tasks.cephfs.test_uninlining:getting inode info #4 of 23:1099511628372 2024-03-30T13:14:31.259 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628372 2024-03-30T13:14:31.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628372 2024-03-30T13:14:31.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.374+0000 7f8dfd24f640 1 Processor -- start 2024-03-30T13:14:31.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.374+0000 7f8dfd24f640 1 -- start start 2024-03-30T13:14:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8dfd24f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8df806cd80 0x7f8df8166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8dfd24f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df806d710 con 0x7f8df813f140 2024-03-30T13:14:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8dfd24f640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df806d9f0 con 0x7f8df815ff40 2024-03-30T13:14:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8dfd24f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8df806e320 con 0x7f8df806cd80 2024-03-30T13:14:31.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8df813f140 0x7f8df81606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35752/0 (socket says 172.21.15.43:35752) 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df6d9d640 1 -- 172.21.15.43:0/1542673569 learned_addr learned my addr 172.21.15.43:0/1542673569 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df759e640 1 --2- 172.21.15.43:0/1542673569 >> v2:172.21.15.113:3300/0 conn(0x7f8df806cd80 0x7f8df8166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2867439285 0 0) 0x7f8df806d710 con 0x7f8df813f140 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8de4003530 con 0x7f8df813f140 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1385214573 0 0) 0x7f8de4003530 con 0x7f8df813f140 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 >> v2:172.21.15.113:3300/0 conn(0x7f8df806cd80 msgr2=0x7f8df8166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 --2- 172.21.15.43:0/1542673569 >> v2:172.21.15.113:3300/0 conn(0x7f8df806cd80 0x7f8df8166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 >> v1:172.21.15.113:6789/0 conn(0x7f8df815ff40 legacy=0x7f8df806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df806dcd0 con 0x7f8df813f140 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f8de8001680 con 0x7f8df813f140 2024-03-30T13:14:31.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 >> v1:172.21.15.43:6789/0 conn(0x7f8df813f140 legacy=0x7f8df81606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de4003a10 0x7f8de4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8de4004810 0x7f8de4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40051d0 0x7f8de400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8df806e320 con 0x7f8de40051d0 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8de400c0d0 con 0x7f8de4003a10 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8de400c2d0 con 0x7f8de4004810 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df759e640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40051d0 0x7f8de400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de4003a10 0x7f8de4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8dee59c640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8de4004810 0x7f8de4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df759e640 1 -- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8de4004810 msgr2=0x7f8de4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df759e640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8de4004810 0x7f8de4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df759e640 1 -- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de4003a10 msgr2=0x7f8de4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df759e640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de4003a10 0x7f8de4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df759e640 1 -- 172.21.15.43:0/1542673569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de4003530 con 0x7f8de40051d0 2024-03-30T13:14:31.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.378+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de4003a10 0x7f8de4003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 -- 172.21.15.43:0/1542673569 reap_dead start 2024-03-30T13:14:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40051d0 0x7f8de400b9d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009f90 tx=0x7f8df001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d936ef3be66b796 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0009c10 con 0x7f8de40051d0 2024-03-30T13:14:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df00a5c90 con 0x7f8de40051d0 2024-03-30T13:14:31.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df659c640 1 -- 172.21.15.43:0/1542673569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0003be0 con 0x7f8de40051d0 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- 172.21.15.43:0/1542673569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40051d0 msgr2=0x7f8de400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40051d0 0x7f8de400b9d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009f90 tx=0x7f8df001cc80 comp rx=0 tx=0).stop 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- 172.21.15.43:0/1542673569 shutdown_connections 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/1542673569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de40051d0 0x7f8de400b9d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- 172.21.15.43:0/1542673569 >> 172.21.15.43:0/1542673569 conn(0x7f8df80d00a0 msgr2=0x7f8df80d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- 172.21.15.43:0/1542673569 shutdown_connections 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- 172.21.15.43:0/1542673569 wait complete. 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 Processor -- start 2024-03-30T13:14:31.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- start start 2024-03-30T13:14:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de40051d0 0x7f8df81f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8df813da70 0x7f8df81f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df813def0 0x7f8df81f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8df813f6a0 con 0x7f8df813def0 2024-03-30T13:14:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8df813f520 con 0x7f8de40051d0 2024-03-30T13:14:31.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dfd24f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8df806c670 con 0x7f8df813da70 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df813def0 0x7f8df81f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df813def0 0x7f8df81f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32850/0 (socket says 172.21.15.43:32850) 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 -- 172.21.15.43:0/3414399361 learned_addr learned my addr 172.21.15.43:0/3414399361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de40051d0 0x7f8df81f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8dee59c640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8df813da70 0x7f8df81f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 -- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8df813da70 msgr2=0x7f8df81f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8df813da70 0x7f8df81f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 -- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de40051d0 msgr2=0x7f8df81f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de40051d0 0x7f8df81f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.382+0000 7f8df759e640 1 -- 172.21.15.43:0/3414399361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df81553f0 con 0x7f8df813def0 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8df759e640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df813def0 0x7f8df81f1ad0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8df001cab0 tx=0x7f8df0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8deffff640 1 -- 172.21.15.43:0/3414399361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0009c10 con 0x7f8df813def0 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8deffff640 1 -- 172.21.15.43:0/3414399361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df00a6e50 con 0x7f8df813def0 2024-03-30T13:14:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df81556d0 con 0x7f8df813def0 2024-03-30T13:14:31.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8deffff640 1 -- 172.21.15.43:0/3414399361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0003e40 con 0x7f8df813def0 2024-03-30T13:14:31.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df8155c90 con 0x7f8df813def0 2024-03-30T13:14:31.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8deffff640 1 -- 172.21.15.43:0/3414399361 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8df00c0020 con 0x7f8df813def0 2024-03-30T13:14:31.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8deffff640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8dd8067fc0 0x7f8dd806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8deffff640 1 -- 172.21.15.43:0/3414399361 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f8df0121560 con 0x7f8df813def0 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8dd8067fc0 0x7f8dd806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8e003d1000 1 Processor -- start 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8dd8067fc0 0x7f8dd806a480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000c10 tx=0x7f8de8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.386+0000 7f8e003d1000 1 -- start start 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe49c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe452830 0x556bfe452c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556bfe2a0aa0 con 0x556bfe452830 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556bfe227af0 con 0x556bfe49bd70 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x556bfe29ca50 con 0x556bfe49c690 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe452830 0x556bfe452c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe452830 0x556bfe452c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32858/0 (socket says 172.21.15.43:32858) 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 -- 172.21.15.43:0/3485729193 learned_addr learned my addr 172.21.15.43:0/3485729193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe49c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 -- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 msgr2=0x556bfe4522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 -- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 msgr2=0x556bfe49c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe49c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 -- 172.21.15.43:0/3485729193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfe225d00 con 0x556bfe452830 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe452830 0x556bfe452c10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8df00ed8b0 tx=0x7f8df0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73e6ecd4ac2ad97 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8decd99640 1 -- 172.21.15.43:0/3485729193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00f1450 con 0x556bfe452830 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8decd99640 1 -- 172.21.15.43:0/3485729193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df00f5800 con 0x556bfe452830 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8decd99640 1 -- 172.21.15.43:0/3485729193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00fa030 con 0x556bfe452830 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- 172.21.15.43:0/3485729193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe452830 msgr2=0x556bfe452c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe452830 0x556bfe452c10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8df00ed8b0 tx=0x7f8df0004a50 comp rx=0 tx=0).stop 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- 172.21.15.43:0/3485729193 shutdown_connections 2024-03-30T13:14:31.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe452830 0x556bfe452c10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3485729193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe49c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- 172.21.15.43:0/3485729193 >> 172.21.15.43:0/3485729193 conn(0x556bfe44e7e0 msgr2=0x556bfe450bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- 172.21.15.43:0/3485729193 shutdown_connections 2024-03-30T13:14:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- 172.21.15.43:0/3485729193 wait complete. 2024-03-30T13:14:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- start start 2024-03-30T13:14:31.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 Processor -- start 2024-03-30T13:14:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe457c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49bd70 0x556bfe480da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4812e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556bfe268270 con 0x556bfe49bd70 2024-03-30T13:14:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556bfe25e6a0 con 0x556bfe452830 2024-03-30T13:14:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x556bfe238880 con 0x556bfe49c690 2024-03-30T13:14:31.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8e003d1000 10 client.0 _ll_register_callbacks cb 0x556bfe43d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49bd70 0x556bfe480da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49bd70 0x556bfe480da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32874/0 (socket says 172.21.15.43:32874) 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 -- 172.21.15.43:0/3639087564 learned_addr learned my addr 172.21.15.43:0/3639087564 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4812e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe457c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 -- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 msgr2=0x556bfe4812e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4812e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 -- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 msgr2=0x556bfe457c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe457c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 -- 172.21.15.43:0/3639087564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfe265730 con 0x556bfe49bd70 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8df759e640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4812e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dee59c640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49bd70 0x556bfe480da0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8de0003980 tx=0x7f8de0003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830c3bd2242812ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.390+0000 7f8dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3639087564 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de0001b50 con 0x556bfe49bd70 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df00f5800 con 0x556bfe49bd70 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3639087564 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f8de0001cf0 con 0x556bfe49bd70 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3639087564 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8de0022050 con 0x556bfe49bd70 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 10 client.6393 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x556bfe258ca0 con 0x556bfe49bd70 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3639087564 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df00f5800 con 0x556bfe49bd70 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3639087564 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f8de0025020 con 0x556bfe49bd70 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 10 client.6393 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 10 client.6393 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 20 client.6393 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a35d0 0x556bfe4a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 4 client.6393 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bfe23bc90 con 0x556bfe4a35d0 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a35d0 0x556bfe4a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a35d0 0x556bfe4a39e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660f3e5b4d52483c server_cookie=fc85919a5821fb52 in_seq=0 out_seq=0 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 10 client.6393 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3639087564 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x556bfe23bc90 con 0x556bfe4a35d0 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 10 client.6393 handle_command_reply: tid=0 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 client.6393 shutdown 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49bd70 msgr2=0x556bfe480da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49bd70 0x556bfe480da0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8de0003980 tx=0x7f8de0003e40 comp rx=0 tx=0).stop 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 shutdown_connections 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a35d0 0x556bfe4a39e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49c690 0x556bfe4812e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49bd70 0x556bfe480da0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3639087564 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe457c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8dd5ffb640 0 client.6393 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 >> 172.21.15.43:0/3639087564 conn(0x556bfe44e7e0 msgr2=0x556bfe451460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 shutdown_connections 2024-03-30T13:14:31.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 1 -- 172.21.15.43:0/3639087564 wait complete. 2024-03-30T13:14:31.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.394+0000 7f8e003d1000 20 client.6393 trim_cache size 0 max 16384 2024-03-30T13:14:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 Processor -- start 2024-03-30T13:14:31.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 -- start start 2024-03-30T13:14:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe47fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49bd70 0x556bfe480290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe480ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81ff410 con 0x556bfe49c690 2024-03-30T13:14:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81fe410 con 0x556bfe452830 2024-03-30T13:14:31.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8df813c380 con 0x556bfe49bd70 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe480ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe480ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32886/0 (socket says 172.21.15.43:32886) 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 -- 172.21.15.43:0/2582292062 learned_addr learned my addr 172.21.15.43:0/2582292062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe47fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8dee59c640 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49bd70 0x556bfe480290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 -- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49bd70 msgr2=0x556bfe480290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49bd70 0x556bfe480290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 -- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 msgr2=0x556bfe47fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe47fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 -- 172.21.15.43:0/2582292062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfe227d00 con 0x556bfe49c690 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe47fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8dee59c640 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49bd70 0x556bfe480290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8df759e640 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe480ca0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8df00987b0 tx=0x7f8df001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b6dfd8316bf92d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8dd7fff640 1 -- 172.21.15.43:0/2582292062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00fa030 con 0x556bfe49c690 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8dd7fff640 1 -- 172.21.15.43:0/2582292062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df0109030 con 0x556bfe49c690 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.410+0000 7f8dd7fff640 1 -- 172.21.15.43:0/2582292062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00f1450 con 0x556bfe49c690 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/2582292062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 msgr2=0x556bfe480ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe480ca0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f8df00987b0 tx=0x7f8df001cc80 comp rx=0 tx=0).stop 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/2582292062 shutdown_connections 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe480ca0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe49bd70 0x556bfe480290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- 172.21.15.43:0/2582292062 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe452830 0x556bfe47fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/2582292062 >> 172.21.15.43:0/2582292062 conn(0x556bfe44e7e0 msgr2=0x556bfe4b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/2582292062 shutdown_connections 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/2582292062 wait complete. 2024-03-30T13:14:31.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- start start 2024-03-30T13:14:31.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 Processor -- start 2024-03-30T13:14:31.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe452830 0x556bfe50dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe50e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe49fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556bfe29fdb0 con 0x556bfe49c690 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81fddc0 con 0x556bfe49bd70 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x556bfe4b3fb0 con 0x556bfe452830 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 10 client.0 _ll_register_callbacks cb 0x7f8de0007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe49fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe49fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32890/0 (socket says 172.21.15.43:32890) 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 -- 172.21.15.43:0/3192804449 learned_addr learned my addr 172.21.15.43:0/3192804449 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe452830 0x556bfe50dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dee59c640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe50e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 -- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe452830 msgr2=0x556bfe50dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe452830 0x556bfe50dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 -- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 msgr2=0x556bfe50e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe50e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 -- 172.21.15.43:0/3192804449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfe321f70 con 0x556bfe49c690 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dee59c640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe50e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df759e640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe49fad0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009e60 tx=0x7f8df011f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c6c48338046f7a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3192804449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df00f1450 con 0x556bfe49c690 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3192804449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f8df00edac0 con 0x556bfe49c690 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df0109030 con 0x556bfe49c690 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3192804449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8df0116050 con 0x556bfe49c690 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 10 client.6399 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x556bfe247760 con 0x556bfe49c690 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3192804449 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df0109030 con 0x556bfe49c690 2024-03-30T13:14:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3192804449 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f8df011f450 con 0x556bfe49c690 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 10 client.6399 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 10 client.6399 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 20 client.6399 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a0430 0x556bfe4a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 4 client.6399 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628372}] 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628372}) -- 0x556bfe2adfe0 con 0x556bfe4a0430 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.414+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a0430 0x556bfe4a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dd5ffb640 10 client.6399 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8df6d9d640 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a0430 0x556bfe4a0840 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c711a18b927b29 server_cookie=a33e79ca28644707 in_seq=0 out_seq=0 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dd5ffb640 1 -- 172.21.15.43:0/3192804449 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3789 (crc 0 0 0) 0x556bfe2adfe0 con 0x556bfe4a0430 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dd5ffb640 10 client.6399 handle_command_reply: tid=0 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 client.6399 shutdown 2024-03-30T13:14:31.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 msgr2=0x556bfe49fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe49fad0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8df0009e60 tx=0x7f8df011f7a0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 shutdown_connections 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556bfe4a0430 0x556bfe4a0840 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556bfe49c690 0x556bfe49fad0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556bfe49bd70 0x556bfe50e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 --2- 172.21.15.43:0/3192804449 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556bfe452830 0x556bfe50dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dd5ffb640 0 client.6399 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 >> 172.21.15.43:0/3192804449 conn(0x556bfe44e7e0 msgr2=0x556bfe4b2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 shutdown_connections 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 1 -- 172.21.15.43:0/3192804449 wait complete. 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8e003d1000 20 client.6399 trim_cache size 0 max 16384 2024-03-30T13:14:31.427 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/io.h", 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628372, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.541504+0000", 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.542919+0000", 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:31.428 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 387, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:31.430 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:31.430 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.542919+0000", 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.431 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 387, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.541504+0000" 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.432 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 387, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.541504+0000" 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 224, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 30, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:31.433 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:31.434 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:31.435 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:31.436 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:31.437 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:31.438 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:31.438 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:31.438 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.438 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:31.439 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:31.440 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:31.441 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8dd8067fc0 msgr2=0x7f8dd806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8dd8067fc0 0x7f8dd806a480 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000c10 tx=0x7f8de8009040 comp rx=0 tx=0).stop 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df813def0 msgr2=0x7f8df81f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df813def0 0x7f8df81f1ad0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8df001cab0 tx=0x7f8df0009750 comp rx=0 tx=0).stop 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 shutdown_connections 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8dd8067fc0 0x7f8dd806a480 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8df813def0 0x7f8df81f1ad0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8df813da70 0x7f8df81f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 --2- 172.21.15.43:0/3414399361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8de40051d0 0x7f8df81f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 >> 172.21.15.43:0/3414399361 conn(0x7f8df80d00a0 msgr2=0x7f8df80af820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 shutdown_connections 2024-03-30T13:14:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.418+0000 7f8dfd24f640 1 -- 172.21.15.43:0/3414399361 wait complete. 2024-03-30T13:14:31.444 DEBUG:tasks.cephfs.test_uninlining:getting inode info #5 of 23:1099511628374 2024-03-30T13:14:31.444 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628374 2024-03-30T13:14:31.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628374 2024-03-30T13:14:31.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.558+0000 7f4938e4b640 1 Processor -- start 2024-03-30T13:14:31.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.558+0000 7f4938e4b640 1 -- start start 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4938e4b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49340cc8c0 0x7f49340d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4938e4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49340ce3c0 0x7f49340d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4938e4b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f493406eab0 con 0x7f49341531a0 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4938e4b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f49340d0de0 con 0x7f49340cc8c0 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4938e4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4934070700 con 0x7f49340ce3c0 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49340cc8c0 0x7f49340d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49340cc8c0 0x7f49340d1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32896/0 (socket says 172.21.15.43:32896) 2024-03-30T13:14:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 -- 172.21.15.43:0/2590612514 learned_addr learned my addr 172.21.15.43:0/2590612514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4932d9d640 1 --2- 172.21.15.43:0/2590612514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49340ce3c0 0x7f49340d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 -- 172.21.15.43:0/2590612514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49340ce3c0 msgr2=0x7f49340d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2590612514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49340ce3c0 0x7f49340d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 -- 172.21.15.43:0/2590612514 >> v1:172.21.15.113:6789/0 conn(0x7f49341531a0 legacy=0x7f493406fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 -- 172.21.15.43:0/2590612514 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493406ed90 con 0x7f49340cc8c0 2024-03-30T13:14:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.562+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2590612514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340cc8c0 0x7f49340d1180 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4924009870 tx=0x7f492401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165f7771d1b8358e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f493159a640 1 -- 172.21.15.43:0/2590612514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240aa600 con 0x7f49340cc8c0 2024-03-30T13:14:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f493159a640 1 -- 172.21.15.43:0/2590612514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49240af070 con 0x7f49340cc8c0 2024-03-30T13:14:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f493159a640 1 -- 172.21.15.43:0/2590612514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240aa9c0 con 0x7f49340cc8c0 2024-03-30T13:14:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- 172.21.15.43:0/2590612514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340cc8c0 msgr2=0x7f49340d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2590612514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340cc8c0 0x7f49340d1180 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4924009870 tx=0x7f492401c4a0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- 172.21.15.43:0/2590612514 shutdown_connections 2024-03-30T13:14:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2590612514 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49340ce3c0 0x7f49340d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2590612514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340cc8c0 0x7f49340d1180 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- 172.21.15.43:0/2590612514 >> 172.21.15.43:0/2590612514 conn(0x7f49340b3c80 msgr2=0x7f49340b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- 172.21.15.43:0/2590612514 shutdown_connections 2024-03-30T13:14:31.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- 172.21.15.43:0/2590612514 wait complete. 2024-03-30T13:14:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 Processor -- start 2024-03-30T13:14:31.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- start start 2024-03-30T13:14:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49340cc8c0 0x7f49340c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340ce3c0 0x7f49340b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49341531a0 0x7f493414a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4934153700 con 0x7f49340ce3c0 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4934153580 con 0x7f49340cc8c0 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4938e4b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f49340d05e0 con 0x7f49341531a0 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340ce3c0 0x7f49340b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340ce3c0 0x7f49340b5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32898/0 (socket says 172.21.15.43:32898) 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 -- 172.21.15.43:0/2793948687 learned_addr learned my addr 172.21.15.43:0/2793948687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f493259c640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49340cc8c0 0x7f49340c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4932d9d640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49341531a0 0x7f493414a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 -- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49341531a0 msgr2=0x7f493414a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49341531a0 0x7f493414a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 -- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49340cc8c0 msgr2=0x7f49340c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49340cc8c0 0x7f49340c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4931d9b640 1 -- 172.21.15.43:0/2793948687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49340c8b00 con 0x7f49340ce3c0 2024-03-30T13:14:31.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.566+0000 7f4932d9d640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49341531a0 0x7f493414a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340ce3c0 0x7f49340b5ff0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f49240ac040 tx=0x7f492401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4922ffd640 1 -- 172.21.15.43:0/2793948687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240bd5f0 con 0x7f49340ce3c0 2024-03-30T13:14:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4922ffd640 1 -- 172.21.15.43:0/2793948687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49240bd790 con 0x7f49340ce3c0 2024-03-30T13:14:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493414cf70 con 0x7f49340ce3c0 2024-03-30T13:14:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4922ffd640 1 -- 172.21.15.43:0/2793948687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240bdb80 con 0x7f49340ce3c0 2024-03-30T13:14:31.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493414d510 con 0x7f49340ce3c0 2024-03-30T13:14:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4922ffd640 1 -- 172.21.15.43:0/2793948687 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f49240a9070 con 0x7f49340ce3c0 2024-03-30T13:14:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4922ffd640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4914067fc0 0x7f491406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f4922ffd640 1 -- 172.21.15.43:0/2793948687 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f4924120f00 con 0x7f49340ce3c0 2024-03-30T13:14:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f493259c640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4914067fc0 0x7f491406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f493bfcd000 1 Processor -- start 2024-03-30T13:14:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f493259c640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4914067fc0 0x7f491406a480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f49280015b0 tx=0x7f4928009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.570+0000 7f493bfcd000 1 -- start start 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e959150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e90f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e90f830 0x55a51e90fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a51e75daa0 con 0x55a51e959690 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a51e6e4af0 con 0x55a51e90f830 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a51e759a50 con 0x55a51e958d70 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e90f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e90f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32906/0 (socket says 172.21.15.43:32906) 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/1150865141 learned_addr learned my addr 172.21.15.43:0/1150865141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493259c640 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e959150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4932d9d640 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e90f830 0x55a51e90fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 msgr2=0x55a51e959150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e959150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e90f830 msgr2=0x55a51e90fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e90f830 0x55a51e90fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/1150865141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51e6e2d00 con 0x55a51e959690 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4932d9d640 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e90f830 0x55a51e90fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e90f2f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f49240ed250 tx=0x7f49240b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8666c93973441f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4913fff640 1 -- 172.21.15.43:0/1150865141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240f51a0 con 0x55a51e959690 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4913fff640 1 -- 172.21.15.43:0/1150865141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49240af070 con 0x55a51e959690 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4913fff640 1 -- 172.21.15.43:0/1150865141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240f99d0 con 0x55a51e959690 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- 172.21.15.43:0/1150865141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 msgr2=0x55a51e90f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e90f2f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f49240ed250 tx=0x7f49240b8000 comp rx=0 tx=0).stop 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- 172.21.15.43:0/1150865141 shutdown_connections 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e90f830 0x55a51e90fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e90f2f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- 172.21.15.43:0/1150865141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e959150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- 172.21.15.43:0/1150865141 >> 172.21.15.43:0/1150865141 conn(0x55a51e90b7e0 msgr2=0x55a51e90dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- 172.21.15.43:0/1150865141 shutdown_connections 2024-03-30T13:14:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- 172.21.15.43:0/1150865141 wait complete. 2024-03-30T13:14:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- start start 2024-03-30T13:14:31.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 Processor -- start 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e914c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e958d70 0x55a51e93dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e93e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a51e725270 con 0x55a51e958d70 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a51e71b6a0 con 0x55a51e959690 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a51e6f5880 con 0x55a51e90f830 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493bfcd000 10 client.0 _ll_register_callbacks cb 0x55a51e8fa490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e958d70 0x55a51e93dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e958d70 0x55a51e93dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32918/0 (socket says 172.21.15.43:32918) 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/2484910669 learned_addr learned my addr 172.21.15.43:0/2484910669 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f493259c640 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e914c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4932d9d640 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e93e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 msgr2=0x55a51e914c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e914c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 msgr2=0x55a51e93e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e93e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.574+0000 7f4931d9b640 1 -- 172.21.15.43:0/2484910669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51e722730 con 0x55a51e958d70 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f4931d9b640 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e958d70 0x55a51e93dda0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f492401ca10 tx=0x7f492411f140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97743619bf49586a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 1 -- 172.21.15.43:0/2484910669 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49240f51a0 con 0x55a51e958d70 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 1 -- 172.21.15.43:0/2484910669 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f4924118bd0 con 0x55a51e958d70 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49240af070 con 0x55a51e958d70 2024-03-30T13:14:31.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 1 -- 172.21.15.43:0/2484910669 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4924118e10 con 0x55a51e958d70 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 10 client.6411 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55a51e715ca0 con 0x55a51e958d70 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 1 -- 172.21.15.43:0/2484910669 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49240af070 con 0x55a51e958d70 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 1 -- 172.21.15.43:0/2484910669 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f49240c6020 con 0x55a51e958d70 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 10 client.6411 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 10 client.6411 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 20 client.6411 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e9605d0 0x55a51e9609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 4 client.6411 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a51e6f8c90 con 0x55a51e9605d0 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493259c640 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e9605d0 0x55a51e9609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493259c640 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e9605d0 0x55a51e9609e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9317d54bb70bee65 server_cookie=a766c2b6f95aa6cd in_seq=0 out_seq=0 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 10 client.6411 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 1 -- 172.21.15.43:0/2484910669 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55a51e6f8c90 con 0x55a51e9605d0 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 10 client.6411 handle_command_reply: tid=0 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 client.6411 shutdown 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e958d70 msgr2=0x55a51e93dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e958d70 0x55a51e93dda0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f492401ca10 tx=0x7f492411f140 comp rx=0 tx=0).stop 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 shutdown_connections 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e9605d0 0x55a51e9609e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e93e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e958d70 0x55a51e93dda0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f49117fa640 0 client.6411 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 --2- 172.21.15.43:0/2484910669 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e914c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 >> 172.21.15.43:0/2484910669 conn(0x55a51e90b7e0 msgr2=0x55a51e90e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 shutdown_connections 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 1 -- 172.21.15.43:0/2484910669 wait complete. 2024-03-30T13:14:31.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.578+0000 7f493bfcd000 20 client.6411 trim_cache size 0 max 16384 2024-03-30T13:14:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 Processor -- start 2024-03-30T13:14:31.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 -- start start 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e90f830 0x55a51e93e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e90dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e90e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4934201b30 con 0x55a51e90f830 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4934200bd0 con 0x55a51e959690 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f493414bc70 con 0x55a51e958d70 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e90f830 0x55a51e93e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e90f830 0x55a51e93e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32926/0 (socket says 172.21.15.43:32926) 2024-03-30T13:14:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 -- 172.21.15.43:0/3025384851 learned_addr learned my addr 172.21.15.43:0/3025384851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f4931d9b640 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e90dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 -- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 msgr2=0x55a51e90dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f4932d9d640 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e90e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e90dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 -- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 msgr2=0x55a51e90e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e90e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 -- 172.21.15.43:0/3025384851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51e6e4d00 con 0x55a51e90f830 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f4931d9b640 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e90dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f493259c640 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e90f830 0x55a51e93e180 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4928005580 tx=0x7f4928012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11e27cd21182b20 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f49137fe640 1 -- 172.21.15.43:0/3025384851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928027040 con 0x55a51e90f830 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f49137fe640 1 -- 172.21.15.43:0/3025384851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492802c020 con 0x55a51e90f830 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.594+0000 7f49137fe640 1 -- 172.21.15.43:0/3025384851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928027200 con 0x55a51e90f830 2024-03-30T13:14:31.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3025384851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e90f830 msgr2=0x55a51e93e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e90f830 0x55a51e93e180 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f4928005580 tx=0x7f4928012000 comp rx=0 tx=0).stop 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3025384851 shutdown_connections 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e959690 0x55a51e90e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e958d70 0x55a51e90dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3025384851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e90f830 0x55a51e93e180 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3025384851 >> 172.21.15.43:0/3025384851 conn(0x55a51e90b7e0 msgr2=0x55a51e96ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3025384851 shutdown_connections 2024-03-30T13:14:31.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3025384851 wait complete. 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- start start 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 Processor -- start 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e9ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e958d70 0x55a51e9cac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e95cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a51e6f5880 con 0x55a51e959690 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f493414c2c0 con 0x55a51e958d70 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a51e759a50 con 0x55a51e90f830 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 10 client.0 _ll_register_callbacks cb 0x7f49240f09c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e95cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e95cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32936/0 (socket says 172.21.15.43:32936) 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 -- 172.21.15.43:0/3275670804 learned_addr learned my addr 172.21.15.43:0/3275670804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493259c640 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e9ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 -- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 msgr2=0x55a51e9ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e9ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4931d9b640 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e958d70 0x55a51e9cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 -- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e958d70 msgr2=0x55a51e9cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e958d70 0x55a51e9cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 -- 172.21.15.43:0/3275670804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51e7def70 con 0x55a51e959690 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f4932d9d640 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e95cad0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f492c007ae0 tx=0x7f492c013250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d46b960e779099 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f49117fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f49117fa640 1 -- 172.21.15.43:0/3275670804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c01d020 con 0x55a51e959690 2024-03-30T13:14:31.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f49117fa640 1 -- 172.21.15.43:0/3275670804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f492c013c30 con 0x55a51e959690 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f492802c020 con 0x55a51e959690 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f49117fa640 1 -- 172.21.15.43:0/3275670804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f492c027050 con 0x55a51e959690 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 10 client.6417 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55a51e704760 con 0x55a51e959690 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f49117fa640 1 -- 172.21.15.43:0/3275670804 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492802c020 con 0x55a51e959690 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f49117fa640 1 -- 172.21.15.43:0/3275670804 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f492c02a020 con 0x55a51e959690 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 10 client.6417 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 10 client.6417 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 20 client.6417 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e95d8b0 0x55a51e95dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 4 client.6417 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628374}] 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628374}) -- 0x55a51e76afe0 con 0x55a51e95d8b0 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.598+0000 7f493259c640 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e95d8b0 0x55a51e95dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493259c640 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e95d8b0 0x55a51e95dcc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18bf2500f733e5d server_cookie=c31afd82358a2632 in_seq=0 out_seq=0 2024-03-30T13:14:31.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f49117fa640 10 client.6417 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f49117fa640 1 -- 172.21.15.43:0/3275670804 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3795 (crc 0 0 0) 0x55a51e76afe0 con 0x55a51e95d8b0 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f49117fa640 10 client.6417 handle_command_reply: tid=0 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 client.6417 shutdown 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 msgr2=0x55a51e95cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e95cad0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f492c007ae0 tx=0x7f492c013250 comp rx=0 tx=0).stop 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 shutdown_connections 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a51e95d8b0 0x55a51e95dcc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a51e959690 0x55a51e95cad0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a51e958d70 0x55a51e9cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f49117fa640 0 client.6417 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 --2- 172.21.15.43:0/3275670804 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a51e90f830 0x55a51e9ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 >> 172.21.15.43:0/3275670804 conn(0x55a51e90b7e0 msgr2=0x55a51e970620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 shutdown_connections 2024-03-30T13:14:31.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 1 -- 172.21.15.43:0/3275670804 wait complete. 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f493bfcd000 20 client.6417 trim_cache size 0 max 16384 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ceph/ioctl.h", 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628374, 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.544424+0000", 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.551070+0000", 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:31.611 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.612 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3724, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.551070+0000", 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:31.613 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3724, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.614 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.544424+0000" 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3724, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.544424+0000" 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 226, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 34, 2024-03-30T13:14:31.615 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:31.616 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.617 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:31.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:31.619 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:31.620 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:31.621 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:31.622 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:31.623 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:31.623 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4914067fc0 msgr2=0x7f491406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4914067fc0 0x7f491406a480 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f49280015b0 tx=0x7f4928009040 comp rx=0 tx=0).stop 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340ce3c0 msgr2=0x7f49340b5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340ce3c0 0x7f49340b5ff0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f49240ac040 tx=0x7f492401ace0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 shutdown_connections 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4914067fc0 0x7f491406a480 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f49341531a0 0x7f493414a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49340ce3c0 0x7f49340b5ff0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 --2- 172.21.15.43:0/2793948687 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f49340cc8c0 0x7f49340c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 >> 172.21.15.43:0/2793948687 conn(0x7f49340b3c80 msgr2=0x7f493415c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 shutdown_connections 2024-03-30T13:14:31.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.602+0000 7f4938e4b640 1 -- 172.21.15.43:0/2793948687 wait complete. 2024-03-30T13:14:31.625 DEBUG:tasks.cephfs.test_uninlining:getting inode info #6 of 23:1099511628386 2024-03-30T13:14:31.625 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628386 2024-03-30T13:14:31.625 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628386 2024-03-30T13:14:31.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb9bd7640 1 Processor -- start 2024-03-30T13:14:31.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb9bd7640 1 -- start start 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb9bd7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5cb40cf6c0 0x7f5cb4169030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb9bd7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb406c850 con 0x7f5cb40b9e30 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb9bd7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb406cb30 con 0x7f5cb415e4c0 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb9bd7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb40ba290 con 0x7f5cb40cf6c0 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5cb40b9e30 0x7f5cb40cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35758/0 (socket says 172.21.15.43:35758) 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3535734160 learned_addr learned my addr 172.21.15.43:0/3535734160 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/3535734160 >> v2:172.21.15.113:3300/0 conn(0x7f5cb40cf6c0 0x7f5cb4169030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3954980368 0 0) 0x7f5cb406c850 con 0x7f5cb40b9e30 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ca0003530 con 0x7f5cb40b9e30 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3191873765 0 0) 0x7f5ca0003530 con 0x7f5cb40b9e30 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 >> v2:172.21.15.113:3300/0 conn(0x7f5cb40cf6c0 msgr2=0x7f5cb4169030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 --2- 172.21.15.43:0/3535734160 >> v2:172.21.15.113:3300/0 conn(0x7f5cb40cf6c0 0x7f5cb4169030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 >> v1:172.21.15.113:6789/0 conn(0x7f5cb415e4c0 legacy=0x7f5cb406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb406ce10 con 0x7f5cb40b9e30 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f5c9c001680 con 0x7f5cb40b9e30 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 >> v1:172.21.15.43:6789/0 conn(0x7f5cb40b9e30 legacy=0x7f5cb40cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca00039c0 0x7f5ca0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ca0004810 0x7f5ca0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ca00051d0 0x7f5ca000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca000c040 con 0x7f5ca0004810 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca000c1f0 con 0x7f5ca00051d0 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5ca000c3f0 con 0x7f5ca00039c0 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ca0004810 0x7f5ca0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ca00051d0 0x7f5ca000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca00039c0 msgr2=0x7f5ca0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca00039c0 0x7f5ca0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca00039c0 0x7f5ca0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ca00051d0 msgr2=0x7f5ca000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.742+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ca00051d0 0x7f5ca000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3535734160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca0003530 con 0x7f5ca0004810 2024-03-30T13:14:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5ca00051d0 0x7f5ca000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb3fff640 1 -- 172.21.15.43:0/3535734160 reap_dead start 2024-03-30T13:14:31.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca00039c0 0x7f5ca0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ca0004810 0x7f5ca0004c90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c240 tx=0x7f5c9c01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c396baa6ab1880be server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c007d20 con 0x7f5ca0004810 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c005ae0 con 0x7f5ca0004810 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb27fc640 1 -- 172.21.15.43:0/3535734160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c007070 con 0x7f5ca0004810 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/3535734160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ca0004810 msgr2=0x7f5ca0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ca0004810 0x7f5ca0004c90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00c240 tx=0x7f5c9c01f3f0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/3535734160 shutdown_connections 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/3535734160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ca0004810 0x7f5ca0004c90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/3535734160 >> 172.21.15.43:0/3535734160 conn(0x7f5cb40b6410 msgr2=0x7f5cb40d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/3535734160 shutdown_connections 2024-03-30T13:14:31.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.746+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/3535734160 wait complete. 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 Processor -- start 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 -- start start 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca0004810 0x7f5cb415b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cb415bf80 0x7f5cb415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb40cc0a0 0x7f5cb40cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb40cf340 con 0x7f5cb415bf80 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb40cf1c0 con 0x7f5cb40cc0a0 2024-03-30T13:14:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb40cf4c0 con 0x7f5ca0004810 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cb415bf80 0x7f5cb415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cb415bf80 0x7f5cb415c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32952/0 (socket says 172.21.15.43:32952) 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/1399245630 learned_addr learned my addr 172.21.15.43:0/1399245630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb40cc0a0 0x7f5cb40cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca0004810 0x7f5cb415b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca0004810 msgr2=0x7f5cb415b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca0004810 0x7f5cb415b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb40cc0a0 msgr2=0x7f5cb40cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb40cc0a0 0x7f5cb40cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/1399245630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb40cca60 con 0x7f5cb415bf80 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca0004810 0x7f5cb415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cb415bf80 0x7f5cb415c400 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c01f220 tx=0x7f5ca0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cabfff640 1 -- 172.21.15.43:0/1399245630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c007d20 con 0x7f5cb415bf80 2024-03-30T13:14:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cabfff640 1 -- 172.21.15.43:0/1399245630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c9c0b6070 con 0x7f5cb415bf80 2024-03-30T13:14:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb41f4b90 con 0x7f5cb415bf80 2024-03-30T13:14:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cabfff640 1 -- 172.21.15.43:0/1399245630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c001680 con 0x7f5cb415bf80 2024-03-30T13:14:31.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb41f40b0 con 0x7f5cb415bf80 2024-03-30T13:14:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.750+0000 7f5cabfff640 1 -- 172.21.15.43:0/1399245630 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5c9c0b8020 con 0x7f5cb415bf80 2024-03-30T13:14:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cabfff640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c94067fc0 0x7f5c9406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cabfff640 1 -- 172.21.15.43:0/1399245630 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f5c9c1238b0 con 0x7f5cb415bf80 2024-03-30T13:14:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c94067fc0 0x7f5c9406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 Processor -- start 2024-03-30T13:14:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- start start 2024-03-30T13:14:31.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c94067fc0 0x7f5c9406a480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5cac000f90 tx=0x7f5cac009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b785150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b73bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f66b589aa0 con 0x55f66b784d70 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f66b510af0 con 0x55f66b73b830 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55f66b585a50 con 0x55f66b785690 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b785150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b785150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32956/0 (socket says 172.21.15.43:32956) 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2953191354 learned_addr learned my addr 172.21.15.43:0/2953191354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b73bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 msgr2=0x55f66b73b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 msgr2=0x55f66b73bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b73bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2953191354 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66b50ed00 con 0x55f66b784d70 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b73bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b785150 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5cac003db0 tx=0x7f5cac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e7ad5cceb11724 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5ca97fa640 1 -- 172.21.15.43:0/2953191354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac026070 con 0x55f66b784d70 2024-03-30T13:14:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5ca97fa640 1 -- 172.21.15.43:0/2953191354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac005650 con 0x55f66b784d70 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5ca97fa640 1 -- 172.21.15.43:0/2953191354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0135f0 con 0x55f66b784d70 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2953191354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 msgr2=0x55f66b785150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b785150 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f5cac003db0 tx=0x7f5cac012000 comp rx=0 tx=0).stop 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2953191354 shutdown_connections 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b73bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2953191354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b785150 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2953191354 >> 172.21.15.43:0/2953191354 conn(0x55f66b7377e0 msgr2=0x55f66b739bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2953191354 shutdown_connections 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.754+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2953191354 wait complete. 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 -- start start 2024-03-30T13:14:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 Processor -- start 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b740c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b769da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b76a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f66b551270 con 0x55f66b784d70 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55f66b5476a0 con 0x55f66b73b830 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55f66b521880 con 0x55f66b785690 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 10 client.0 _ll_register_callbacks cb 0x55f66b726490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b769da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b769da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32960/0 (socket says 172.21.15.43:32960) 2024-03-30T13:14:31.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3264894858 learned_addr learned my addr 172.21.15.43:0/3264894858 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b76a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b740c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 msgr2=0x55f66b76a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b76a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 msgr2=0x55f66b740c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b740c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3264894858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66b54e730 con 0x55f66b784d70 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b769da0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0f3730 tx=0x7f5c9c0f3100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562af3b8e3260e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5c927fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5c927fc640 1 -- 172.21.15.43:0/3264894858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0f37a0 con 0x55f66b784d70 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5c927fc640 1 -- 172.21.15.43:0/3264894858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f5c9c11cdf0 con 0x55f66b784d70 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cac005650 con 0x55f66b784d70 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5c927fc640 1 -- 172.21.15.43:0/3264894858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c0f7b50 con 0x55f66b784d70 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 10 client.6429 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f66b541ca0 con 0x55f66b784d70 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5c927fc640 1 -- 172.21.15.43:0/3264894858 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac005650 con 0x55f66b784d70 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5c927fc640 1 -- 172.21.15.43:0/3264894858 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f5c9c0f8ce0 con 0x55f66b784d70 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 10 client.6429 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 10 client.6429 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 20 client.6429 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b78c5d0 0x55f66b78c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 4 client.6429 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f66b524c90 con 0x55f66b78c5d0 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b78c5d0 0x55f66b78c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b78c5d0 0x55f66b78c9e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=179c4bc8cdf61cf9 server_cookie=3324d1a19feaf4f1 in_seq=0 out_seq=0 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.758+0000 7f5c927fc640 10 client.6429 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5c927fc640 1 -- 172.21.15.43:0/3264894858 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55f66b524c90 con 0x55f66b78c5d0 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5c927fc640 10 client.6429 handle_command_reply: tid=0 2024-03-30T13:14:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 client.6429 shutdown 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 msgr2=0x55f66b769da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b769da0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0f3730 tx=0x7f5c9c0f3100 comp rx=0 tx=0).stop 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 shutdown_connections 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b78c5d0 0x55f66b78c9e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b76a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b769da0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3264894858 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b740c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 >> 172.21.15.43:0/3264894858 conn(0x55f66b7377e0 msgr2=0x55f66b73a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5c927fc640 0 client.6429 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 shutdown_connections 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3264894858 wait complete. 2024-03-30T13:14:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.762+0000 7f5cbcd59000 20 client.6429 trim_cache size 0 max 16384 2024-03-30T13:14:31.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.774+0000 7f5cbcd59000 1 Processor -- start 2024-03-30T13:14:31.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.774+0000 7f5cbcd59000 1 -- start start 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b73b830 0x55f66b76a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b784d70 0x55f66b739bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb4201560 con 0x55f66b73b830 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb42005a0 con 0x55f66b784d70 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb4160d70 con 0x55f66b785690 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b73b830 0x55f66b76a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b73b830 0x55f66b76a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32976/0 (socket says 172.21.15.43:32976) 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2291697229 learned_addr learned my addr 172.21.15.43:0/2291697229 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b784d70 0x55f66b739bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2291697229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 msgr2=0x55f66b73a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2291697229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b784d70 msgr2=0x55f66b739bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b784d70 0x55f66b739bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 -- 172.21.15.43:0/2291697229 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f66b510d00 con 0x55f66b73b830 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b73b830 0x55f66b76a180 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5cac024040 tx=0x7f5cac005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208940ada3c61f53 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5ca8ff9640 1 -- 172.21.15.43:0/2291697229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac026050 con 0x55f66b73b830 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5ca8ff9640 1 -- 172.21.15.43:0/2291697229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac027040 con 0x55f66b73b830 2024-03-30T13:14:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5ca8ff9640 1 -- 172.21.15.43:0/2291697229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cac032e60 con 0x55f66b73b830 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2291697229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b73b830 msgr2=0x55f66b76a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b73b830 0x55f66b76a180 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5cac024040 tx=0x7f5cac005000 comp rx=0 tx=0).stop 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2291697229 shutdown_connections 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b73a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b784d70 0x55f66b739bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/2291697229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b73b830 0x55f66b76a180 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2291697229 >> 172.21.15.43:0/2291697229 conn(0x55f66b7377e0 msgr2=0x55f66b79bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2291697229 shutdown_connections 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- 172.21.15.43:0/2291697229 wait complete. 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- start start 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 Processor -- start 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b7f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b7f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b788ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55f66b588db0 con 0x55f66b784d70 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb41fff80 con 0x55f66b73b830 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55f66b645b00 con 0x55f66b785690 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cbcd59000 10 client.0 _ll_register_callbacks cb 0x7f5c9c007ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b7f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b7f75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32986/0 (socket says 172.21.15.43:32986) 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3871424670 learned_addr learned my addr 172.21.15.43:0/3871424670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b7f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b788ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 msgr2=0x55f66b788ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b788ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 msgr2=0x55f66b7f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b7f70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.778+0000 7f5cb2ffd640 1 -- 172.21.15.43:0/3871424670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f66b60af70 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb3fff640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b788ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb2ffd640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b7f75e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0018f0 tx=0x7f5c9c0f7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae7df088e288144 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 1 -- 172.21.15.43:0/3871424670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0fa0b0 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 1 -- 172.21.15.43:0/3871424670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f5c9c0ef4f0 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cac027040 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 1 -- 172.21.15.43:0/3871424670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c111050 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 10 client.6435 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55f66b530760 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 1 -- 172.21.15.43:0/3871424670 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cac027040 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 1 -- 172.21.15.43:0/3871424670 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f5c9c0f8ce0 con 0x55f66b784d70 2024-03-30T13:14:31.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 10 client.6435 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 10 client.6435 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 20 client.6435 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b789490 0x55f66b7898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 4 client.6435 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628386}] 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628386}) -- 0x55f66b596fe0 con 0x55f66b789490 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b789490 0x55f66b7898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 10 client.6435 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb37fe640 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b789490 0x55f66b7898a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6faba3a69c5d3c6b server_cookie=c8dd61a658f98a82 in_seq=0 out_seq=0 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 1 -- 172.21.15.43:0/3871424670 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3794 (crc 0 0 0) 0x55f66b596fe0 con 0x55f66b789490 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 10 client.6435 handle_command_reply: tid=0 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 client.6435 shutdown 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 msgr2=0x55f66b7f75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b7f75e0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0018f0 tx=0x7f5c9c0f7040 comp rx=0 tx=0).stop 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 shutdown_connections 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55f66b789490 0x55f66b7898a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55f66b785690 0x55f66b788ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f66b784d70 0x55f66b7f75e0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 --2- 172.21.15.43:0/3871424670 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55f66b73b830 0x55f66b7f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5c927fc640 0 client.6435 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 >> 172.21.15.43:0/3871424670 conn(0x55f66b7377e0 msgr2=0x55f66b79c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 shutdown_connections 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 1 -- 172.21.15.43:0/3871424670 wait complete. 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cbcd59000 20 client.6435 trim_cache size 0 max 16384 2024-03-30T13:14:31.792 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/Kconfig", 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628386, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.552718+0000", 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.644788+0000", 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:31.793 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1240, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:31.794 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.644788+0000", 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.795 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1240, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.552718+0000" 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.796 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1240, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.552718+0000" 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 142, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 2, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:31.797 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:31.798 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:31.799 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:31.801 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.802 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:31.803 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:31.804 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c94067fc0 msgr2=0x7f5c9406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c94067fc0 0x7f5c9406a480 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5cac000f90 tx=0x7f5cac009040 comp rx=0 tx=0).stop 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cb415bf80 msgr2=0x7f5cb415c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cb415bf80 0x7f5cb415c400 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c01f220 tx=0x7f5ca0005840 comp rx=0 tx=0).stop 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 shutdown_connections 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5c94067fc0 0x7f5c9406a480 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5cb40cc0a0 0x7f5cb40cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5cb415bf80 0x7f5cb415c400 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 --2- 172.21.15.43:0/1399245630 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5ca0004810 0x7f5cb415b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 >> 172.21.15.43:0/1399245630 conn(0x7f5cb40b6410 msgr2=0x7f5cb40b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 shutdown_connections 2024-03-30T13:14:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.782+0000 7f5cb9bd7640 1 -- 172.21.15.43:0/1399245630 wait complete. 2024-03-30T13:14:31.807 DEBUG:tasks.cephfs.test_uninlining:getting inode info #7 of 23:1099511628404 2024-03-30T13:14:31.807 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628404 2024-03-30T13:14:31.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628404 2024-03-30T13:14:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b535ff640 1 Processor -- start 2024-03-30T13:14:31.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b535ff640 1 -- start start 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b535ff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c13f140 0x7f4b4c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b535ff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4b4c06cd80 0x7f4b4c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b535ff640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b4c06d710 con 0x7f4b4c15ff40 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b535ff640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4b4c06e320 con 0x7f4b4c06cd80 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b535ff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c06d9f0 con 0x7f4b4c13f140 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b51b9c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4b4c06cd80 0x7f4b4c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b51b9c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4b4c06cd80 0x7f4b4c0ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32988/0 (socket says 172.21.15.43:32988) 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.922+0000 7f4b51b9c640 1 -- 172.21.15.43:0/3051631404 learned_addr learned my addr 172.21.15.43:0/3051631404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b5139b640 1 --2- 172.21.15.43:0/3051631404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c13f140 0x7f4b4c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b51b9c640 1 -- 172.21.15.43:0/3051631404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c13f140 msgr2=0x7f4b4c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/3051631404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c13f140 0x7f4b4c1606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b51b9c640 1 -- 172.21.15.43:0/3051631404 >> v1:172.21.15.113:6789/0 conn(0x7f4b4c15ff40 legacy=0x7f4b4c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b51b9c640 1 -- 172.21.15.43:0/3051631404 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c06dcd0 con 0x7f4b4c06cd80 2024-03-30T13:14:31.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/3051631404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c0ce940 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e50 tx=0x7f4b4401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca7245faff0c484 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b43fff640 1 -- 172.21.15.43:0/3051631404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440a58f0 con 0x7f4b4c06cd80 2024-03-30T13:14:31.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b43fff640 1 -- 172.21.15.43:0/3051631404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b440a5a90 con 0x7f4b4c06cd80 2024-03-30T13:14:31.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b43fff640 1 -- 172.21.15.43:0/3051631404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440a5e60 con 0x7f4b4c06cd80 2024-03-30T13:14:31.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b535ff640 1 -- 172.21.15.43:0/3051631404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 msgr2=0x7f4b4c0ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b535ff640 1 --2- 172.21.15.43:0/3051631404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c0ce940 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009e50 tx=0x7f4b4401ca00 comp rx=0 tx=0).stop 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b535ff640 1 -- 172.21.15.43:0/3051631404 shutdown_connections 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b535ff640 1 --2- 172.21.15.43:0/3051631404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c0ce940 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b535ff640 1 --2- 172.21.15.43:0/3051631404 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c13f140 0x7f4b4c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.926+0000 7f4b535ff640 1 -- 172.21.15.43:0/3051631404 >> 172.21.15.43:0/3051631404 conn(0x7f4b4c0d00a0 msgr2=0x7f4b4c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- 172.21.15.43:0/3051631404 shutdown_connections 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- 172.21.15.43:0/3051631404 wait complete. 2024-03-30T13:14:31.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 Processor -- start 2024-03-30T13:14:31.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- start start 2024-03-30T13:14:31.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b4c13f140 0x7f4b4c13d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c15ff40 0x7f4b4c15b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0cf810 con 0x7f4b4c06cd80 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c0cf690 con 0x7f4b4c13f140 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c0cf990 con 0x7f4b4c15ff40 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:32996/0 (socket says 172.21.15.43:32996) 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 -- 172.21.15.43:0/418886086 learned_addr learned my addr 172.21.15.43:0/418886086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b4c13f140 0x7f4b4c13d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c15ff40 0x7f4b4c15b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 -- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c15ff40 msgr2=0x7f4b4c15b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c15ff40 0x7f4b4c15b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 -- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b4c13f140 msgr2=0x7f4b4c13d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b4c13f140 0x7f4b4c13d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 -- 172.21.15.43:0/418886086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c13dbe0 con 0x7f4b4c06cd80 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b4c13f140 0x7f4b4c13d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b5139b640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c155a60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d830 tx=0x7f4b4800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b41ffb640 1 -- 172.21.15.43:0/418886086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800b6c0 con 0x7f4b4c06cd80 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b41ffb640 1 -- 172.21.15.43:0/418886086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b48019070 con 0x7f4b4c06cd80 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c13dec0 con 0x7f4b4c06cd80 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b41ffb640 1 -- 172.21.15.43:0/418886086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800ba60 con 0x7f4b4c06cd80 2024-03-30T13:14:31.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.930+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c153be0 con 0x7f4b4c06cd80 2024-03-30T13:14:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b41ffb640 1 -- 172.21.15.43:0/418886086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4b4800bc00 con 0x7f4b4c06cd80 2024-03-30T13:14:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b41ffb640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4b30067ff0 0x7f4b3006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b41ffb640 1 -- 172.21.15.43:0/418886086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f4b48086b60 con 0x7f4b4c06cd80 2024-03-30T13:14:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4b30067ff0 0x7f4b3006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 Processor -- start 2024-03-30T13:14:31.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 -- start start 2024-03-30T13:14:31.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4b30067ff0 0x7f4b3006a4b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c15bf50 tx=0x7f4b3c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc05150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fc05690 0x56155fbbb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fbbb830 0x56155fbbbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56155fa09aa0 con 0x56155fbbb830 2024-03-30T13:14:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56155f990af0 con 0x56155fc05690 2024-03-30T13:14:31.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56155fa05a50 con 0x56155fc04d70 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fbbb830 0x56155fbbbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fbbb830 0x56155fbbbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33012/0 (socket says 172.21.15.43:33012) 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 -- 172.21.15.43:0/56579429 learned_addr learned my addr 172.21.15.43:0/56579429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fc05690 0x56155fbbb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b5139b640 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc05150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 -- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 msgr2=0x56155fc05150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc05150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 -- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fc05690 msgr2=0x56155fbbb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fc05690 0x56155fbbb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 -- 172.21.15.43:0/56579429 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155f98ed00 con 0x56155fbbb830 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fbbb830 0x56155fbbbc10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4b440af040 tx=0x7f4b44004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7203251510e43443 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b36ffd640 1 -- 172.21.15.43:0/56579429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440a58f0 con 0x56155fbbb830 2024-03-30T13:14:31.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b36ffd640 1 -- 172.21.15.43:0/56579429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b440a5a90 con 0x56155fbbb830 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.934+0000 7f4b36ffd640 1 -- 172.21.15.43:0/56579429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440b9690 con 0x56155fbbb830 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/56579429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fbbb830 msgr2=0x56155fbbbc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fbbb830 0x56155fbbbc10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4b440af040 tx=0x7f4b44004ac0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/56579429 shutdown_connections 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fbbb830 0x56155fbbbc10 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fc05690 0x56155fbbb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- 172.21.15.43:0/56579429 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc05150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/56579429 >> 172.21.15.43:0/56579429 conn(0x56155fbb77e0 msgr2=0x56155fbb9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/56579429 shutdown_connections 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/56579429 wait complete. 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- start start 2024-03-30T13:14:31.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 Processor -- start 2024-03-30T13:14:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbc0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc04d70 0x56155fbe9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc05690 0x56155fbea2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56155f9d1270 con 0x56155fc04d70 2024-03-30T13:14:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x56155f9c76a0 con 0x56155fbbb830 2024-03-30T13:14:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56155f9a1880 con 0x56155fc05690 2024-03-30T13:14:31.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 10 client.0 _ll_register_callbacks cb 0x56155fba6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc04d70 0x56155fbe9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc04d70 0x56155fbe9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33028/0 (socket says 172.21.15.43:33028) 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 -- 172.21.15.43:0/1838861953 learned_addr learned my addr 172.21.15.43:0/1838861953 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b5139b640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbc0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc05690 0x56155fbea2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 -- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc05690 msgr2=0x56155fbea2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc05690 0x56155fbea2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 -- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 msgr2=0x56155fbc0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbc0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 -- 172.21.15.43:0/1838861953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56155f9ce730 con 0x56155fc04d70 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b5139b640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbc0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc04d70 0x56155fbe9da0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00dce0 tx=0x7f4b3c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ad7f984473fa7f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b2bfff640 1 -- 172.21.15.43:0/1838861953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c020420 con 0x56155fc04d70 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b2bfff640 1 -- 172.21.15.43:0/1838861953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f4b3c0205c0 con 0x56155fc04d70 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b440a5a90 con 0x56155fc04d70 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b2bfff640 1 -- 172.21.15.43:0/1838861953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c02d050 con 0x56155fc04d70 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 10 client.6447 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56155f9c1ca0 con 0x56155fc04d70 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b2bfff640 1 -- 172.21.15.43:0/1838861953 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b440a5a90 con 0x56155fc04d70 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b2bfff640 1 -- 172.21.15.43:0/1838861953 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f4b3c030020 con 0x56155fc04d70 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 10 client.6447 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 10 client.6447 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 20 client.6447 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc0c5d0 0x56155fc0c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 4 client.6447 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:31.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56155f9a4c90 con 0x56155fc0c5d0 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.938+0000 7f4b5139b640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc0c5d0 0x56155fc0c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b2bfff640 10 client.6447 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b5139b640 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc0c5d0 0x56155fc0c9b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f64cd1f681b2dee server_cookie=ca49f2edd0b32123 in_seq=0 out_seq=0 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b2bfff640 1 -- 172.21.15.43:0/1838861953 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x56155f9a4c90 con 0x56155fc0c5d0 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b2bfff640 10 client.6447 handle_command_reply: tid=0 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 client.6447 shutdown 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc04d70 msgr2=0x56155fbe9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc04d70 0x56155fbe9da0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00dce0 tx=0x7f4b3c000bd0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 shutdown_connections 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc0c5d0 0x56155fc0c9b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc05690 0x56155fbea2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc04d70 0x56155fbe9da0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b2bfff640 0 client.6447 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 --2- 172.21.15.43:0/1838861953 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbc0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 >> 172.21.15.43:0/1838861953 conn(0x56155fbb77e0 msgr2=0x56155fbba460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 shutdown_connections 2024-03-30T13:14:31.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 1 -- 172.21.15.43:0/1838861953 wait complete. 2024-03-30T13:14:31.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.942+0000 7f4b56781000 20 client.6447 trim_cache size 0 max 16384 2024-03-30T13:14:31.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 Processor -- start 2024-03-30T13:14:31.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- start start 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbea180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fbb9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fbba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c203f20 con 0x56155fc05690 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c202f90 con 0x56155fbbb830 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c154d20 con 0x56155fc04d70 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fbba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fbba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33044/0 (socket says 172.21.15.43:33044) 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 -- 172.21.15.43:0/1911858608 learned_addr learned my addr 172.21.15.43:0/1911858608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b5139b640 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fbb9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 -- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 msgr2=0x56155fbb9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fbb9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 -- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 msgr2=0x56155fbea180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbea180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 -- 172.21.15.43:0/1911858608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56155f990d00 con 0x56155fc05690 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fbba110 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000f90 tx=0x7f4b440a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a58a670fae7397 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b367fc640 1 -- 172.21.15.43:0/1911858608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44009190 con 0x56155fc05690 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b367fc640 1 -- 172.21.15.43:0/1911858608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b440a58f0 con 0x56155fc05690 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b367fc640 1 -- 172.21.15.43:0/1911858608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440039a0 con 0x56155fc05690 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- 172.21.15.43:0/1911858608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 msgr2=0x56155fbba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fbba110 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000f90 tx=0x7f4b440a7000 comp rx=0 tx=0).stop 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- 172.21.15.43:0/1911858608 shutdown_connections 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fbba110 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fbb9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 --2- 172.21.15.43:0/1911858608 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fbea180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- 172.21.15.43:0/1911858608 >> 172.21.15.43:0/1911858608 conn(0x56155fbb77e0 msgr2=0x56155fc1bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- 172.21.15.43:0/1911858608 shutdown_connections 2024-03-30T13:14:31.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- 172.21.15.43:0/1911858608 wait complete. 2024-03-30T13:14:31.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 -- start start 2024-03-30T13:14:31.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.958+0000 7f4b56781000 1 Processor -- start 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fc76f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc77470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fc08ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x56155fa08db0 con 0x56155fc05690 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c155370 con 0x56155fbbb830 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x56155fa106b0 con 0x56155fc04d70 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 10 client.0 _ll_register_callbacks cb 0x7f4b3c00d490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fc08ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fc08ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33056/0 (socket says 172.21.15.43:33056) 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 -- 172.21.15.43:0/2145927828 learned_addr learned my addr 172.21.15.43:0/2145927828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:31.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc77470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b5139b640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fc76f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 -- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 msgr2=0x56155fc77470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc77470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 -- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 msgr2=0x56155fc76f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fc76f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 -- 172.21.15.43:0/2145927828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56155fa8af70 con 0x56155fc05690 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b50b9a640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc77470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b5139b640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fc76f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b51b9c640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fc08ad0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4b440c4040 tx=0x7f4b4401b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228d45b44781e8c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 1 -- 172.21.15.43:0/2145927828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440a5aa0 con 0x56155fc05690 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 1 -- 172.21.15.43:0/2145927828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6159+0+0 (secure 0 0 0) 0x7f4b44009190 con 0x56155fc05690 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b440a58f0 con 0x56155fc05690 2024-03-30T13:14:31.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 1 -- 172.21.15.43:0/2145927828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4b440d3050 con 0x56155fc05690 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 10 client.6453 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x56155f9b0760 con 0x56155fc05690 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 1 -- 172.21.15.43:0/2145927828 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b440a58f0 con 0x56155fc05690 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 1 -- 172.21.15.43:0/2145927828 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f4b440d6020 con 0x56155fc05690 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 10 client.6453 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 10 client.6453 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 20 client.6453 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc09490 0x56155fc098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 4 client.6453 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628404}] 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628404}) -- 0x56155fa16fe0 con 0x56155fc09490 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b5139b640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc09490 0x56155fc098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 10 client.6453 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b5139b640 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc09490 0x56155fc098a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7865c819e2d2e942 server_cookie=1c1d1aa8850e4cb2 in_seq=0 out_seq=0 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 1 -- 172.21.15.43:0/2145927828 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3800 (crc 0 0 0) 0x56155fa16fe0 con 0x56155fc09490 2024-03-30T13:14:31.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 10 client.6453 handle_command_reply: tid=0 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 client.6453 shutdown 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 msgr2=0x56155fc08ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fc08ad0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4b440c4040 tx=0x7f4b4401b240 comp rx=0 tx=0).stop 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 shutdown_connections 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x56155fc09490 0x56155fc098a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x56155fc05690 0x56155fc08ad0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x56155fc04d70 0x56155fc77470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b2bfff640 0 client.6453 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 --2- 172.21.15.43:0/2145927828 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x56155fbbb830 0x56155fc76f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 >> 172.21.15.43:0/2145927828 conn(0x56155fbb77e0 msgr2=0x56155fc1c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 shutdown_connections 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 1 -- 172.21.15.43:0/2145927828 wait complete. 2024-03-30T13:14:31.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.962+0000 7f4b56781000 20 client.6453 trim_cache size 0 max 16384 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr_user.c", 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628404, 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.559013+0000", 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.760452+0000", 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:31.974 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:31.975 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1232, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.760452+0000", 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:31.976 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1232, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.559013+0000" 2024-03-30T13:14:31.977 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1232, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.559013+0000" 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 148, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 38, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:31.978 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.979 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.980 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.981 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:31.982 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:31.983 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:31.984 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:31.985 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4b30067ff0 msgr2=0x7f4b3006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4b30067ff0 0x7f4b3006a4b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c15bf50 tx=0x7f4b3c009040 comp rx=0 tx=0).stop 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 msgr2=0x7f4b4c155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c155a60 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800d830 tx=0x7f4b4800dcf0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 shutdown_connections 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4b30067ff0 0x7f4b3006a4b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4b4c15ff40 0x7f4b4c15b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4b4c13f140 0x7f4b4c13d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 --2- 172.21.15.43:0/418886086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b4c06cd80 0x7f4b4c155a60 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 >> 172.21.15.43:0/418886086 conn(0x7f4b4c0d00a0 msgr2=0x7f4b4c072370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:31.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 shutdown_connections 2024-03-30T13:14:31.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:31.966+0000 7f4b535ff640 1 -- 172.21.15.43:0/418886086 wait complete. 2024-03-30T13:14:31.988 DEBUG:tasks.cephfs.test_uninlining:getting inode info #8 of 23:1099511628402 2024-03-30T13:14:31.988 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628402 2024-03-30T13:14:31.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628402 2024-03-30T13:14:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 Processor -- start 2024-03-30T13:14:32.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 -- start start 2024-03-30T13:14:32.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc15f880 0x7f90cc0b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90cc070640 0x7f90cc06e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f90cc06eec0 0x7f90cc0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f90cc070aa0 con 0x7f90cc070640 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f90cc0ce4d0 con 0x7f90cc06eec0 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90d0f52640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f90cc0cc7f0 con 0x7f90cc15f880 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90cc070640 0x7f90cc06e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f90cc070640 0x7f90cc06e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33066/0 (socket says 172.21.15.43:33066) 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 -- 172.21.15.43:0/3936398725 learned_addr learned my addr 172.21.15.43:0/3936398725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90ca59c640 1 --2- 172.21.15.43:0/3936398725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc15f880 0x7f90cc0b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90cad9d640 1 --2- 172.21.15.43:0/3936398725 >> v2:172.21.15.113:3300/0 conn(0x7f90cc06eec0 0x7f90cc0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 -- 172.21.15.43:0/3936398725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc15f880 msgr2=0x7f90cc0b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/3936398725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc15f880 0x7f90cc0b4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 -- 172.21.15.43:0/3936398725 >> v2:172.21.15.113:3300/0 conn(0x7f90cc06eec0 msgr2=0x7f90cc0ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/3936398725 >> v2:172.21.15.113:3300/0 conn(0x7f90cc06eec0 0x7f90cc0ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.110+0000 7f90c1d9b640 1 -- 172.21.15.43:0/3936398725 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc0ccad0 con 0x7f90cc070640 2024-03-30T13:14:32.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/3936398725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc070640 0x7f90cc06e980 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0cc6e0 tx=0x7f90b401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce719b214f3c606f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90c9d9b640 1 -- 172.21.15.43:0/3936398725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b40b0070 con 0x7f90cc070640 2024-03-30T13:14:32.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90c9d9b640 1 -- 172.21.15.43:0/3936398725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90b40a4e50 con 0x7f90cc070640 2024-03-30T13:14:32.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90c9d9b640 1 -- 172.21.15.43:0/3936398725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b40ab820 con 0x7f90cc070640 2024-03-30T13:14:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 -- 172.21.15.43:0/3936398725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc070640 msgr2=0x7f90cc06e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 --2- 172.21.15.43:0/3936398725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc070640 0x7f90cc06e980 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0cc6e0 tx=0x7f90b401c400 comp rx=0 tx=0).stop 2024-03-30T13:14:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 -- 172.21.15.43:0/3936398725 shutdown_connections 2024-03-30T13:14:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 --2- 172.21.15.43:0/3936398725 >> v2:172.21.15.113:3300/0 conn(0x7f90cc06eec0 0x7f90cc0ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 --2- 172.21.15.43:0/3936398725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc070640 0x7f90cc06e980 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 --2- 172.21.15.43:0/3936398725 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc15f880 0x7f90cc0b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 -- 172.21.15.43:0/3936398725 >> 172.21.15.43:0/3936398725 conn(0x7f90cc0b3cf0 msgr2=0x7f90cc0b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 -- 172.21.15.43:0/3936398725 shutdown_connections 2024-03-30T13:14:32.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 -- 172.21.15.43:0/3936398725 wait complete. 2024-03-30T13:14:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 Processor -- start 2024-03-30T13:14:32.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.114+0000 7f90d0f52640 1 -- start start 2024-03-30T13:14:32.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90cc06eec0 0x7f90cc1543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc070640 0x7f90cc143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc15f880 0x7f90cc0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc1609d0 con 0x7f90cc15f880 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc160850 con 0x7f90cc06eec0 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f90cc0d0000 con 0x7f90cc070640 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc15f880 0x7f90cc0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc15f880 0x7f90cc0b57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33078/0 (socket says 172.21.15.43:33078) 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 -- 172.21.15.43:0/1197572535 learned_addr learned my addr 172.21.15.43:0/1197572535 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc070640 0x7f90cc143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90ca59c640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90cc06eec0 0x7f90cc1543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 -- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc070640 msgr2=0x7f90cc143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc070640 0x7f90cc143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 -- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90cc06eec0 msgr2=0x7f90cc1543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90cc06eec0 0x7f90cc1543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 -- 172.21.15.43:0/1197572535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc15ce80 con 0x7f90cc15f880 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90cad9d640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc15f880 0x7f90cc0b57c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f90c400cda0 tx=0x7f90c400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90c37fe640 1 -- 172.21.15.43:0/1197572535 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4003ba0 con 0x7f90cc15f880 2024-03-30T13:14:32.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc143c90 con 0x7f90cc15f880 2024-03-30T13:14:32.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc144250 con 0x7f90cc15f880 2024-03-30T13:14:32.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90c37fe640 1 -- 172.21.15.43:0/1197572535 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c4003d40 con 0x7f90cc15f880 2024-03-30T13:14:32.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90c37fe640 1 -- 172.21.15.43:0/1197572535 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c4004240 con 0x7f90cc15f880 2024-03-30T13:14:32.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.118+0000 7f90c37fe640 1 -- 172.21.15.43:0/1197572535 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f90c40043e0 con 0x7f90cc15f880 2024-03-30T13:14:32.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90c37fe640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90ac067fc0 0x7f90ac06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90c37fe640 1 -- 172.21.15.43:0/1197572535 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f90c4087100 con 0x7f90cc15f880 2024-03-30T13:14:32.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90ac067fc0 0x7f90ac06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90ac067fc0 0x7f90ac06a480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009a40 tx=0x7f90bc009190 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 Processor -- start 2024-03-30T13:14:32.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 -- start start 2024-03-30T13:14:32.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510ed70 0x55cb3510f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb350c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb34f13aa0 con 0x55cb3510ed70 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cb34e9aaf0 con 0x55cb3510f690 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55cb34f0fa50 con 0x55cb350c5830 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510ed70 0x55cb3510f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510ed70 0x55cb3510f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33080/0 (socket says 172.21.15.43:33080) 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 -- 172.21.15.43:0/4033499697 learned_addr learned my addr 172.21.15.43:0/4033499697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90cad9d640 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb350c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 -- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 msgr2=0x55cb350c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb350c5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 -- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 msgr2=0x55cb350c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 -- 172.21.15.43:0/4033499697 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb34e98d00 con 0x55cb3510ed70 2024-03-30T13:14:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.122+0000 7f90ca59c640 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510ed70 0x55cb3510f150 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f90bc011dc0 tx=0x7f90bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e711324697bf966 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90b3fff640 1 -- 172.21.15.43:0/4033499697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc01b070 con 0x55cb3510ed70 2024-03-30T13:14:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90b3fff640 1 -- 172.21.15.43:0/4033499697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90bc027070 con 0x55cb3510ed70 2024-03-30T13:14:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90b3fff640 1 -- 172.21.15.43:0/4033499697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc003a30 con 0x55cb3510ed70 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- 172.21.15.43:0/4033499697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510ed70 msgr2=0x55cb3510f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510ed70 0x55cb3510f150 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f90bc011dc0 tx=0x7f90bc014000 comp rx=0 tx=0).stop 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- 172.21.15.43:0/4033499697 shutdown_connections 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb350c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 --2- 172.21.15.43:0/4033499697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510ed70 0x55cb3510f150 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- 172.21.15.43:0/4033499697 >> 172.21.15.43:0/4033499697 conn(0x55cb350c17e0 msgr2=0x55cb350c3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- 172.21.15.43:0/4033499697 shutdown_connections 2024-03-30T13:14:32.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- 172.21.15.43:0/4033499697 wait complete. 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- start start 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 Processor -- start 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb350cac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510ed70 0x55cb350f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55cb34edb270 con 0x55cb350c5830 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cb34ed16a0 con 0x55cb3510f690 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55cb34eab880 con 0x55cb3510ed70 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90d40d4000 10 client.0 _ll_register_callbacks cb 0x55cb350b0490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb350cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb350cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33092/0 (socket says 172.21.15.43:33092) 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 -- 172.21.15.43:0/3942815729 learned_addr learned my addr 172.21.15.43:0/3942815729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510ed70 0x55cb350f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90cad9d640 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 -- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510ed70 msgr2=0x55cb350f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510ed70 0x55cb350f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 -- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 msgr2=0x55cb350f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 -- 172.21.15.43:0/3942815729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb34ed8730 con 0x55cb350c5830 2024-03-30T13:14:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90ca59c640 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb350cac50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x55cb34fd3c80 tx=0x7f90bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d31f882229d0d3c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.126+0000 7f90b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 1 -- 172.21.15.43:0/3942815729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc01b040 con 0x55cb350c5830 2024-03-30T13:14:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 1 -- 172.21.15.43:0/3942815729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f90bc036d00 con 0x55cb350c5830 2024-03-30T13:14:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90bc027070 con 0x55cb350c5830 2024-03-30T13:14:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 1 -- 172.21.15.43:0/3942815729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90bc03d050 con 0x55cb350c5830 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 10 client.6465 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55cb34ecbca0 con 0x55cb350c5830 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 1 -- 172.21.15.43:0/3942815729 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90bc027070 con 0x55cb350c5830 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 1 -- 172.21.15.43:0/3942815729 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f90bc040020 con 0x55cb350c5830 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 10 client.6465 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 10 client.6465 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 20 client.6465 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb351165d0 0x55cb351169e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 4 client.6465 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb34eaec90 con 0x55cb351165d0 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb351165d0 0x55cb351169e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb351165d0 0x55cb351169e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af67d248d4a896b server_cookie=e6a250cbba473c08 in_seq=0 out_seq=0 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 10 client.6465 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 1 -- 172.21.15.43:0/3942815729 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55cb34eaec90 con 0x55cb351165d0 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 10 client.6465 handle_command_reply: tid=0 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 client.6465 shutdown 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 msgr2=0x55cb350cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb350cac50 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x55cb34fd3c80 tx=0x7f90bc004740 comp rx=0 tx=0).stop 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 shutdown_connections 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb351165d0 0x55cb351169e0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510f690 0x55cb350f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510ed70 0x55cb350f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 --2- 172.21.15.43:0/3942815729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb350cac50 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 >> 172.21.15.43:0/3942815729 conn(0x55cb350c17e0 msgr2=0x55cb350c4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90b17fa640 0 client.6465 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 shutdown_connections 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 1 -- 172.21.15.43:0/3942815729 wait complete. 2024-03-30T13:14:32.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.130+0000 7f90d40d4000 20 client.6465 trim_cache size 0 max 16384 2024-03-30T13:14:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.146+0000 7f90d40d4000 1 Processor -- start 2024-03-30T13:14:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.146+0000 7f90d40d4000 1 -- start start 2024-03-30T13:14:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb35114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb350f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510f690 0x55cb350f3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90a800d080 con 0x55cb3510f690 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f90a800c060 con 0x55cb3510ed70 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f90a800b340 con 0x55cb350c5830 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510f690 0x55cb350f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510f690 0x55cb350f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33100/0 (socket says 172.21.15.43:33100) 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 -- 172.21.15.43:0/2527598264 learned_addr learned my addr 172.21.15.43:0/2527598264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb35114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb350f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 -- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 msgr2=0x55cb35114ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb35114ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 -- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 msgr2=0x55cb350f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb350f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 -- 172.21.15.43:0/2527598264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb34e9ad00 con 0x55cb3510f690 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb35114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb350f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510f690 0x55cb350f3290 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f90c4056cb0 tx=0x7f90c40569d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d1b972ee893d66 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90b37fe640 1 -- 172.21.15.43:0/2527598264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40741c0 con 0x55cb3510f690 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90b37fe640 1 -- 172.21.15.43:0/2527598264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c406f070 con 0x55cb3510f690 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90b37fe640 1 -- 172.21.15.43:0/2527598264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c40552b0 con 0x55cb3510f690 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- 172.21.15.43:0/2527598264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510f690 msgr2=0x55cb350f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510f690 0x55cb350f3290 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f90c4056cb0 tx=0x7f90c40569d0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- 172.21.15.43:0/2527598264 shutdown_connections 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb3510f690 0x55cb350f3290 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb350f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- 172.21.15.43:0/2527598264 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb350c5830 0x55cb35114ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- 172.21.15.43:0/2527598264 >> 172.21.15.43:0/2527598264 conn(0x55cb350c17e0 msgr2=0x55cb350f22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- 172.21.15.43:0/2527598264 shutdown_connections 2024-03-30T13:14:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- 172.21.15.43:0/2527598264 wait complete. 2024-03-30T13:14:32.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- start start 2024-03-30T13:14:32.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 Processor -- start 2024-03-30T13:14:32.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb3517fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb35111a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510f690 0x55cb35111fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f90a800b990 con 0x55cb350c5830 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55cb34fcfb00 con 0x55cb3510ed70 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55cb350f29c0 con 0x55cb3510f690 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90d40d4000 10 client.0 _ll_register_callbacks cb 0x7f90bc009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb3517fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb3517fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33110/0 (socket says 172.21.15.43:33110) 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 -- 172.21.15.43:0/773286110 learned_addr learned my addr 172.21.15.43:0/773286110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90cad9d640 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510f690 0x55cb35111fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb35111a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 -- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510f690 msgr2=0x55cb35111fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510f690 0x55cb35111fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 -- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 msgr2=0x55cb35111a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb35111a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 -- 172.21.15.43:0/773286110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb34f94f70 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90ca59c640 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb3517fdd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0044b0 tx=0x7f90bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da10e475adf13fd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.150+0000 7f90b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 1 -- 172.21.15.43:0/773286110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc01b040 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 1 -- 172.21.15.43:0/773286110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f90bc022c00 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90c406f070 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 1 -- 172.21.15.43:0/773286110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f90bc048050 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 10 client.6471 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55cb34eba760 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 1 -- 172.21.15.43:0/773286110 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c406f070 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 1 -- 172.21.15.43:0/773286110 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f90bc04b020 con 0x55cb350c5830 2024-03-30T13:14:32.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 10 client.6471 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 10 client.6471 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 20 client.6471 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb35112500 0x55cb35112910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 4 client.6471 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628402}] 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628402}) -- 0x55cb34f20fe0 con 0x55cb35112500 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb35112500 0x55cb35112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90c1d9b640 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb35112500 0x55cb35112910 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ceca87696ed87 server_cookie=a18e9d42adb52fa1 in_seq=0 out_seq=0 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 10 client.6471 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 1 -- 172.21.15.43:0/773286110 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3804 (crc 0 0 0) 0x55cb34f20fe0 con 0x55cb35112500 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 10 client.6471 handle_command_reply: tid=0 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 client.6471 shutdown 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 msgr2=0x55cb3517fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb3517fdd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f90bc0044b0 tx=0x7f90bc018000 comp rx=0 tx=0).stop 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 shutdown_connections 2024-03-30T13:14:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55cb35112500 0x55cb35112910 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55cb3510f690 0x55cb35111fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55cb3510ed70 0x55cb35111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 --2- 172.21.15.43:0/773286110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55cb350c5830 0x55cb3517fdd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 >> 172.21.15.43:0/773286110 conn(0x55cb350c17e0 msgr2=0x55cb35125680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90b17fa640 0 client.6471 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 shutdown_connections 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 1 -- 172.21.15.43:0/773286110 wait complete. 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d40d4000 20 client.6471 trim_cache size 0 max 16384 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:32.164 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr_security.c", 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628402, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.563226+0000", 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.752454+0000", 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.165 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1473, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:32.166 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.752454+0000", 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.167 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1473, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.563226+0000" 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1473, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.563226+0000" 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 152, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 34, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:32.169 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:32.170 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:32.171 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.172 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:32.173 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.174 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:32.175 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:32.176 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90ac067fc0 msgr2=0x7f90ac06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90ac067fc0 0x7f90ac06a480 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009a40 tx=0x7f90bc009190 comp rx=0 tx=0).stop 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc15f880 msgr2=0x7f90cc0b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc15f880 0x7f90cc0b57c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f90c400cda0 tx=0x7f90c400a660 comp rx=0 tx=0).stop 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 shutdown_connections 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f90ac067fc0 0x7f90ac06a480 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90cc15f880 0x7f90cc0b57c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f90cc070640 0x7f90cc143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 --2- 172.21.15.43:0/1197572535 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f90cc06eec0 0x7f90cc1543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 >> 172.21.15.43:0/1197572535 conn(0x7f90cc0b3cf0 msgr2=0x7f90cc15fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 shutdown_connections 2024-03-30T13:14:32.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.154+0000 7f90d0f52640 1 -- 172.21.15.43:0/1197572535 wait complete. 2024-03-30T13:14:32.179 DEBUG:tasks.cephfs.test_uninlining:getting inode info #9 of 23:1099511628387 2024-03-30T13:14:32.179 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628387 2024-03-30T13:14:32.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628387 2024-03-30T13:14:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.290+0000 7f163f3b8640 1 Processor -- start 2024-03-30T13:14:32.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.290+0000 7f163f3b8640 1 -- start start 2024-03-30T13:14:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163f3b8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16380b46f0 0x7f163806f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163f3b8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f163806ff20 0x7f16380ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163f3b8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16380b4ba0 con 0x7f163815afc0 2024-03-30T13:14:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163f3b8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1638070c30 con 0x7f163806ff20 2024-03-30T13:14:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163f3b8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f163815ec60 con 0x7f16380b46f0 2024-03-30T13:14:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163c953640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16380b46f0 0x7f163806f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163c953640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16380b46f0 0x7f163806f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:40192/0 (socket says 172.21.15.43:40192) 2024-03-30T13:14:32.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163c953640 1 -- 172.21.15.43:0/399857301 learned_addr learned my addr 172.21.15.43:0/399857301 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163d955640 1 --2- 172.21.15.43:0/399857301 >> v2:172.21.15.113:3300/0 conn(0x7f163806ff20 0x7f16380ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163d955640 1 -- 172.21.15.43:0/399857301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16380b46f0 msgr2=0x7f163806f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163d955640 1 --2- 172.21.15.43:0/399857301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16380b46f0 0x7f163806f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163d955640 1 -- 172.21.15.43:0/399857301 >> v1:172.21.15.113:6789/0 conn(0x7f163815afc0 legacy=0x7f16380b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.294+0000 7f163d955640 1 -- 172.21.15.43:0/399857301 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163815ef40 con 0x7f163806ff20 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f163d955640 1 --2- 172.21.15.43:0/399857301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163806ff20 0x7f16380ce7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16280097c0 tx=0x7f162801c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74091d47b4a00e15 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f162ffff640 1 -- 172.21.15.43:0/399857301 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280b1070 con 0x7f163806ff20 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f162ffff640 1 -- 172.21.15.43:0/399857301 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16280ac640 con 0x7f163806ff20 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f162ffff640 1 -- 172.21.15.43:0/399857301 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280aca10 con 0x7f163806ff20 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f163f3b8640 1 -- 172.21.15.43:0/399857301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163806ff20 msgr2=0x7f16380ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f163f3b8640 1 --2- 172.21.15.43:0/399857301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163806ff20 0x7f16380ce7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16280097c0 tx=0x7f162801c460 comp rx=0 tx=0).stop 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f163f3b8640 1 -- 172.21.15.43:0/399857301 shutdown_connections 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f163f3b8640 1 --2- 172.21.15.43:0/399857301 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f163806ff20 0x7f16380ce7b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.302+0000 7f163f3b8640 1 --2- 172.21.15.43:0/399857301 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f16380b46f0 0x7f163806f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.306+0000 7f163f3b8640 1 -- 172.21.15.43:0/399857301 >> 172.21.15.43:0/399857301 conn(0x7f16380d1070 msgr2=0x7f16380d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.306+0000 7f163f3b8640 1 -- 172.21.15.43:0/399857301 shutdown_connections 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.306+0000 7f163f3b8640 1 -- 172.21.15.43:0/399857301 wait complete. 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.306+0000 7f163f3b8640 1 Processor -- start 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.306+0000 7f163f3b8640 1 -- start start 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163806ff20 0x7f1638148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16380b46f0 0x7f1638168220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f163815afc0 0x7f1638168760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f163815b520 con 0x7f163815afc0 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f163815b3a0 con 0x7f16380b46f0 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f16380cf500 con 0x7f163806ff20 2024-03-30T13:14:32.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f163815afc0 0x7f1638168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163c953640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16380b46f0 0x7f1638168220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f163815afc0 0x7f1638168760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33126/0 (socket says 172.21.15.43:33126) 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 -- 172.21.15.43:0/377241742 learned_addr learned my addr 172.21.15.43:0/377241742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d154640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163806ff20 0x7f1638148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 -- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163806ff20 msgr2=0x7f1638148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163806ff20 0x7f1638148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 -- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16380b46f0 msgr2=0x7f1638168220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16380b46f0 0x7f1638168220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 -- 172.21.15.43:0/377241742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1638161060 con 0x7f163815afc0 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163d955640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f163815afc0 0x7f1638168760 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f162801a090 tx=0x7f1628019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163c953640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16380b46f0 0x7f1638168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f162dffb640 1 -- 172.21.15.43:0/377241742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280b1040 con 0x7f163815afc0 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1638168dc0 con 0x7f163815afc0 2024-03-30T13:14:32.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1638169380 con 0x7f163815afc0 2024-03-30T13:14:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f162dffb640 1 -- 172.21.15.43:0/377241742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16280bd070 con 0x7f163815afc0 2024-03-30T13:14:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.310+0000 7f162dffb640 1 -- 172.21.15.43:0/377241742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280b6880 con 0x7f163815afc0 2024-03-30T13:14:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.314+0000 7f162dffb640 1 -- 172.21.15.43:0/377241742 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f16280b4070 con 0x7f163815afc0 2024-03-30T13:14:32.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.314+0000 7f162dffb640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f161c067fc0 0x7f161c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.314+0000 7f162dffb640 1 -- 172.21.15.43:0/377241742 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f1628120610 con 0x7f163815afc0 2024-03-30T13:14:32.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.314+0000 7f163d154640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f161c067fc0 0x7f161c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.314+0000 7f163d154640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f161c067fc0 0x7f161c06a480 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f16340098e0 tx=0x7f1634009060 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 Processor -- start 2024-03-30T13:14:32.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 -- start start 2024-03-30T13:14:32.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e16150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e16690 0x55e944dcc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944dcc830 0x55e944dccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e944c1aaa0 con 0x55e944e15d70 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e944ba1af0 con 0x55e944e16690 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e944c16a50 con 0x55e944dcc830 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e16690 0x55e944dcc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163d955640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944dcc830 0x55e944dccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e16690 0x55e944dcc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:43544/0 (socket says 172.21.15.43:43544) 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 -- 172.21.15.43:0/671052444 learned_addr learned my addr 172.21.15.43:0/671052444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163d154640 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e16150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 -- 172.21.15.43:0/671052444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944dcc830 msgr2=0x55e944dccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944dcc830 0x55e944dccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 -- 172.21.15.43:0/671052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 msgr2=0x55e944e16150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e16150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 -- 172.21.15.43:0/671052444 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e944b9fd00 con 0x55e944e16690 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163d154640 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e16150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.322+0000 7f163c953640 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e16690 0x55e944dcc2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f163000baf0 tx=0x7f163000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9dfaf6db170cba6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f1622ffd640 1 -- 172.21.15.43:0/671052444 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630014620 con 0x55e944e16690 2024-03-30T13:14:32.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f1622ffd640 1 -- 172.21.15.43:0/671052444 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1630019070 con 0x55e944e16690 2024-03-30T13:14:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f164253a000 1 -- 172.21.15.43:0/671052444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e16690 msgr2=0x55e944dcc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f164253a000 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e16690 0x55e944dcc2f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f163000baf0 tx=0x7f163000b040 comp rx=0 tx=0).stop 2024-03-30T13:14:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f164253a000 1 -- 172.21.15.43:0/671052444 shutdown_connections 2024-03-30T13:14:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f164253a000 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944dcc830 0x55e944dccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f164253a000 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e16690 0x55e944dcc2f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f164253a000 1 --2- 172.21.15.43:0/671052444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e16150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.326+0000 7f164253a000 1 -- 172.21.15.43:0/671052444 >> 172.21.15.43:0/671052444 conn(0x55e944dc87e0 msgr2=0x55e944dcabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.330+0000 7f164253a000 1 -- 172.21.15.43:0/671052444 shutdown_connections 2024-03-30T13:14:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.330+0000 7f164253a000 1 -- 172.21.15.43:0/671052444 wait complete. 2024-03-30T13:14:32.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.330+0000 7f164253a000 1 -- start start 2024-03-30T13:14:32.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 Processor -- start 2024-03-30T13:14:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e1c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e1cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dfa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e944be2270 con 0x55e944e15d70 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e944bd86a0 con 0x55e944dcc830 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e944bb2880 con 0x55e944df9d50 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 10 client.0 _ll_register_callbacks cb 0x55e944db7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163c953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e1cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d154640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e1c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d955640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dfa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163c953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e1cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33150/0 (socket says 172.21.15.43:33150) 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163c953640 1 -- 172.21.15.43:0/1245347216 learned_addr learned my addr 172.21.15.43:0/1245347216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d955640 1 -- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 msgr2=0x55e944e1c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d955640 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e1c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d955640 1 -- 172.21.15.43:0/1245347216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 msgr2=0x55e944e1cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d955640 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e1cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d955640 1 -- 172.21.15.43:0/1245347216 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({osdmap=0}) -- 0x55e944bdf730 con 0x55e944df9d50 2024-03-30T13:14:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f163d955640 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dfa130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f162811e970 tx=0x7f16280f01c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55716f313eb1d70d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f1617fff640 10 client.0 ms_handle_connect on v2:172.21.15.113:3301/0 2024-03-30T13:14:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f1617fff640 1 -- 172.21.15.43:0/1245347216 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280f4970 con 0x55e944df9d50 2024-03-30T13:14:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e944bb5c90 con 0x55e944df9d50 2024-03-30T13:14:32.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.334+0000 7f1617fff640 1 -- 172.21.15.43:0/1245347216 <== mon.2 v2:172.21.15.113:3301/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f16280f8d10 con 0x55e944df9d50 2024-03-30T13:14:32.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f1617fff640 1 -- 172.21.15.43:0/1245347216 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16280f91a0 con 0x55e944df9d50 2024-03-30T13:14:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 10 client.5468 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55e944bd2ca0 con 0x55e944df9d50 2024-03-30T13:14:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f1617fff640 1 -- 172.21.15.43:0/1245347216 <== mon.2 v2:172.21.15.113:3301/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e944bb5c90 con 0x55e944df9d50 2024-03-30T13:14:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f1617fff640 1 -- 172.21.15.43:0/1245347216 <== mon.2 v2:172.21.15.113:3301/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f16280efa10 con 0x55e944df9d50 2024-03-30T13:14:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 10 client.5468 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 10 client.5468 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 20 client.5468 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944dfaed0 0x55e944dfb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 4 client.5468 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e944c27fe0 con 0x55e944dfaed0 2024-03-30T13:14:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f163d154640 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944dfaed0 0x55e944dfb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f1617fff640 10 client.5468 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.338+0000 7f163d154640 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944dfaed0 0x55e944dfb2e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d5918167341683 server_cookie=9b06f8c1e15198a4 in_seq=0 out_seq=0 2024-03-30T13:14:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f1617fff640 1 -- 172.21.15.43:0/1245347216 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55e944c27fe0 con 0x55e944dfaed0 2024-03-30T13:14:32.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f1617fff640 10 client.5468 handle_command_reply: tid=0 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 client.5468 shutdown 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 msgr2=0x55e944dfa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dfa130 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f162811e970 tx=0x7f16280f01c0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 shutdown_connections 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944dfaed0 0x55e944dfb2e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dfa130 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944e15d70 0x55e944e1cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 --2- 172.21.15.43:0/1245347216 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e1c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 >> 172.21.15.43:0/1245347216 conn(0x55e944dc87e0 msgr2=0x55e944dcb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f1617fff640 0 client.5468 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 shutdown_connections 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 1 -- 172.21.15.43:0/1245347216 wait complete. 2024-03-30T13:14:32.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.342+0000 7f164253a000 20 client.5468 trim_cache size 0 max 16384 2024-03-30T13:14:32.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.366+0000 7f164253a000 1 Processor -- start 2024-03-30T13:14:32.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.366+0000 7f164253a000 1 -- start start 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944dcc830 0x55e944dcabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dcb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e15d70 0x55e944dcb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f161800d140 con 0x55e944dcc830 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f161800c120 con 0x55e944e15d70 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f161800b400 con 0x55e944df9d50 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944dcc830 0x55e944dcabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944dcc830 0x55e944dcabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33160/0 (socket says 172.21.15.43:33160) 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 -- 172.21.15.43:0/268069350 learned_addr learned my addr 172.21.15.43:0/268069350 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d955640 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e15d70 0x55e944dcb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163c953640 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dcb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 -- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 msgr2=0x55e944dcb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dcb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 -- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e15d70 msgr2=0x55e944dcb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e15d70 0x55e944dcb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 -- 172.21.15.43:0/268069350 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e944dfa4a0 con 0x55e944dcc830 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944dcc830 0x55e944dcabd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f163400ea30 tx=0x7f1634016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9e8f5f1d35cec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f16227fc640 1 -- 172.21.15.43:0/268069350 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163401b040 con 0x55e944dcc830 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f16227fc640 1 -- 172.21.15.43:0/268069350 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1634015be0 con 0x55e944dcc830 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f16227fc640 1 -- 172.21.15.43:0/268069350 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1634029b40 con 0x55e944dcc830 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- 172.21.15.43:0/268069350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944dcc830 msgr2=0x55e944dcabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944dcc830 0x55e944dcabd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f163400ea30 tx=0x7f1634016000 comp rx=0 tx=0).stop 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- 172.21.15.43:0/268069350 shutdown_connections 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944e15d70 0x55e944dcb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944df9d50 0x55e944dcb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- 172.21.15.43:0/268069350 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944dcc830 0x55e944dcabd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- 172.21.15.43:0/268069350 >> 172.21.15.43:0/268069350 conn(0x55e944dc87e0 msgr2=0x55e944df92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- 172.21.15.43:0/268069350 shutdown_connections 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- 172.21.15.43:0/268069350 wait complete. 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- start start 2024-03-30T13:14:32.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 Processor -- start 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e87730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944df9d50 0x55e944e87c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944e15d70 0x55e944e191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e944bb2880 con 0x55e944df9d50 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f161800ba50 con 0x55e944dcc830 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e944df99c0 con 0x55e944e15d70 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f164253a000 10 client.0 _ll_register_callbacks cb 0x7f16280f0480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163c953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944df9d50 0x55e944e87c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163c953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944df9d50 0x55e944e87c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33176/0 (socket says 172.21.15.43:33176) 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163c953640 1 -- 172.21.15.43:0/1620263407 learned_addr learned my addr 172.21.15.43:0/1620263407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d154640 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e87730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.370+0000 7f163d955640 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944e15d70 0x55e944e191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163c953640 1 -- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944e15d70 msgr2=0x55e944e191b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163c953640 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944e15d70 0x55e944e191b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163c953640 1 -- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 msgr2=0x55e944e87730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163c953640 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e87730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163c953640 1 -- 172.21.15.43:0/1620263407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e944ba1d00 con 0x55e944df9d50 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163c953640 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944df9d50 0x55e944e87c70 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1630009fd0 tx=0x7f1630004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a427b79ac671d7bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 1 -- 172.21.15.43:0/1620263407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1630014d90 con 0x55e944df9d50 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 1 -- 172.21.15.43:0/1620263407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f1630028ce0 con 0x55e944df9d50 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1634015be0 con 0x55e944df9d50 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 1 -- 172.21.15.43:0/1620263407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f16300039a0 con 0x55e944df9d50 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 10 client.6486 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55e944c9bf70 con 0x55e944df9d50 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 1 -- 172.21.15.43:0/1620263407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1634015be0 con 0x55e944df9d50 2024-03-30T13:14:32.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 1 -- 172.21.15.43:0/1620263407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f1630003b40 con 0x55e944df9d50 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 10 client.6486 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 10 client.6486 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 20 client.6486 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944e196f0 0x55e944e19b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 4 client.6486 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628387}] 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628387}) -- 0x55e944c22330 con 0x55e944e196f0 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163d154640 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944e196f0 0x55e944e19b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 10 client.6486 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163d154640 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944e196f0 0x55e944e19b00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8391d717a93b85cb server_cookie=f25fc1470f98fe46 in_seq=0 out_seq=0 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 1 -- 172.21.15.43:0/1620263407 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3792 (crc 0 0 0) 0x55e944c22330 con 0x55e944e196f0 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 10 client.6486 handle_command_reply: tid=0 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 client.6486 shutdown 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944df9d50 msgr2=0x55e944e87c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944df9d50 0x55e944e87c70 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f1630009fd0 tx=0x7f1630004ac0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 shutdown_connections 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e944e196f0 0x55e944e19b00 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f1617fff640 0 client.6486 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e944e15d70 0x55e944e191b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e944df9d50 0x55e944e87c70 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 --2- 172.21.15.43:0/1620263407 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e944dcc830 0x55e944e87730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 >> 172.21.15.43:0/1620263407 conn(0x55e944dc87e0 msgr2=0x55e944dd12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 shutdown_connections 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 1 -- 172.21.15.43:0/1620263407 wait complete. 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f164253a000 20 client.6486 trim_cache size 0 max 16384 2024-03-30T13:14:32.384 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/Makefile", 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628387, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.580822+0000", 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.649021+0000", 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:32.385 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 378, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:32.386 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.649021+0000", 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:32.387 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 378, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.580822+0000" 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 378, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.388 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.580822+0000" 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 160, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 4, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:32.389 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:32.390 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:32.391 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.392 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:32.393 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.394 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:32.395 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:32.396 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:32.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f161c067fc0 msgr2=0x7f161c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f161c067fc0 0x7f161c06a480 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f16340098e0 tx=0x7f1634009060 comp rx=0 tx=0).stop 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f163815afc0 msgr2=0x7f1638168760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f163815afc0 0x7f1638168760 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f162801a090 tx=0x7f1628019200 comp rx=0 tx=0).stop 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 shutdown_connections 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f161c067fc0 0x7f161c06a480 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f163815afc0 0x7f1638168760 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f16380b46f0 0x7f1638168220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 --2- 172.21.15.43:0/377241742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f163806ff20 0x7f1638148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 >> 172.21.15.43:0/377241742 conn(0x7f16380d1070 msgr2=0x7f163806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 shutdown_connections 2024-03-30T13:14:32.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.374+0000 7f163f3b8640 1 -- 172.21.15.43:0/377241742 wait complete. 2024-03-30T13:14:32.399 DEBUG:tasks.cephfs.test_uninlining:getting inode info #10 of 23:1099511628399 2024-03-30T13:14:32.399 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628399 2024-03-30T13:14:32.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628399 2024-03-30T13:14:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.510+0000 7f1e3ba98640 1 Processor -- start 2024-03-30T13:14:32.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.510+0000 7f1e3ba98640 1 -- start start 2024-03-30T13:14:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3ba98640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3415e4c0 0x7f1e340b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3ba98640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e340cf6c0 0x7f1e340d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3ba98640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e3406c850 con 0x7f1e3406c070 2024-03-30T13:14:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3ba98640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1e3415e8a0 con 0x7f1e340cf6c0 2024-03-30T13:14:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3ba98640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1e3406cb30 con 0x7f1e3415e4c0 2024-03-30T13:14:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e340cf6c0 0x7f1e340d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1e340cf6c0 0x7f1e340d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:43586/0 (socket says 172.21.15.43:43586) 2024-03-30T13:14:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 -- 172.21.15.43:0/590483786 learned_addr learned my addr 172.21.15.43:0/590483786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e39834640 1 --2- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3415e4c0 0x7f1e340b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 -- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3415e4c0 msgr2=0x7f1e340b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 --2- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3415e4c0 0x7f1e340b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 -- 172.21.15.43:0/590483786 >> v1:172.21.15.113:6789/0 conn(0x7f1e3406c070 legacy=0x7f1e340cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e38832640 1 -- 172.21.15.43:0/590483786 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689718302 0 0) 0x7f1e3406c850 con 0x7f1e3406c070 2024-03-30T13:14:32.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 -- 172.21.15.43:0/590483786 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e3406ce10 con 0x7f1e340cf6c0 2024-03-30T13:14:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.514+0000 7f1e3a035640 1 --2- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e340d2d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e240098a0 tx=0x7f1e2401c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4adc4c745f010c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e38832640 1 -- 172.21.15.43:0/590483786 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e240b0070 con 0x7f1e340cf6c0 2024-03-30T13:14:32.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e38832640 1 -- 172.21.15.43:0/590483786 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e240ab600 con 0x7f1e340cf6c0 2024-03-30T13:14:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e38832640 1 -- 172.21.15.43:0/590483786 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e240ab9d0 con 0x7f1e340cf6c0 2024-03-30T13:14:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 msgr2=0x7f1e340d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e340d2d70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1e240098a0 tx=0x7f1e2401c540 comp rx=0 tx=0).stop 2024-03-30T13:14:32.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- 172.21.15.43:0/590483786 shutdown_connections 2024-03-30T13:14:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e340d2d70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/590483786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3415e4c0 0x7f1e340b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- 172.21.15.43:0/590483786 >> 172.21.15.43:0/590483786 conn(0x7f1e340b6410 msgr2=0x7f1e340d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- 172.21.15.43:0/590483786 shutdown_connections 2024-03-30T13:14:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- 172.21.15.43:0/590483786 wait complete. 2024-03-30T13:14:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 Processor -- start 2024-03-30T13:14:32.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- start start 2024-03-30T13:14:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3406c070 0x7f1e34153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e34158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e3415e4c0 0x7f1e34151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e340d3e50 con 0x7f1e3415e4c0 2024-03-30T13:14:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e340d3cd0 con 0x7f1e340cf6c0 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1e340d3fd0 con 0x7f1e3406c070 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e3415e4c0 0x7f1e34151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e3415e4c0 0x7f1e34151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33192/0 (socket says 172.21.15.43:33192) 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 -- 172.21.15.43:0/1847399463 learned_addr learned my addr 172.21.15.43:0/1847399463 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e39834640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3406c070 0x7f1e34153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e39033640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e34158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 -- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3406c070 msgr2=0x7f1e34153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3406c070 0x7f1e34153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 -- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 msgr2=0x7f1e34158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e34158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 -- 172.21.15.43:0/1847399463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e34151df0 con 0x7f1e3415e4c0 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e39033640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e34158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e39834640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3406c070 0x7f1e34153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3a035640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e3415e4c0 0x7f1e34151510 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1e2401abf0 tx=0x7f1e2401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e2a7fc640 1 -- 172.21.15.43:0/1847399463 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e240ba070 con 0x7f1e3415e4c0 2024-03-30T13:14:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e2a7fc640 1 -- 172.21.15.43:0/1847399463 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e24004030 con 0x7f1e3415e4c0 2024-03-30T13:14:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e341520d0 con 0x7f1e3415e4c0 2024-03-30T13:14:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e2a7fc640 1 -- 172.21.15.43:0/1847399463 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e240b0040 con 0x7f1e3415e4c0 2024-03-30T13:14:32.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.518+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e34152690 con 0x7f1e3415e4c0 2024-03-30T13:14:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e2a7fc640 1 -- 172.21.15.43:0/1847399463 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1e240bd070 con 0x7f1e3415e4c0 2024-03-30T13:14:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e2a7fc640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1e20067fc0 0x7f1e2006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e2a7fc640 1 -- 172.21.15.43:0/1847399463 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f1e2411f960 con 0x7f1e3415e4c0 2024-03-30T13:14:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e39834640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1e20067fc0 0x7f1e2006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e39834640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1e20067fc0 0x7f1e2006a480 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e340ca940 tx=0x7f1e2c00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 Processor -- start 2024-03-30T13:14:32.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 -- start start 2024-03-30T13:14:32.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f673150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f629c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c73f477aa0 con 0x55c73f629830 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c73f3feaf0 con 0x55c73f672d70 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c73f473a50 con 0x55c73f673690 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f629c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f629c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33200/0 (socket says 172.21.15.43:33200) 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 -- 172.21.15.43:0/1060987759 learned_addr learned my addr 172.21.15.43:0/1060987759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e39834640 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f673150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e39033640 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 -- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 msgr2=0x55c73f6292f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6292f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 -- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 msgr2=0x55c73f673150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f673150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 -- 172.21.15.43:0/1060987759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73f3fcd00 con 0x55c73f629830 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e39834640 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f673150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.522+0000 7f1e3a035640 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f629c10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ae040 tx=0x7f1e240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bda601d0c5066ab server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/1060987759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e240ed940 con 0x55c73f629830 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/1060987759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e240edba0 con 0x55c73f629830 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/1060987759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e240ebf10 con 0x55c73f629830 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1060987759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 msgr2=0x55c73f629c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f629c10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1e240ae040 tx=0x7f1e240095c0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1060987759 shutdown_connections 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f629c10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1060987759 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f673150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1060987759 >> 172.21.15.43:0/1060987759 conn(0x55c73f6257e0 msgr2=0x55c73f627bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1060987759 shutdown_connections 2024-03-30T13:14:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1060987759 wait complete. 2024-03-30T13:14:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- start start 2024-03-30T13:14:32.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 Processor -- start 2024-03-30T13:14:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f62ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f657da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c73f43f270 con 0x55c73f629830 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c73f4356a0 con 0x55c73f672d70 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c73f40f880 con 0x55c73f673690 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 10 client.0 _ll_register_callbacks cb 0x55c73f614490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f62ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f62ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33216/0 (socket says 172.21.15.43:33216) 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 -- 172.21.15.43:0/1516455576 learned_addr learned my addr 172.21.15.43:0/1516455576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39033640 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f657da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3a035640 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 -- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 msgr2=0x55c73f6582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 -- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 msgr2=0x55c73f657da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f657da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 -- 172.21.15.43:0/1516455576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73f43c730 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39834640 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f62ec80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009ec0 tx=0x7f1e2c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71aafd9ddb36e683 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e18ff9640 1 -- 172.21.15.43:0/1516455576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c01d070 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e18ff9640 1 -- 172.21.15.43:0/1516455576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f1e2c023630 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e240edba0 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e18ff9640 1 -- 172.21.15.43:0/1516455576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1e2c030050 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 10 client.6495 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55c73f42fca0 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e18ff9640 1 -- 172.21.15.43:0/1516455576 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e240edba0 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e18ff9640 1 -- 172.21.15.43:0/1516455576 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f1e2c033020 con 0x55c73f629830 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 10 client.6495 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 10 client.6495 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 20 client.6495 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f67a5d0 0x55c73f67a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 4 client.6495 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73f412c90 con 0x55c73f67a5d0 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.526+0000 7f1e39033640 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f67a5d0 0x55c73f67a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e39033640 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f67a5d0 0x55c73f67a9e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fca28e4635e8ae0 server_cookie=9ddd5d006896911e in_seq=0 out_seq=0 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e18ff9640 10 client.6495 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e18ff9640 1 -- 172.21.15.43:0/1516455576 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55c73f412c90 con 0x55c73f67a5d0 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e18ff9640 10 client.6495 handle_command_reply: tid=0 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 client.6495 shutdown 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 msgr2=0x55c73f62ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f62ec80 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009ec0 tx=0x7f1e2c004ac0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 shutdown_connections 2024-03-30T13:14:32.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f67a5d0 0x55c73f67a9e0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f673690 0x55c73f6582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e18ff9640 0 client.6495 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f672d70 0x55c73f657da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/1516455576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f62ec80 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 >> 172.21.15.43:0/1516455576 conn(0x55c73f6257e0 msgr2=0x55c73f628460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 shutdown_connections 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/1516455576 wait complete. 2024-03-30T13:14:32.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.530+0000 7f1e3ec1a000 20 client.6495 trim_cache size 0 max 16384 2024-03-30T13:14:32.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 Processor -- start 2024-03-30T13:14:32.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- start start 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f656d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f657290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f673690 0x55c73f657a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34203ca0 con 0x55c73f629830 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34202cc0 con 0x55c73f673690 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1e34202120 con 0x55c73f672d70 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f656d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f656d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33224/0 (socket says 172.21.15.43:33224) 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 -- 172.21.15.43:0/686997388 learned_addr learned my addr 172.21.15.43:0/686997388 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3a035640 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f673690 0x55c73f657a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39033640 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f657290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 -- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 msgr2=0x55c73f657290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f657290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 -- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f673690 msgr2=0x55c73f657a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f673690 0x55c73f657a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 -- 172.21.15.43:0/686997388 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c73f3fed00 con 0x55c73f629830 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3a035640 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f673690 0x55c73f657a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39033640 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f657290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e39834640 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f656d50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009980 tx=0x7f1e2c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82375027f203b307 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e1affd640 1 -- 172.21.15.43:0/686997388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c01d070 con 0x55c73f629830 2024-03-30T13:14:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e1affd640 1 -- 172.21.15.43:0/686997388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e2c039070 con 0x55c73f629830 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e1affd640 1 -- 172.21.15.43:0/686997388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c023ad0 con 0x55c73f629830 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/686997388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 msgr2=0x55c73f656d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f656d50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009980 tx=0x7f1e2c018000 comp rx=0 tx=0).stop 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/686997388 shutdown_connections 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f673690 0x55c73f657a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f657290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/686997388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f629830 0x55c73f656d50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/686997388 >> 172.21.15.43:0/686997388 conn(0x55c73f6257e0 msgr2=0x55c73f689680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/686997388 shutdown_connections 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/686997388 wait complete. 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 -- start start 2024-03-30T13:14:32.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.546+0000 7f1e3ec1a000 1 Processor -- start 2024-03-30T13:14:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f629830 0x55c73f628fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f676ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f673690 0x55c73f677010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c73f43f270 con 0x55c73f673690 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1e34202670 con 0x55c73f629830 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c73f40f880 con 0x55c73f672d70 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 10 client.0 _ll_register_callbacks cb 0x7f1e2c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f673690 0x55c73f677010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f673690 0x55c73f677010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33226/0 (socket says 172.21.15.43:33226) 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 -- 172.21.15.43:0/480158742 learned_addr learned my addr 172.21.15.43:0/480158742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e39834640 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f629830 0x55c73f628fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e39033640 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f676ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 -- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 msgr2=0x55c73f676ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f676ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 -- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f629830 msgr2=0x55c73f628fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f629830 0x55c73f628fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 -- 172.21.15.43:0/480158742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c73f4f8f70 con 0x55c73f673690 2024-03-30T13:14:32.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3a035640 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f673690 0x55c73f677010 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1e24097e10 tx=0x7f1e2401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354e51b07c71a87f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 1 -- 172.21.15.43:0/480158742 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e240ef670 con 0x55c73f673690 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 1 -- 172.21.15.43:0/480158742 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f1e240043e0 con 0x55c73f673690 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e2c039070 con 0x55c73f673690 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 1 -- 172.21.15.43:0/480158742 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1e2410c050 con 0x55c73f673690 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 10 client.6501 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55c73f41e760 con 0x55c73f673690 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 1 -- 172.21.15.43:0/480158742 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e2c039070 con 0x55c73f673690 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 1 -- 172.21.15.43:0/480158742 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f1e240b3050 con 0x55c73f673690 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 10 client.6501 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 10 client.6501 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 20 client.6501 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f6e4bc0 0x55c73f6e4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 4 client.6501 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628399}] 2024-03-30T13:14:32.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628399}) -- 0x55c73f484fe0 con 0x55c73f6e4bc0 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e39834640 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f6e4bc0 0x55c73f6e4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e39834640 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f6e4bc0 0x55c73f6e4fa0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9be12f7aca6615fe server_cookie=804715cbf4b5be72 in_seq=0 out_seq=0 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 10 client.6501 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 1 -- 172.21.15.43:0/480158742 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3794 (crc 0 0 0) 0x55c73f484fe0 con 0x55c73f6e4bc0 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 10 client.6501 handle_command_reply: tid=0 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 client.6501 shutdown 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f673690 msgr2=0x55c73f677010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f673690 0x55c73f677010 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1e24097e10 tx=0x7f1e2401c540 comp rx=0 tx=0).stop 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 shutdown_connections 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c73f6e4bc0 0x55c73f6e4fa0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c73f673690 0x55c73f677010 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c73f672d70 0x55c73f676ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e18ff9640 0 client.6501 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 --2- 172.21.15.43:0/480158742 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c73f629830 0x55c73f628fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 >> 172.21.15.43:0/480158742 conn(0x55c73f6257e0 msgr2=0x55c73f689f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 shutdown_connections 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 1 -- 172.21.15.43:0/480158742 wait complete. 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.550+0000 7f1e3ec1a000 20 client.6501 trim_cache size 0 max 16384 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/symlink.c", 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628399, 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.583333+0000", 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.739149+0000", 2024-03-30T13:14:32.562 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:32.563 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 916, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.739149+0000", 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:32.564 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 916, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.565 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.583333+0000" 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 916, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.583333+0000" 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 162, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:32.566 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 28, 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:32.567 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:32.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.568 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:32.568 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:32.568 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.569 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.570 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:32.571 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:32.572 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:32.573 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:32.574 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1e20067fc0 msgr2=0x7f1e2006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1e20067fc0 0x7f1e2006a480 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1e340ca940 tx=0x7f1e2c00b040 comp rx=0 tx=0).stop 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e3415e4c0 msgr2=0x7f1e34151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e3415e4c0 0x7f1e34151510 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1e2401abf0 tx=0x7f1e2401ad80 comp rx=0 tx=0).stop 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 shutdown_connections 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1e20067fc0 0x7f1e2006a480 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e3415e4c0 0x7f1e34151510 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1e340cf6c0 0x7f1e34158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 --2- 172.21.15.43:0/1847399463 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1e3406c070 0x7f1e34153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 >> 172.21.15.43:0/1847399463 conn(0x7f1e340b6410 msgr2=0x7f1e340cec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 shutdown_connections 2024-03-30T13:14:32.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.554+0000 7f1e3ba98640 1 -- 172.21.15.43:0/1847399463 wait complete. 2024-03-30T13:14:32.577 DEBUG:tasks.cephfs.test_uninlining:getting inode info #11 of 23:1099511628389 2024-03-30T13:14:32.577 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628389 2024-03-30T13:14:32.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628389 2024-03-30T13:14:32.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792cbf7640 1 Processor -- start 2024-03-30T13:14:32.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792cbf7640 1 -- start start 2024-03-30T13:14:32.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792cbf7640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f792815e4c0 0x7f79280b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792cbf7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f792806c070 0x7f79280cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792cbf7640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f792806c850 con 0x7f79280cf4f0 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792cbf7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f792815e8a0 con 0x7f792806c070 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792cbf7640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f79280cf920 con 0x7f792815e4c0 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f791ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f792806c070 0x7f79280cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f791ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f792806c070 0x7f79280cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33242/0 (socket says 172.21.15.43:33242) 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f791ffff640 1 -- 172.21.15.43:0/3125706308 learned_addr learned my addr 172.21.15.43:0/3125706308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f792659c640 1 --2- 172.21.15.43:0/3125706308 >> v2:172.21.15.113:3300/0 conn(0x7f792815e4c0 0x7f79280b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f791ffff640 1 -- 172.21.15.43:0/3125706308 >> v2:172.21.15.113:3300/0 conn(0x7f792815e4c0 msgr2=0x7f79280b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f791ffff640 1 --2- 172.21.15.43:0/3125706308 >> v2:172.21.15.113:3300/0 conn(0x7f792815e4c0 0x7f79280b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f791ffff640 1 -- 172.21.15.43:0/3125706308 >> v1:172.21.15.113:6789/0 conn(0x7f79280cf4f0 legacy=0x7f7928161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.686+0000 7f791ffff640 1 -- 172.21.15.43:0/3125706308 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792806cb30 con 0x7f792806c070 2024-03-30T13:14:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f791ffff640 1 --2- 172.21.15.43:0/3125706308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792806c070 0x7f79280cefb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f79280d0cc0 tx=0x7f791001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e17da9cc7e6357 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f7925d9b640 1 -- 172.21.15.43:0/3125706308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100b0070 con 0x7f792806c070 2024-03-30T13:14:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f7925d9b640 1 -- 172.21.15.43:0/3125706308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79100ab600 con 0x7f792806c070 2024-03-30T13:14:32.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f7925d9b640 1 -- 172.21.15.43:0/3125706308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100ab9d0 con 0x7f792806c070 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- 172.21.15.43:0/3125706308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792806c070 msgr2=0x7f79280cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3125706308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792806c070 0x7f79280cefb0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f79280d0cc0 tx=0x7f791001c520 comp rx=0 tx=0).stop 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- 172.21.15.43:0/3125706308 shutdown_connections 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3125706308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f792806c070 0x7f79280cefb0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3125706308 >> v2:172.21.15.113:3300/0 conn(0x7f792815e4c0 0x7f79280b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- 172.21.15.43:0/3125706308 >> 172.21.15.43:0/3125706308 conn(0x7f79280b6410 msgr2=0x7f79280d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- 172.21.15.43:0/3125706308 shutdown_connections 2024-03-30T13:14:32.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- 172.21.15.43:0/3125706308 wait complete. 2024-03-30T13:14:32.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 Processor -- start 2024-03-30T13:14:32.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- start start 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f792806c070 0x7f792815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79280cf4f0 0x7f7928156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f792815e4c0 0x7f792814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f792816b3d0 con 0x7f79280cf4f0 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f792816b250 con 0x7f792815e4c0 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792cbf7640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f792816b550 con 0x7f792806c070 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f791ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79280cf4f0 0x7f7928156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f791ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79280cf4f0 0x7f7928156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33258/0 (socket says 172.21.15.43:33258) 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f791ffff640 1 -- 172.21.15.43:0/3552261183 learned_addr learned my addr 172.21.15.43:0/3552261183 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f7926d9d640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f792815e4c0 0x7f792814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.690+0000 7f792659c640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f792806c070 0x7f792815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791ffff640 1 -- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f792806c070 msgr2=0x7f792815d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791ffff640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f792806c070 0x7f792815d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791ffff640 1 -- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f792815e4c0 msgr2=0x7f792814cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791ffff640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f792815e4c0 0x7f792814cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791ffff640 1 -- 172.21.15.43:0/3552261183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7928161010 con 0x7f79280cf4f0 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f7926d9d640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f792815e4c0 0x7f792814cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791ffff640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79280cf4f0 0x7f7928156000 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f79100ae040 tx=0x7f79100192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791effd640 1 -- 172.21.15.43:0/3552261183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100b8020 con 0x7f79280cf4f0 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791effd640 1 -- 172.21.15.43:0/3552261183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79100b4410 con 0x7f79280cf4f0 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792814d050 con 0x7f79280cf4f0 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791effd640 1 -- 172.21.15.43:0/3552261183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100b0040 con 0x7f79280cf4f0 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7928161590 con 0x7f79280cf4f0 2024-03-30T13:14:32.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791effd640 1 -- 172.21.15.43:0/3552261183 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f79100bb020 con 0x7f79280cf4f0 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791effd640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f790c067ff0 0x7f790c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f791effd640 1 -- 172.21.15.43:0/3552261183 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f791011fde0 con 0x7f79280cf4f0 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792659c640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f790c067ff0 0x7f790c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 Processor -- start 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 -- start start 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792659c640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f790c067ff0 0x7f790c06a4b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f79280b53d0 tx=0x7f7918008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bd7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058b8d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058b8dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650589dbaa0 con 0x565058bd6d70 2024-03-30T13:14:32.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x565058962af0 con 0x565058b8d830 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5650589d7a50 con 0x565058bd7690 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.694+0000 7f792659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bd7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bd7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33260/0 (socket says 172.21.15.43:33260) 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 -- 172.21.15.43:0/3149824586 learned_addr learned my addr 172.21.15.43:0/3149824586 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f7926d9d640 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058b8dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058b8d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 -- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 msgr2=0x565058b8d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058b8d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 -- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 msgr2=0x565058b8dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058b8dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 -- 172.21.15.43:0/3149824586 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565058960d00 con 0x565058bd6d70 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bd7150 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7918006a80 tx=0x7f7918014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ec2798d087124a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f7917fff640 1 -- 172.21.15.43:0/3149824586 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918011ce0 con 0x565058bd6d70 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f7917fff640 1 -- 172.21.15.43:0/3149824586 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7918026070 con 0x565058bd6d70 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f7917fff640 1 -- 172.21.15.43:0/3149824586 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7918021640 con 0x565058bd6d70 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/3149824586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 msgr2=0x565058bd7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bd7150 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7918006a80 tx=0x7f7918014000 comp rx=0 tx=0).stop 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/3149824586 shutdown_connections 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058b8dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058b8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- 172.21.15.43:0/3149824586 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bd7150 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/3149824586 >> 172.21.15.43:0/3149824586 conn(0x565058b897e0 msgr2=0x565058b8bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/3149824586 shutdown_connections 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/3149824586 wait complete. 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- start start 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 Processor -- start 2024-03-30T13:14:32.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058b8d830 0x565058b92c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bbbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058bd7690 0x565058bbc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650589a3270 con 0x565058bd6d70 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5650589996a0 con 0x565058bd7690 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x565058973880 con 0x565058b8d830 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 10 client.0 _ll_register_callbacks cb 0x565058b78490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bbbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bbbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33264/0 (socket says 172.21.15.43:33264) 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 -- 172.21.15.43:0/1923181516 learned_addr learned my addr 172.21.15.43:0/1923181516 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792659c640 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058b8d830 0x565058b92c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f7926d9d640 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058bd7690 0x565058bbc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 -- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058b8d830 msgr2=0x565058b92c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058b8d830 0x565058b92c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 -- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058bd7690 msgr2=0x565058bbc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058bd7690 0x565058bbc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 -- 172.21.15.43:0/1923181516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650589a0730 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f791ffff640 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bbbda0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f791001abd0 tx=0x7f79100ef720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c723c5655e5d1af3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f79157fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f79157fa640 1 -- 172.21.15.43:0/1923181516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100f0160 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f79157fa640 1 -- 172.21.15.43:0/1923181516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f79100f8450 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7918026070 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f79157fa640 1 -- 172.21.15.43:0/1923181516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79100f40b0 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 10 client.6513 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x565058993ca0 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f79157fa640 1 -- 172.21.15.43:0/1923181516 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f791011d030 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f79157fa640 1 -- 172.21.15.43:0/1923181516 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f79100b3050 con 0x565058bd6d70 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 10 client.6513 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 10 client.6513 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 20 client.6513 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bde5d0 0x565058bde9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 4 client.6513 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.698+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565058976c90 con 0x565058bde5d0 2024-03-30T13:14:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792659c640 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bde5d0 0x565058bde9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f79157fa640 10 client.6513 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792659c640 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bde5d0 0x565058bde9b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21cf77fdf0600dc1 server_cookie=96cef3a87e840e71 in_seq=0 out_seq=0 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f79157fa640 1 -- 172.21.15.43:0/1923181516 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x565058976c90 con 0x565058bde5d0 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f79157fa640 10 client.6513 handle_command_reply: tid=0 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 client.6513 shutdown 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 msgr2=0x565058bbbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bbbda0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f791001abd0 tx=0x7f79100ef720 comp rx=0 tx=0).stop 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 shutdown_connections 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bde5d0 0x565058bde9b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058bd7690 0x565058bbc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058bbbda0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f79157fa640 0 client.6513 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 --2- 172.21.15.43:0/1923181516 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058b8d830 0x565058b92c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 >> 172.21.15.43:0/1923181516 conn(0x565058b897e0 msgr2=0x565058b8c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 shutdown_connections 2024-03-30T13:14:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 1 -- 172.21.15.43:0/1923181516 wait complete. 2024-03-30T13:14:32.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.702+0000 7f792fd79000 20 client.6513 trim_cache size 0 max 16384 2024-03-30T13:14:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 Processor -- start 2024-03-30T13:14:32.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- start start 2024-03-30T13:14:32.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058bbc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd6d70 0x565058b8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd7690 0x565058b8c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7928204600 con 0x565058bd7690 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f79282035e0 con 0x565058b8d830 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7928202950 con 0x565058bd6d70 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd7690 0x565058b8c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd7690 0x565058b8c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33270/0 (socket says 172.21.15.43:33270) 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 -- 172.21.15.43:0/1653340110 learned_addr learned my addr 172.21.15.43:0/1653340110 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792659c640 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058bbc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f791ffff640 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd6d70 0x565058b8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 -- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd6d70 msgr2=0x565058b8bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd6d70 0x565058b8bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 -- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 msgr2=0x565058bbc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058bbc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 -- 172.21.15.43:0/1653340110 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565058962d00 con 0x565058bd7690 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f791ffff640 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd6d70 0x565058b8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f7926d9d640 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd7690 0x565058b8c110 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f79200051b0 tx=0x7f7920003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b10a206f5ba2117 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f79177fe640 1 -- 172.21.15.43:0/1653340110 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920006b60 con 0x565058bd7690 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f79177fe640 1 -- 172.21.15.43:0/1653340110 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7920006d00 con 0x565058bd7690 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f79177fe640 1 -- 172.21.15.43:0/1653340110 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7920001c10 con 0x565058bd7690 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- 172.21.15.43:0/1653340110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd7690 msgr2=0x565058b8c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd7690 0x565058b8c110 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f79200051b0 tx=0x7f7920003f90 comp rx=0 tx=0).stop 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- 172.21.15.43:0/1653340110 shutdown_connections 2024-03-30T13:14:32.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd7690 0x565058b8c110 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd6d70 0x565058b8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- 172.21.15.43:0/1653340110 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058bbc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- 172.21.15.43:0/1653340110 >> 172.21.15.43:0/1653340110 conn(0x565058b897e0 msgr2=0x565058bedf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- 172.21.15.43:0/1653340110 shutdown_connections 2024-03-30T13:14:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- 172.21.15.43:0/1653340110 wait complete. 2024-03-30T13:14:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- start start 2024-03-30T13:14:32.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 Processor -- start 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058c48fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058c49500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058bdaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5650589996a0 con 0x565058bd6d70 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7928202f50 con 0x565058b8d830 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5650589ddaf0 con 0x565058bd7690 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.718+0000 7f792fd79000 10 client.0 _ll_register_callbacks cb 0x7f79100a4f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058c49500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058c49500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33284/0 (socket says 172.21.15.43:33284) 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 -- 172.21.15.43:0/531943973 learned_addr learned my addr 172.21.15.43:0/531943973 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792659c640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058c48fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f7926d9d640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058bdaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 -- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 msgr2=0x565058bdaad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058bdaad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 -- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 msgr2=0x565058c48fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058c48fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 -- 172.21.15.43:0/531943973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565058a5cf70 con 0x565058bd6d70 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f7926d9d640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058bdaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f791ffff640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058c49500 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f79100ec680 tx=0x7f791011e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e24fc09c3ab7078 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 1 -- 172.21.15.43:0/531943973 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100ef840 con 0x565058bd6d70 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 1 -- 172.21.15.43:0/531943973 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f7910105df0 con 0x565058bd6d70 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7920006d00 con 0x565058bd6d70 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 1 -- 172.21.15.43:0/531943973 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f79100f62b0 con 0x565058bd6d70 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 10 client.6519 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x565058982760 con 0x565058bd6d70 2024-03-30T13:14:32.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 1 -- 172.21.15.43:0/531943973 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7920006d00 con 0x565058bd6d70 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 1 -- 172.21.15.43:0/531943973 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f7910109020 con 0x565058bd6d70 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 10 client.6519 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 10 client.6519 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 20 client.6519 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bdb490 0x565058bdb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 4 client.6519 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628389}] 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628389}) -- 0x5650589e8fe0 con 0x565058bdb490 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792659c640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bdb490 0x565058bdb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 10 client.6519 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792659c640 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bdb490 0x565058bdb870 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5df552fc32629977 server_cookie=830bbb0fe21f52f1 in_seq=0 out_seq=0 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 1 -- 172.21.15.43:0/531943973 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3792 (crc 0 0 0) 0x5650589e8fe0 con 0x565058bdb490 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 10 client.6519 handle_command_reply: tid=0 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 client.6519 shutdown 2024-03-30T13:14:32.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 msgr2=0x565058c49500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058c49500 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f79100ec680 tx=0x7f791011e050 comp rx=0 tx=0).stop 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 shutdown_connections 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x565058bdb490 0x565058bdb870 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x565058bd7690 0x565058bdaad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f79157fa640 0 client.6519 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x565058bd6d70 0x565058c49500 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 --2- 172.21.15.43:0/531943973 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x565058b8d830 0x565058c48fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 >> 172.21.15.43:0/531943973 conn(0x565058b897e0 msgr2=0x565058bee620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 shutdown_connections 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 1 -- 172.21.15.43:0/531943973 wait complete. 2024-03-30T13:14:32.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792fd79000 20 client.6519 trim_cache size 0 max 16384 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/acl.h", 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628389, 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.593839+0000", 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.662219+0000", 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:32.734 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1502, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:32.735 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.662219+0000", 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:32.736 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1502, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.593839+0000" 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1502, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.737 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.593839+0000" 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 164, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 8, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:32.738 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:32.739 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:32.740 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.741 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.742 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.743 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:32.744 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:32.745 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:32.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f790c067ff0 msgr2=0x7f790c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f790c067ff0 0x7f790c06a4b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f79280b53d0 tx=0x7f7918008040 comp rx=0 tx=0).stop 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79280cf4f0 msgr2=0x7f7928156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79280cf4f0 0x7f7928156000 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f79100ae040 tx=0x7f79100192c0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 shutdown_connections 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f790c067ff0 0x7f790c06a4b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f792815e4c0 0x7f792814cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79280cf4f0 0x7f7928156000 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 --2- 172.21.15.43:0/3552261183 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f792806c070 0x7f792815d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 >> 172.21.15.43:0/3552261183 conn(0x7f79280b6410 msgr2=0x7f792815a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 shutdown_connections 2024-03-30T13:14:32.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.722+0000 7f792cbf7640 1 -- 172.21.15.43:0/3552261183 wait complete. 2024-03-30T13:14:32.748 DEBUG:tasks.cephfs.test_uninlining:getting inode info #12 of 23:1099511628403 2024-03-30T13:14:32.748 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628403 2024-03-30T13:14:32.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628403 2024-03-30T13:14:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.854+0000 7f9ef8b83640 1 Processor -- start 2024-03-30T13:14:32.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.854+0000 7f9ef8b83640 1 -- start start 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ef413f140 0x7f9ef41606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9ef415ff40 0x7f9ef406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ef406d710 con 0x7f9ef406cbb0 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef406e320 con 0x7f9ef413f140 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef406d9f0 con 0x7f9ef415ff40 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9ef406cbb0 0x7f9ef4141a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35770/0 (socket says 172.21.15.43:35770) 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ef413f140 0x7f9ef41606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef2d9d640 1 -- 172.21.15.43:0/1484399661 learned_addr learned my addr 172.21.15.43:0/1484399661 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9eebfff640 1 --2- 172.21.15.43:0/1484399661 >> v2:172.21.15.113:3300/0 conn(0x7f9ef415ff40 0x7f9ef406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef1d9b640 1 -- 172.21.15.43:0/1484399661 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1741974164 0 0) 0x7f9ef406d710 con 0x7f9ef406cbb0 2024-03-30T13:14:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef1d9b640 1 -- 172.21.15.43:0/1484399661 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ee0003530 con 0x7f9ef406cbb0 2024-03-30T13:14:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef259c640 1 -- 172.21.15.43:0/1484399661 >> v2:172.21.15.113:3300/0 conn(0x7f9ef415ff40 msgr2=0x7f9ef406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef259c640 1 --2- 172.21.15.43:0/1484399661 >> v2:172.21.15.113:3300/0 conn(0x7f9ef415ff40 0x7f9ef406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef259c640 1 -- 172.21.15.43:0/1484399661 >> v1:172.21.15.43:6789/0 conn(0x7f9ef406cbb0 legacy=0x7f9ef4141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef259c640 1 -- 172.21.15.43:0/1484399661 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef406dcd0 con 0x7f9ef413f140 2024-03-30T13:14:32.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef259c640 1 --2- 172.21.15.43:0/1484399661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef413f140 0x7f9ef41606d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4153930 tx=0x7f9ee401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fadb1844a458ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef1d9b640 1 -- 172.21.15.43:0/1484399661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40b1070 con 0x7f9ef413f140 2024-03-30T13:14:32.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef1d9b640 1 -- 172.21.15.43:0/1484399661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee40ac5f0 con 0x7f9ef413f140 2024-03-30T13:14:32.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef1d9b640 1 -- 172.21.15.43:0/1484399661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40ac9c0 con 0x7f9ef413f140 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 -- 172.21.15.43:0/1484399661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef413f140 msgr2=0x7f9ef41606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/1484399661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef413f140 0x7f9ef41606d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4153930 tx=0x7f9ee401c480 comp rx=0 tx=0).stop 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 -- 172.21.15.43:0/1484399661 shutdown_connections 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/1484399661 >> v2:172.21.15.113:3300/0 conn(0x7f9ef415ff40 0x7f9ef406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/1484399661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef413f140 0x7f9ef41606d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 -- 172.21.15.43:0/1484399661 >> 172.21.15.43:0/1484399661 conn(0x7f9ef40d00a0 msgr2=0x7f9ef40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.858+0000 7f9ef8b83640 1 -- 172.21.15.43:0/1484399661 shutdown_connections 2024-03-30T13:14:32.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 -- 172.21.15.43:0/1484399661 wait complete. 2024-03-30T13:14:32.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 Processor -- start 2024-03-30T13:14:32.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 -- start start 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ef406cbb0 0x7f9ef40b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ef413f140 0x7f9ef415b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef415ff40 0x7f9ef415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4169170 con 0x7f9ef415ff40 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4168ff0 con 0x7f9ef406cbb0 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef41692f0 con 0x7f9ef413f140 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef415ff40 0x7f9ef415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef415ff40 0x7f9ef415b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33296/0 (socket says 172.21.15.43:33296) 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 -- 172.21.15.43:0/3446480304 learned_addr learned my addr 172.21.15.43:0/3446480304 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef259c640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ef406cbb0 0x7f9ef40b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9eebfff640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ef413f140 0x7f9ef415b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 -- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ef413f140 msgr2=0x7f9ef415b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ef413f140 0x7f9ef415b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 -- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ef406cbb0 msgr2=0x7f9ef40b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ef406cbb0 0x7f9ef40b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 -- 172.21.15.43:0/3446480304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef4156420 con 0x7f9ef415ff40 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef2d9d640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef415ff40 0x7f9ef415b880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00ca20 tx=0x7f9eec00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9eeaffd640 1 -- 172.21.15.43:0/3446480304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec017070 con 0x7f9ef415ff40 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9eeaffd640 1 -- 172.21.15.43:0/3446480304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec00abd0 con 0x7f9ef415ff40 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef4156700 con 0x7f9ef415ff40 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9eeaffd640 1 -- 172.21.15.43:0/3446480304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec012400 con 0x7f9ef415ff40 2024-03-30T13:14:32.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.862+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef414dc20 con 0x7f9ef415ff40 2024-03-30T13:14:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9eeaffd640 1 -- 172.21.15.43:0/3446480304 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9eec01a070 con 0x7f9ef415ff40 2024-03-30T13:14:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9eeaffd640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ed4067fc0 0x7f9ed406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9eeaffd640 1 -- 172.21.15.43:0/3446480304 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f9eec0864a0 con 0x7f9ef415ff40 2024-03-30T13:14:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ed4067fc0 0x7f9ed406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ed4067fc0 0x7f9ed406a480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40ae000 tx=0x7f9ee4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 Processor -- start 2024-03-30T13:14:32.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- start start 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c1964a6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a6690 0x55c19645c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c19645c830 0x55c19645cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c1962aaaa0 con 0x55c1964a5d70 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c196231af0 con 0x55c1964a6690 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c1962a6a50 con 0x55c19645c830 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c1964a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c1964a6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33300/0 (socket says 172.21.15.43:33300) 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 -- 172.21.15.43:0/4188785595 learned_addr learned my addr 172.21.15.43:0/4188785595 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9eebfff640 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a6690 0x55c19645c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef2d9d640 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c19645c830 0x55c19645cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 -- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c19645c830 msgr2=0x55c19645cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c19645c830 0x55c19645cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 -- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a6690 msgr2=0x55c19645c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a6690 0x55c19645c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 -- 172.21.15.43:0/4188785595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19622fd00 con 0x55c1964a5d70 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ef259c640 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c1964a6150 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40af040 tx=0x7f9ee40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492e6365e4473f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ed3fff640 1 -- 172.21.15.43:0/4188785595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40c8070 con 0x55c1964a5d70 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ed3fff640 1 -- 172.21.15.43:0/4188785595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee40b2040 con 0x55c1964a5d70 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9ed3fff640 1 -- 172.21.15.43:0/4188785595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40b89f0 con 0x55c1964a5d70 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- 172.21.15.43:0/4188785595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 msgr2=0x55c1964a6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c1964a6150 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40af040 tx=0x7f9ee40a7000 comp rx=0 tx=0).stop 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- 172.21.15.43:0/4188785595 shutdown_connections 2024-03-30T13:14:32.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c19645c830 0x55c19645cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a6690 0x55c19645c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 --2- 172.21.15.43:0/4188785595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c1964a6150 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- 172.21.15.43:0/4188785595 >> 172.21.15.43:0/4188785595 conn(0x55c1964587e0 msgr2=0x55c19645abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- 172.21.15.43:0/4188785595 shutdown_connections 2024-03-30T13:14:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.866+0000 7f9efbd05000 1 -- 172.21.15.43:0/4188785595 wait complete. 2024-03-30T13:14:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 -- start start 2024-03-30T13:14:32.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 Processor -- start 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c19645c830 0x55c196461c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c19648ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c19648b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c196272270 con 0x55c1964a5d70 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55c1962686a0 con 0x55c19645c830 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c196242880 con 0x55c1964a6690 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 10 client.0 _ll_register_callbacks cb 0x55c196447490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c19648ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c19648ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33308/0 (socket says 172.21.15.43:33308) 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 -- 172.21.15.43:0/56272249 learned_addr learned my addr 172.21.15.43:0/56272249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ef259c640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c19645c830 0x55c196461c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ef2d9d640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c19648b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 -- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 msgr2=0x55c19648b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c19648b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 -- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c19645c830 msgr2=0x55c196461c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c19645c830 0x55c196461c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 -- 172.21.15.43:0/56272249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19626f730 con 0x55c1964a5d70 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ef259c640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c19645c830 0x55c196461c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9eebfff640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c19648ada0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9edc012580 tx=0x7f9edc012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1ca98dab129f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ed17fa640 1 -- 172.21.15.43:0/56272249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc004440 con 0x55c1964a5d70 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ed17fa640 1 -- 172.21.15.43:0/56272249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f9edc003620 con 0x55c1964a5d70 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee40b2040 con 0x55c1964a5d70 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ed17fa640 1 -- 172.21.15.43:0/56272249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9edc025050 con 0x55c1964a5d70 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 10 client.6534 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55c196262ca0 con 0x55c1964a5d70 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ed17fa640 1 -- 172.21.15.43:0/56272249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee40b2040 con 0x55c1964a5d70 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ed17fa640 1 -- 172.21.15.43:0/56272249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f9edc028020 con 0x55c1964a5d70 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 10 client.6534 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 10 client.6534 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 20 client.6534 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964ad5d0 0x55c1964ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 4 client.6534 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c196245c90 con 0x55c1964ad5d0 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ef259c640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964ad5d0 0x55c1964ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ef259c640 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964ad5d0 0x55c1964ad9e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39bff0d969bba9e8 server_cookie=f43c2f1de585c555 in_seq=0 out_seq=0 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.870+0000 7f9ed17fa640 10 client.6534 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9ed17fa640 1 -- 172.21.15.43:0/56272249 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55c196245c90 con 0x55c1964ad5d0 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9ed17fa640 10 client.6534 handle_command_reply: tid=0 2024-03-30T13:14:32.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 client.6534 shutdown 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 msgr2=0x55c19648ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c19648ada0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9edc012580 tx=0x7f9edc012a40 comp rx=0 tx=0).stop 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 shutdown_connections 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964ad5d0 0x55c1964ad9e0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c19648b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1964a5d70 0x55c19648ada0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9ed17fa640 0 client.6534 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 --2- 172.21.15.43:0/56272249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c19645c830 0x55c196461c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 >> 172.21.15.43:0/56272249 conn(0x55c1964587e0 msgr2=0x55c19645b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 shutdown_connections 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 1 -- 172.21.15.43:0/56272249 wait complete. 2024-03-30T13:14:32.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.874+0000 7f9efbd05000 20 client.6534 trim_cache size 0 max 16384 2024-03-30T13:14:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.886+0000 7f9efbd05000 1 Processor -- start 2024-03-30T13:14:32.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.886+0000 7f9efbd05000 1 -- start start 2024-03-30T13:14:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c19648b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c1964a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4204400 con 0x55c19645c830 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4203400 con 0x55c1964a5d70 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef414f090 con 0x55c1964a6690 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c19648b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c19648b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33322/0 (socket says 172.21.15.43:33322) 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/2320740091 learned_addr learned my addr 172.21.15.43:0/2320740091 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9eebfff640 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c1964a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef2d9d640 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 msgr2=0x55c1964a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 msgr2=0x55c1964a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c1964a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/2320740091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c196231d00 con 0x55c19645c830 2024-03-30T13:14:32.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9eebfff640 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c1964a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c19648b180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40af040 tx=0x7f9ee4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d25016ebc798d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed37fe640 1 -- 172.21.15.43:0/2320740091 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40c8040 con 0x55c19645c830 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed37fe640 1 -- 172.21.15.43:0/2320740091 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee40d4e50 con 0x55c19645c830 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed37fe640 1 -- 172.21.15.43:0/2320740091 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40d45d0 con 0x55c19645c830 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- 172.21.15.43:0/2320740091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 msgr2=0x55c19648b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c19648b180 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40af040 tx=0x7f9ee4004740 comp rx=0 tx=0).stop 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- 172.21.15.43:0/2320740091 shutdown_connections 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c1964a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- 172.21.15.43:0/2320740091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c19648b180 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- 172.21.15.43:0/2320740091 >> 172.21.15.43:0/2320740091 conn(0x55c1964587e0 msgr2=0x55c1964bcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- 172.21.15.43:0/2320740091 shutdown_connections 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- 172.21.15.43:0/2320740091 wait complete. 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- start start 2024-03-30T13:14:32.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 Processor -- start 2024-03-30T13:14:32.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c196517ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c196518400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55c196366b00 con 0x55c19645c830 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4202db0 con 0x55c1964a5d70 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55c196272270 con 0x55c1964a6690 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 10 client.0 _ll_register_callbacks cb 0x7f9edc0094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c196517ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c196517ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33330/0 (socket says 172.21.15.43:33330) 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/1363507187 learned_addr learned my addr 172.21.15.43:0/1363507187 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9eebfff640 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c196518400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef2d9d640 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 msgr2=0x55c1964a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 msgr2=0x55c196518400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c196518400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 -- 172.21.15.43:0/1363507187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19632bf70 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ef259c640 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c196517ec0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40b8fd0 tx=0x7f9ee401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860333e2e07c3bfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed17fa640 1 -- 172.21.15.43:0/1363507187 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee40c8040 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee40d4e50 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed17fa640 1 -- 172.21.15.43:0/1363507187 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f9ee40d43d0 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed17fa640 1 -- 172.21.15.43:0/1363507187 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9ee40d45d0 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 10 client.6540 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55c196251760 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.890+0000 7f9ed17fa640 1 -- 172.21.15.43:0/1363507187 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee40d4e50 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ed17fa640 1 -- 172.21.15.43:0/1363507187 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f9ee40c2070 con 0x55c19645c830 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 10 client.6540 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:32.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 10 client.6540 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 20 client.6540 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964aa490 0x55c1964aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 4 client.6540 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628403}] 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628403}) -- 0x55c1962b7fe0 con 0x55c1964aa490 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9eebfff640 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964aa490 0x55c1964aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ed17fa640 10 client.6540 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9eebfff640 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964aa490 0x55c1964aa8a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c204552a4f8318 server_cookie=719bea4a9ff795d0 in_seq=0 out_seq=0 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ed17fa640 1 -- 172.21.15.43:0/1363507187 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3803 (crc 0 0 0) 0x55c1962b7fe0 con 0x55c1964aa490 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ed17fa640 10 client.6540 handle_command_reply: tid=0 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 client.6540 shutdown 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 msgr2=0x55c196517ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c196517ec0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40b8fd0 tx=0x7f9ee401acc0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 shutdown_connections 2024-03-30T13:14:32.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55c1964aa490 0x55c1964aa8a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55c1964a6690 0x55c1964a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ed17fa640 0 client.6540 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55c1964a5d70 0x55c196518400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 --2- 172.21.15.43:0/1363507187 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c19645c830 0x55c196517ec0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 >> 172.21.15.43:0/1363507187 conn(0x55c1964587e0 msgr2=0x55c1964bd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 shutdown_connections 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 1 -- 172.21.15.43:0/1363507187 wait complete. 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9efbd05000 20 client.6540 trim_cache size 0 max 16384 2024-03-30T13:14:32.904 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr_trusted.c", 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628403, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.596676+0000", 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.756505+0000", 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:32.905 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:32.906 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 1094, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.756505+0000", 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:32.907 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1094, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.596676+0000" 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:32.908 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 1094, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.596676+0000" 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 166, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 36, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:32.909 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:32.910 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:32.911 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.912 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:32.913 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:32.914 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:32.915 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:32.916 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ed4067fc0 msgr2=0x7f9ed406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ed4067fc0 0x7f9ed406a480 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9ee40ae000 tx=0x7f9ee4008040 comp rx=0 tx=0).stop 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef415ff40 msgr2=0x7f9ef415b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef415ff40 0x7f9ef415b880 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00ca20 tx=0x7f9eec00cee0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 shutdown_connections 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ed4067fc0 0x7f9ed406a480 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ef415ff40 0x7f9ef415b880 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ef413f140 0x7f9ef415b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 --2- 172.21.15.43:0/3446480304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ef406cbb0 0x7f9ef40b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 >> 172.21.15.43:0/3446480304 conn(0x7f9ef40d00a0 msgr2=0x7f9ef414ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 shutdown_connections 2024-03-30T13:14:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:32.894+0000 7f9ef8b83640 1 -- 172.21.15.43:0/3446480304 wait complete. 2024-03-30T13:14:32.919 DEBUG:tasks.cephfs.test_uninlining:getting inode info #13 of 23:1099511628401 2024-03-30T13:14:32.919 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628401 2024-03-30T13:14:32.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628401 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.030+0000 7f0e77db2640 1 Processor -- start 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.030+0000 7f0e77db2640 1 -- start start 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e77db2640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0e700b4870 0x7f0e7006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e77db2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 0x7f0e700ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e77db2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e700b3660 con 0x7f0e70144930 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e77db2640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0e700cdb90 con 0x7f0e700b4870 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e77db2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e700b3940 con 0x7f0e7006cbb0 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0e70144930 0x7f0e700b2e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35784/0 (socket says 172.21.15.43:35784) 2024-03-30T13:14:33.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 -- 172.21.15.43:0/1820637303 learned_addr learned my addr 172.21.15.43:0/1820637303 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e7534d640 1 --2- 172.21.15.43:0/1820637303 >> v2:172.21.15.113:3300/0 conn(0x7f0e700b4870 0x7f0e7006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e7634f640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 0x7f0e700ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044076215 0 0) 0x7f0e700b3660 con 0x7f0e70144930 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e5c003530 con 0x7f0e70144930 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 195943582 0 0) 0x7f0e5c003530 con 0x7f0e70144930 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 msgr2=0x7f0e700ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 0x7f0e700ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 >> v2:172.21.15.113:3300/0 conn(0x7f0e700b4870 msgr2=0x7f0e7006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 --2- 172.21.15.43:0/1820637303 >> v2:172.21.15.113:3300/0 conn(0x7f0e700b4870 0x7f0e7006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e700b3c20 con 0x7f0e70144930 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f0e6c001680 con 0x7f0e70144930 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 >> v1:172.21.15.43:6789/0 conn(0x7f0e70144930 legacy=0x7f0e700b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e5c0039c0 0x7f0e5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 0x7f0e5c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c004d50 0x7f0e5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700cdb90 con 0x7f0e5c0039c0 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e700b3940 con 0x7f0e5c004d50 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e5c00b9d0 con 0x7f0e7006cbb0 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e5c0039c0 0x7f0e5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e7634f640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c004d50 0x7f0e5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e7534d640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 0x7f0e5c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 -- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 msgr2=0x7f0e5c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e7006cbb0 0x7f0e5c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 -- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c004d50 msgr2=0x7f0e5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c004d50 0x7f0e5c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.034+0000 7f0e75b4e640 1 -- 172.21.15.43:0/1820637303 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5c003530 con 0x7f0e5c0039c0 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e5c0039c0 0x7f0e5c003de0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c210 tx=0x7f0e6c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaf8405f14effa0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c007d60 con 0x7f0e5c0039c0 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0a9020 con 0x7f0e5c0039c0 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e74b4c640 1 -- 172.21.15.43:0/1820637303 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c005b20 con 0x7f0e5c0039c0 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- 172.21.15.43:0/1820637303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e5c0039c0 msgr2=0x7f0e5c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 --2- 172.21.15.43:0/1820637303 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e5c0039c0 0x7f0e5c003de0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c210 tx=0x7f0e6c01cca0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e7634f640 1 -- 172.21.15.43:0/1820637303 reap_dead start 2024-03-30T13:14:33.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- 172.21.15.43:0/1820637303 shutdown_connections 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- 172.21.15.43:0/1820637303 >> 172.21.15.43:0/1820637303 conn(0x7f0e700d00a0 msgr2=0x7f0e700d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- 172.21.15.43:0/1820637303 shutdown_connections 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- 172.21.15.43:0/1820637303 wait complete. 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 Processor -- start 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- start start 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c0039c0 0x7f0e7013d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e70155930 0x7f0e70141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e701418c0 0x7f0e701405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7006e250 con 0x7f0e70155930 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7006e0d0 con 0x7f0e5c0039c0 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e77db2640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7006e3d0 con 0x7f0e701418c0 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e70155930 0x7f0e70141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e70155930 0x7f0e70141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33356/0 (socket says 172.21.15.43:33356) 2024-03-30T13:14:33.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e7534d640 1 -- 172.21.15.43:0/3006432886 learned_addr learned my addr 172.21.15.43:0/3006432886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c0039c0 0x7f0e7013d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.038+0000 7f0e7634f640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e701418c0 0x7f0e701405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7534d640 1 -- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e701418c0 msgr2=0x7f0e701405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e701418c0 0x7f0e701405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7534d640 1 -- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c0039c0 msgr2=0x7f0e7013d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c0039c0 0x7f0e7013d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7534d640 1 -- 172.21.15.43:0/3006432886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e70154890 con 0x7f0e70155930 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e70155930 0x7f0e70141380 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0e60011610 tx=0x7f0e60011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e667fc640 1 -- 172.21.15.43:0/3006432886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000f8f0 con 0x7f0e70155930 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e667fc640 1 -- 172.21.15.43:0/3006432886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e6000fa90 con 0x7f0e70155930 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e70154b70 con 0x7f0e70155930 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e667fc640 1 -- 172.21.15.43:0/3006432886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000fe40 con 0x7f0e70155930 2024-03-30T13:14:33.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e70155130 con 0x7f0e70155930 2024-03-30T13:14:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e667fc640 1 -- 172.21.15.43:0/3006432886 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0e60015460 con 0x7f0e70155930 2024-03-30T13:14:33.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e667fc640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e54067ff0 0x7f0e5406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e667fc640 1 -- 172.21.15.43:0/3006432886 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f0e600870d0 con 0x7f0e70155930 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e54067ff0 0x7f0e5406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7af34000 1 Processor -- start 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e7af34000 1 -- start start 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.042+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e54067ff0 0x7f0e5406a4b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0aa000 tx=0x7f0e6c000ff0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005dfc0 0x5572e005e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e0014700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e0014c40 0x5572e0015020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572dfe5eaa0 con 0x5572e0014c40 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5572dfde5af0 con 0x5572e005e8e0 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5572dfe5aa50 con 0x5572e005dfc0 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e0014c40 0x5572e0015020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e0014c40 0x5572e0015020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33368/0 (socket says 172.21.15.43:33368) 2024-03-30T13:14:33.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 -- 172.21.15.43:0/326593679 learned_addr learned my addr 172.21.15.43:0/326593679 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005dfc0 0x5572e005e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e0014700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 -- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005dfc0 msgr2=0x5572e005e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005dfc0 0x5572e005e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 -- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 msgr2=0x5572e0014700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e0014700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 -- 172.21.15.43:0/326593679 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572dfde3d00 con 0x5572e0014c40 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e0014c40 0x5572e0015020 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0e7006d230 tx=0x7f0e6800b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb85ecedbcf71457 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e537fe640 1 -- 172.21.15.43:0/326593679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6801d070 con 0x5572e0014c40 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e537fe640 1 -- 172.21.15.43:0/326593679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e68003ba0 con 0x5572e0014c40 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e537fe640 1 -- 172.21.15.43:0/326593679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e68004030 con 0x5572e0014c40 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- 172.21.15.43:0/326593679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e0014c40 msgr2=0x5572e0015020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e0014c40 0x5572e0015020 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f0e7006d230 tx=0x7f0e6800b750 comp rx=0 tx=0).stop 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- 172.21.15.43:0/326593679 shutdown_connections 2024-03-30T13:14:33.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e0014c40 0x5572e0015020 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e0014700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- 172.21.15.43:0/326593679 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005dfc0 0x5572e005e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- 172.21.15.43:0/326593679 >> 172.21.15.43:0/326593679 conn(0x5572e000c7e0 msgr2=0x5572e000ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- 172.21.15.43:0/326593679 shutdown_connections 2024-03-30T13:14:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- 172.21.15.43:0/326593679 wait complete. 2024-03-30T13:14:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- start start 2024-03-30T13:14:33.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 Processor -- start 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e005d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e001a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005e8e0 0x5572e001ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572dfe26270 con 0x5572e005dfc0 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5572dfe1c6a0 con 0x5572e0014c40 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5572dfdf6880 con 0x5572e005e8e0 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7af34000 10 client.0 _ll_register_callbacks cb 0x5572dfffb490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e001a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e001a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33378/0 (socket says 172.21.15.43:33378) 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 -- 172.21.15.43:0/3728026707 learned_addr learned my addr 172.21.15.43:0/3728026707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e005d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7634f640 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005e8e0 0x5572e001ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 -- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005e8e0 msgr2=0x5572e001ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005e8e0 0x5572e001ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 -- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 msgr2=0x5572e005d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e005d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 -- 172.21.15.43:0/3728026707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572dfe23730 con 0x5572e005dfc0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e001a910 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0e60056bc0 tx=0x7f0e60056f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17ac59d00a02fce server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.046+0000 7f0e50ff9640 1 -- 172.21.15.43:0/3728026707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60083060 con 0x5572e005dfc0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e68003ba0 con 0x5572e005dfc0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 1 -- 172.21.15.43:0/3728026707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 6247+0+0 (secure 0 0 0) 0x7f0e6005f610 con 0x5572e005dfc0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 1 -- 172.21.15.43:0/3728026707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0e60074050 con 0x5572e005dfc0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 10 client.6552 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5572dfe16ca0 con 0x5572e005dfc0 2024-03-30T13:14:33.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 1 -- 172.21.15.43:0/3728026707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e68003ba0 con 0x5572e005dfc0 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 1 -- 172.21.15.43:0/3728026707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f0e60056310 con 0x5572e005dfc0 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 10 client.6552 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 10 client.6552 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 20 client.6552 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e001bbf0 0x5572e0070280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 4 client.6552 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572dfdf9c90 con 0x5572e001bbf0 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e001bbf0 0x5572e0070280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 10 client.6552 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e001bbf0 0x5572e0070280 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9accfdf6a53938bb server_cookie=1784d51fc5792dd in_seq=0 out_seq=0 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 1 -- 172.21.15.43:0/3728026707 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5572dfdf9c90 con 0x5572e001bbf0 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 10 client.6552 handle_command_reply: tid=0 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 client.6552 shutdown 2024-03-30T13:14:33.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 msgr2=0x5572e001a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e001a910 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0e60056bc0 tx=0x7f0e60056f80 comp rx=0 tx=0).stop 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 shutdown_connections 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e001bbf0 0x5572e0070280 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e005e8e0 0x5572e001ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e001a910 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e50ff9640 0 client.6552 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3728026707 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e005d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 >> 172.21.15.43:0/3728026707 conn(0x5572e000c7e0 msgr2=0x5572e00158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 shutdown_connections 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 1 -- 172.21.15.43:0/3728026707 wait complete. 2024-03-30T13:14:33.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.050+0000 7f0e7af34000 20 client.6552 trim_cache size 0 max 16384 2024-03-30T13:14:33.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 Processor -- start 2024-03-30T13:14:33.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- start start 2024-03-30T13:14:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e0014c40 0x5572e003d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e00732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e00737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0e702003b0 con 0x5572e005dfc0 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701ff3f0 con 0x5572e005e8e0 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0e701fe750 con 0x5572e0014c40 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e00732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e00732b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33386/0 (socket says 172.21.15.43:33386) 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 -- 172.21.15.43:0/3876608453 learned_addr learned my addr 172.21.15.43:0/3876608453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7634f640 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e00737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e0014c40 0x5572e003d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 -- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e0014c40 msgr2=0x5572e003d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e0014c40 0x5572e003d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 -- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 msgr2=0x5572e00737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e00737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 -- 172.21.15.43:0/3876608453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572dfde5d00 con 0x5572e005dfc0 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7634f640 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e00737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7534d640 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e00732b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0e60053840 tx=0x7f0e60059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433141c2a0e4d53e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e52ffd640 1 -- 172.21.15.43:0/3876608453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60079020 con 0x5572e005dfc0 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e52ffd640 1 -- 172.21.15.43:0/3876608453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e600575c0 con 0x5572e005dfc0 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e52ffd640 1 -- 172.21.15.43:0/3876608453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6005b270 con 0x5572e005dfc0 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- 172.21.15.43:0/3876608453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 msgr2=0x5572e00732b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e00732b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0e60053840 tx=0x7f0e60059000 comp rx=0 tx=0).stop 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- 172.21.15.43:0/3876608453 shutdown_connections 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e005e8e0 0x5572e00737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005dfc0 0x5572e00732b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 --2- 172.21.15.43:0/3876608453 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e0014c40 0x5572e003d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- 172.21.15.43:0/3876608453 >> 172.21.15.43:0/3876608453 conn(0x5572e000c7e0 msgr2=0x5572e003e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- 172.21.15.43:0/3876608453 shutdown_connections 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- 172.21.15.43:0/3876608453 wait complete. 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.066+0000 7f0e7af34000 1 -- start start 2024-03-30T13:14:33.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 1 Processor -- start 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e00d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005e8e0 0x5572e00d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e00712d0 0x5572e00716b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5572dfe26270 con 0x5572e005e8e0 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0e701feda0 con 0x5572e0014c40 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5572dfe5ddb0 con 0x5572e00712d0 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7af34000 10 client.0 _ll_register_callbacks cb 0x7f0e60053160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005e8e0 0x5572e00d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005e8e0 0x5572e00d0f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33388/0 (socket says 172.21.15.43:33388) 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 -- 172.21.15.43:0/1806883678 learned_addr learned my addr 172.21.15.43:0/1806883678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e00d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7634f640 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e00712d0 0x5572e00716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 -- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e00712d0 msgr2=0x5572e00716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e00712d0 0x5572e00716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 -- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 msgr2=0x5572e00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e00d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 -- 172.21.15.43:0/1806883678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572dfedff70 con 0x5572e005e8e0 2024-03-30T13:14:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e7534d640 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005e8e0 0x5572e00d0f50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0e60056bc0 tx=0x7f0e60085210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9818b5629d6ba server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.070+0000 7f0e50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e50ff9640 1 -- 172.21.15.43:0/1806883678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e60079030 con 0x5572e005e8e0 2024-03-30T13:14:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e50ff9640 1 -- 172.21.15.43:0/1806883678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f0e60052c10 con 0x5572e005e8e0 2024-03-30T13:14:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e600575c0 con 0x5572e005e8e0 2024-03-30T13:14:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e50ff9640 1 -- 172.21.15.43:0/1806883678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0e60052e50 con 0x5572e005e8e0 2024-03-30T13:14:33.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 10 client.6558 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5572dfe05760 con 0x5572e005e8e0 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e50ff9640 1 -- 172.21.15.43:0/1806883678 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e600575c0 con 0x5572e005e8e0 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e50ff9640 1 -- 172.21.15.43:0/1806883678 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f0e60055d80 con 0x5572e005e8e0 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 10 client.6558 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 10 client.6558 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 20 client.6558 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e0071bf0 0x5572e0071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 4 client.6558 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628401}] 2024-03-30T13:14:33.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628401}) -- 0x5572dfe6bfe0 con 0x5572e0071bf0 2024-03-30T13:14:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.074+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e0071bf0 0x5572e0071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e75b4e640 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e0071bf0 0x5572e0071fd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1292057c1974cd server_cookie=ca02cee0d4b00e18 in_seq=0 out_seq=0 2024-03-30T13:14:33.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e50ff9640 10 client.6558 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e50ff9640 1 -- 172.21.15.43:0/1806883678 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3795 (crc 0 0 0) 0x5572dfe6bfe0 con 0x5572e0071bf0 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e50ff9640 10 client.6558 handle_command_reply: tid=0 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 client.6558 shutdown 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005e8e0 msgr2=0x5572e00d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005e8e0 0x5572e00d0f50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0e60056bc0 tx=0x7f0e60085210 comp rx=0 tx=0).stop 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 shutdown_connections 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5572e0071bf0 0x5572e0071fd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5572e00712d0 0x5572e00716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5572e005e8e0 0x5572e00d0f50 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e50ff9640 0 client.6558 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 --2- 172.21.15.43:0/1806883678 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5572e0014c40 0x5572e00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 >> 172.21.15.43:0/1806883678 conn(0x5572e000c7e0 msgr2=0x5572e000ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 shutdown_connections 2024-03-30T13:14:33.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 1 -- 172.21.15.43:0/1806883678 wait complete. 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.078+0000 7f0e7af34000 20 client.6558 trim_cache size 0 max 16384 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/ext2/xattr.h", 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628401, 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.598068+0000", 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.747720+0000", 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:33.086 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:33.087 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3340, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.747720+0000", 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:33.088 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3340, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.089 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.598068+0000" 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3340, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.598068+0000" 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 168, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:33.090 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 32, 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:33.091 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.092 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.093 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.095 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:33.096 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.097 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:33.098 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:33.098 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e54067ff0 msgr2=0x7f0e5406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e54067ff0 0x7f0e5406a4b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0aa000 tx=0x7f0e6c000ff0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e70155930 msgr2=0x7f0e70141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e70155930 0x7f0e70141380 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f0e60011610 tx=0x7f0e60011ad0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 shutdown_connections 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0e54067ff0 0x7f0e5406a4b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0e701418c0 0x7f0e701405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0e70155930 0x7f0e70141380 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 --2- 172.21.15.43:0/3006432886 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0e5c0039c0 0x7f0e7013d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 >> 172.21.15.43:0/3006432886 conn(0x7f0e700d00a0 msgr2=0x7f0e700b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 shutdown_connections 2024-03-30T13:14:33.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.082+0000 7f0e77db2640 1 -- 172.21.15.43:0/3006432886 wait complete. 2024-03-30T13:14:33.101 DEBUG:tasks.cephfs.test_uninlining:getting inode info #14 of 23:1099511628406 2024-03-30T13:14:33.101 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628406 2024-03-30T13:14:33.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628406 2024-03-30T13:14:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.210+0000 7f6ef896b640 1 Processor -- start 2024-03-30T13:14:33.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.210+0000 7f6ef896b640 1 -- start start 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef896b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ef40b4870 0x7f6ef406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef896b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef40ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef896b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef40b3660 con 0x7f6ef4144930 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef896b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef40cdb90 con 0x7f6ef40b4870 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef896b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef40b3940 con 0x7f6ef406cbb0 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ef40b4870 0x7f6ef406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6ef40b4870 0x7f6ef406c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33400/0 (socket says 172.21.15.43:33400) 2024-03-30T13:14:33.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 -- 172.21.15.43:0/2962071697 learned_addr learned my addr 172.21.15.43:0/2962071697 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2962071697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef40ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 -- 172.21.15.43:0/2962071697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 msgr2=0x7f6ef40ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 --2- 172.21.15.43:0/2962071697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef40ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 -- 172.21.15.43:0/2962071697 >> v1:172.21.15.113:6789/0 conn(0x7f6ef4144930 legacy=0x7f6ef40b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 -- 172.21.15.43:0/2962071697 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef40b3c20 con 0x7f6ef40b4870 2024-03-30T13:14:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6eebfff640 1 --2- 172.21.15.43:0/2962071697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef406c670 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009870 tx=0x7f6edc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a717cec2fbb245f4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef1d9b640 1 -- 172.21.15.43:0/2962071697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0b2070 con 0x7f6ef40b4870 2024-03-30T13:14:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef1d9b640 1 -- 172.21.15.43:0/2962071697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6edc0a9070 con 0x7f6ef40b4870 2024-03-30T13:14:33.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.214+0000 7f6ef1d9b640 1 -- 172.21.15.43:0/2962071697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0ad880 con 0x7f6ef40b4870 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- 172.21.15.43:0/2962071697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 msgr2=0x7f6ef406c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2962071697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef406c670 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009870 tx=0x7f6edc01c490 comp rx=0 tx=0).stop 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- 172.21.15.43:0/2962071697 shutdown_connections 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2962071697 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef40ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2962071697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef406c670 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- 172.21.15.43:0/2962071697 >> 172.21.15.43:0/2962071697 conn(0x7f6ef40d00a0 msgr2=0x7f6ef40d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- 172.21.15.43:0/2962071697 shutdown_connections 2024-03-30T13:14:33.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- 172.21.15.43:0/2962071697 wait complete. 2024-03-30T13:14:33.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 Processor -- start 2024-03-30T13:14:33.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- start start 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef4165630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef41f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ef4144930 0x7f6ef40aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40cf180 con 0x7f6ef40b4870 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40cf000 con 0x7f6ef4144930 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef40cf300 con 0x7f6ef406cbb0 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef41f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef41f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33410/0 (socket says 172.21.15.43:33410) 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 -- 172.21.15.43:0/2506170034 learned_addr learned my addr 172.21.15.43:0/2506170034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef259c640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef4165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ef4144930 0x7f6ef40aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 -- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 msgr2=0x7f6ef4165630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef4165630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 -- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ef4144930 msgr2=0x7f6ef40aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ef4144930 0x7f6ef40aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 -- 172.21.15.43:0/2506170034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef40ab810 con 0x7f6ef40b4870 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ef4144930 0x7f6ef40aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eebfff640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef41f51c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b0040 tx=0x7f6edc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eeaffd640 1 -- 172.21.15.43:0/2506170034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0bc020 con 0x7f6ef40b4870 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eeaffd640 1 -- 172.21.15.43:0/2506170034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6edc0ade50 con 0x7f6ef40b4870 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef40abaf0 con 0x7f6ef40b4870 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6eeaffd640 1 -- 172.21.15.43:0/2506170034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0b2040 con 0x7f6ef40b4870 2024-03-30T13:14:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.218+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef40ac0b0 con 0x7f6ef40b4870 2024-03-30T13:14:33.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6eeaffd640 1 -- 172.21.15.43:0/2506170034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6edc0bf020 con 0x7f6ef40b4870 2024-03-30T13:14:33.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6eeaffd640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6ed4067fc0 0x7f6ed406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6eeaffd640 1 -- 172.21.15.43:0/2506170034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f6edc1209c0 con 0x7f6ef40b4870 2024-03-30T13:14:33.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6ed4067fc0 0x7f6ed406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 Processor -- start 2024-03-30T13:14:33.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 -- start start 2024-03-30T13:14:33.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6ed4067fc0 0x7f6ed406a480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4004e30 tx=0x7f6ee4009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0beefc0 0x55a7e0bef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0bef8e0 0x55a7e0ba5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0ba6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a7e09efaa0 con 0x55a7e0beefc0 2024-03-30T13:14:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a7e0976af0 con 0x55a7e0ba5c40 2024-03-30T13:14:33.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a7e09eba50 con 0x55a7e0bef8e0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0beefc0 0x55a7e0bef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0beefc0 0x55a7e0bef3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33422/0 (socket says 172.21.15.43:33422) 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 -- 172.21.15.43:0/2782690156 learned_addr learned my addr 172.21.15.43:0/2782690156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6eebfff640 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0bef8e0 0x55a7e0ba5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0ba6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 -- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0bef8e0 msgr2=0x55a7e0ba5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0bef8e0 0x55a7e0ba5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 -- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 msgr2=0x55a7e0ba6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0ba6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 -- 172.21.15.43:0/2782690156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e0974d00 con 0x55a7e0beefc0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0ba6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ef259c640 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0beefc0 0x55a7e0bef3a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40055d0 tx=0x7f6ee4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f2abb29203a7dd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ed3fff640 1 -- 172.21.15.43:0/2782690156 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4025070 con 0x55a7e0beefc0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ed3fff640 1 -- 172.21.15.43:0/2782690156 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee4015880 con 0x55a7e0beefc0 2024-03-30T13:14:33.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6ed3fff640 1 -- 172.21.15.43:0/2782690156 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4015c30 con 0x55a7e0beefc0 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 -- 172.21.15.43:0/2782690156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0beefc0 msgr2=0x55a7e0bef3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.222+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0beefc0 0x55a7e0bef3a0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40055d0 tx=0x7f6ee4011000 comp rx=0 tx=0).stop 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- 172.21.15.43:0/2782690156 shutdown_connections 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0ba6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0bef8e0 0x55a7e0ba5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2782690156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0beefc0 0x55a7e0bef3a0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- 172.21.15.43:0/2782690156 >> 172.21.15.43:0/2782690156 conn(0x55a7e0b9d7e0 msgr2=0x55a7e0b9fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- 172.21.15.43:0/2782690156 shutdown_connections 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- 172.21.15.43:0/2782690156 wait complete. 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- start start 2024-03-30T13:14:33.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 Processor -- start 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0ba5c40 0x55a7e0bf6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0beefc0 0x55a7e0bf6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bd30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a7e09b7270 con 0x55a7e0bef8e0 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55a7e09ad6a0 con 0x55a7e0beefc0 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a7e0987880 con 0x55a7e0ba5c40 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 10 client.0 _ll_register_callbacks cb 0x55a7e0b8c490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bd30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bd30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33428/0 (socket says 172.21.15.43:33428) 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/363201416 learned_addr learned my addr 172.21.15.43:0/363201416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0ba5c40 msgr2=0x55a7e0bf6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0ba5c40 0x55a7e0bf6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0beefc0 msgr2=0x55a7e0bf6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0beefc0 0x55a7e0bf6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/363201416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e09b4730 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bd30b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0079d0 tx=0x7f6eec007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ceba5be1225f6c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ed17fa640 1 -- 172.21.15.43:0/363201416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec019020 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ed17fa640 1 -- 172.21.15.43:0/363201416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f6eec010c70 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee4015880 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ed17fa640 1 -- 172.21.15.43:0/363201416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6eec023050 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 10 client.6570 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55a7e09a7ca0 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ed17fa640 1 -- 172.21.15.43:0/363201416 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee4015880 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ed17fa640 1 -- 172.21.15.43:0/363201416 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f6eec026020 con 0x55a7e0bef8e0 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 10 client.6570 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 10 client.6570 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 20 client.6570 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bd3df0 0x55a7e0bd41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 4 client.6570 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7e098ac90 con 0x55a7e0bd3df0 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef259c640 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bd3df0 0x55a7e0bd41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ef259c640 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bd3df0 0x55a7e0bd41d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0de524dc464251a server_cookie=d2e0939f6078b367 in_seq=0 out_seq=0 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.226+0000 7f6ed17fa640 10 client.6570 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6ed17fa640 1 -- 172.21.15.43:0/363201416 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55a7e098ac90 con 0x55a7e0bd3df0 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6ed17fa640 10 client.6570 handle_command_reply: tid=0 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 client.6570 shutdown 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 msgr2=0x55a7e0bd30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bd30b0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0079d0 tx=0x7f6eec007e90 comp rx=0 tx=0).stop 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 shutdown_connections 2024-03-30T13:14:33.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bd3df0 0x55a7e0bd41d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bd30b0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0beefc0 0x55a7e0bf6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 --2- 172.21.15.43:0/363201416 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0ba5c40 0x55a7e0bf6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 >> 172.21.15.43:0/363201416 conn(0x55a7e0b9d7e0 msgr2=0x55a7e0ba68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6ed17fa640 0 client.6570 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 shutdown_connections 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 1 -- 172.21.15.43:0/363201416 wait complete. 2024-03-30T13:14:33.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.230+0000 7f6efbaed000 20 client.6570 trim_cache size 0 max 16384 2024-03-30T13:14:33.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.242+0000 7f6efbaed000 1 Processor -- start 2024-03-30T13:14:33.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.242+0000 7f6efbaed000 1 -- start start 2024-03-30T13:14:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 0x55a7e0c06c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bf4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef41ffac0 con 0x55a7e0bef8e0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef40b0d20 con 0x55a7e0ba5c40 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef40ad340 con 0x55a7e0beefc0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bf4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bf4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33442/0 (socket says 172.21.15.43:33442) 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/2963307936 learned_addr learned my addr 172.21.15.43:0/2963307936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef259c640 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/2963307936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 msgr2=0x55a7e0c06c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 0x55a7e0c06c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/2963307936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 msgr2=0x55a7e0c066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c066d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/2963307936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7e0bef710 con 0x55a7e0bef8e0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bf4d00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6eec002240 tx=0x7f6eec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864299126396ca76 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed37fe640 1 -- 172.21.15.43:0/2963307936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec019040 con 0x55a7e0bef8e0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed37fe640 1 -- 172.21.15.43:0/2963307936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eec00e070 con 0x55a7e0bef8e0 2024-03-30T13:14:33.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed37fe640 1 -- 172.21.15.43:0/2963307936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec003be0 con 0x55a7e0bef8e0 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- 172.21.15.43:0/2963307936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 msgr2=0x55a7e0bf4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bf4d00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6eec002240 tx=0x7f6eec015000 comp rx=0 tx=0).stop 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- 172.21.15.43:0/2963307936 shutdown_connections 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0bf4d00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 0x55a7e0c06c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- 172.21.15.43:0/2963307936 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- 172.21.15.43:0/2963307936 >> 172.21.15.43:0/2963307936 conn(0x55a7e0b9d7e0 msgr2=0x55a7e0bee750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- 172.21.15.43:0/2963307936 shutdown_connections 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- 172.21.15.43:0/2963307936 wait complete. 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- start start 2024-03-30T13:14:33.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 Processor -- start 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c05ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 0x55a7e0c06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0c06a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55a7e09ad6a0 con 0x55a7e0bef8e0 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef415dad0 con 0x55a7e0ba5c40 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55a7e0bd3470 con 0x55a7e0beefc0 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 10 client.0 _ll_register_callbacks cb 0x7f6eec0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0c06a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0c06a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33444/0 (socket says 172.21.15.43:33444) 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/4155180058 learned_addr learned my addr 172.21.15.43:0/4155180058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6eebfff640 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 0x55a7e0c06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 msgr2=0x55a7e0c06550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef259c640 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c05ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 0x55a7e0c06550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 msgr2=0x55a7e0c05ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c05ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 -- 172.21.15.43:0/4155180058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7e0976d00 con 0x55a7e0bef8e0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ef2d9d640 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0c06a90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0078f0 tx=0x7f6eec004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684aa9e05a846df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed17fa640 1 -- 172.21.15.43:0/4155180058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec019040 con 0x55a7e0bef8e0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed17fa640 1 -- 172.21.15.43:0/4155180058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f6eec031d40 con 0x55a7e0bef8e0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eec00e070 con 0x55a7e0bef8e0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed17fa640 1 -- 172.21.15.43:0/4155180058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6eec004040 con 0x55a7e0bef8e0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 10 client.6576 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55a7e0a70f70 con 0x55a7e0bef8e0 2024-03-30T13:14:33.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed17fa640 1 -- 172.21.15.43:0/4155180058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eec00e070 con 0x55a7e0bef8e0 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.246+0000 7f6ed17fa640 1 -- 172.21.15.43:0/4155180058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f6eec026070 con 0x55a7e0bef8e0 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 10 client.6576 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 10 client.6576 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 20 client.6576 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bf2150 0x55a7e0bf2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 4 client.6576 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628406}] 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628406}) -- 0x55a7e09fcfe0 con 0x55a7e0bf2150 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef259c640 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bf2150 0x55a7e0bf2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef259c640 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bf2150 0x55a7e0bf2560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d929de5932fbd11 server_cookie=c46b47beacbc9288 in_seq=0 out_seq=0 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ed17fa640 10 client.6576 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ed17fa640 1 -- 172.21.15.43:0/4155180058 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3795 (crc 0 0 0) 0x55a7e09fcfe0 con 0x55a7e0bf2150 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ed17fa640 10 client.6576 handle_command_reply: tid=0 2024-03-30T13:14:33.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 client.6576 shutdown 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 msgr2=0x55a7e0c06a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0c06a90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0078f0 tx=0x7f6eec004a50 comp rx=0 tx=0).stop 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 shutdown_connections 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55a7e0bf2150 0x55a7e0bf2560 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a7e0bef8e0 0x55a7e0c06a90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55a7e0beefc0 0x55a7e0c06550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 --2- 172.21.15.43:0/4155180058 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55a7e0ba5c40 0x55a7e0c05ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 >> 172.21.15.43:0/4155180058 conn(0x55a7e0b9d7e0 msgr2=0x55a7e0bced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ed17fa640 0 client.6576 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 shutdown_connections 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 1 -- 172.21.15.43:0/4155180058 wait complete. 2024-03-30T13:14:33.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6efbaed000 20 client.6576 trim_cache size 0 max 16384 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/Kconfig", 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628406, 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.605165+0000", 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.915709+0000", 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:33.260 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 228, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:33.261 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.915709+0000", 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:33.262 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 228, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.605165+0000" 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 228, 2024-03-30T13:14:33.263 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.605165+0000" 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 198, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 2, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:33.264 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.265 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:33.266 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.267 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.268 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.269 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:33.270 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:33.271 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:33.271 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:33.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6ed4067fc0 msgr2=0x7f6ed406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6ed4067fc0 0x7f6ed406a480 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4004e30 tx=0x7f6ee4009040 comp rx=0 tx=0).stop 2024-03-30T13:14:33.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 msgr2=0x7f6ef41f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef41f51c0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0b0040 tx=0x7f6edc01acd0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 shutdown_connections 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6ed4067fc0 0x7f6ed406a480 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6ef4144930 0x7f6ef40aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ef40b4870 0x7f6ef41f51c0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 --2- 172.21.15.43:0/2506170034 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6ef406cbb0 0x7f6ef4165630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 >> 172.21.15.43:0/2506170034 conn(0x7f6ef40d00a0 msgr2=0x7f6ef41417a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 shutdown_connections 2024-03-30T13:14:33.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.250+0000 7f6ef896b640 1 -- 172.21.15.43:0/2506170034 wait complete. 2024-03-30T13:14:33.273 DEBUG:tasks.cephfs.test_uninlining:getting inode info #15 of 23:1099511628419 2024-03-30T13:14:33.273 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628419 2024-03-30T13:14:33.274 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628419 2024-03-30T13:14:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 Processor -- start 2024-03-30T13:14:33.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 -- start start 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58015e4c0 0x7fc5800b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc58006c070 0x7fc5800cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc5800cf4f0 0x7fc580161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc58015e8a0 con 0x7fc58006c070 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc58006c850 con 0x7fc5800cf4f0 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc5861fe640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc58006cb30 con 0x7fc58015e4c0 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc58006c070 0x7fc5800cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc58006c070 0x7fc5800cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33450/0 (socket says 172.21.15.43:33450) 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 -- 172.21.15.43:0/1740111886 learned_addr learned my addr 172.21.15.43:0/1740111886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57ffff640 1 --2- 172.21.15.43:0/1740111886 >> v2:172.21.15.113:3300/0 conn(0x7fc5800cf4f0 0x7fc580161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/1740111886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58015e4c0 0x7fc5800b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 -- 172.21.15.43:0/1740111886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58015e4c0 msgr2=0x7fc5800b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 --2- 172.21.15.43:0/1740111886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58015e4c0 0x7fc5800b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 -- 172.21.15.43:0/1740111886 >> v2:172.21.15.113:3300/0 conn(0x7fc5800cf4f0 msgr2=0x7fc580161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 --2- 172.21.15.43:0/1740111886 >> v2:172.21.15.113:3300/0 conn(0x7fc5800cf4f0 0x7fc580161f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57effd640 1 -- 172.21.15.43:0/1740111886 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58006ce10 con 0x7fc58006c070 2024-03-30T13:14:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/1740111886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58015e4c0 0x7fc5800b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.378+0000 7fc57ffff640 1 --2- 172.21.15.43:0/1740111886 >> v2:172.21.15.113:3300/0 conn(0x7fc5800cf4f0 0x7fc580161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57effd640 1 --2- 172.21.15.43:0/1740111886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58006c070 0x7fc5800cefb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc5800ce8c0 tx=0x7fc57401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c602cc1eab45f7ea server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57e7fc640 1 -- 172.21.15.43:0/1740111886 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740b0070 con 0x7fc58006c070 2024-03-30T13:14:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57e7fc640 1 -- 172.21.15.43:0/1740111886 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5740ab600 con 0x7fc58006c070 2024-03-30T13:14:33.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57e7fc640 1 -- 172.21.15.43:0/1740111886 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740ab9d0 con 0x7fc58006c070 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- 172.21.15.43:0/1740111886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58006c070 msgr2=0x7fc5800cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 --2- 172.21.15.43:0/1740111886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58006c070 0x7fc5800cefb0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fc5800ce8c0 tx=0x7fc57401c260 comp rx=0 tx=0).stop 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- 172.21.15.43:0/1740111886 shutdown_connections 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 --2- 172.21.15.43:0/1740111886 >> v2:172.21.15.113:3300/0 conn(0x7fc5800cf4f0 0x7fc580161f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 --2- 172.21.15.43:0/1740111886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58006c070 0x7fc5800cefb0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 --2- 172.21.15.43:0/1740111886 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58015e4c0 0x7fc5800b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- 172.21.15.43:0/1740111886 >> 172.21.15.43:0/1740111886 conn(0x7fc5800b6410 msgr2=0x7fc5800d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- 172.21.15.43:0/1740111886 shutdown_connections 2024-03-30T13:14:33.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- 172.21.15.43:0/1740111886 wait complete. 2024-03-30T13:14:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 Processor -- start 2024-03-30T13:14:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- start start 2024-03-30T13:14:33.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58006c070 0x7fc58015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800cf4f0 0x7fc580156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58015e4c0 0x7fc58014d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5800d38f0 con 0x7fc58015e4c0 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc5800d3770 con 0x7fc5800cf4f0 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc5800d3a70 con 0x7fc58006c070 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58015e4c0 0x7fc58014d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58015e4c0 0x7fc58014d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33454/0 (socket says 172.21.15.43:33454) 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 -- 172.21.15.43:0/3742645243 learned_addr learned my addr 172.21.15.43:0/3742645243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57effd640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800cf4f0 0x7fc580156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58006c070 0x7fc58015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 -- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58006c070 msgr2=0x7fc58015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58006c070 0x7fc58015ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 -- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800cf4f0 msgr2=0x7fc580156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800cf4f0 0x7fc580156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 -- 172.21.15.43:0/3742645243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc580157420 con 0x7fc58015e4c0 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57effd640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800cf4f0 0x7fc580156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58006c070 0x7fc58015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc57ffff640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58015e4c0 0x7fc58014d800 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc57800cef0 tx=0x7fc57800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc567fff640 1 -- 172.21.15.43:0/3742645243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57801a070 con 0x7fc58015e4c0 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc567fff640 1 -- 172.21.15.43:0/3742645243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57800d840 con 0x7fc58015e4c0 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58015b2d0 con 0x7fc58015e4c0 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc567fff640 1 -- 172.21.15.43:0/3742645243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57800dc30 con 0x7fc58015e4c0 2024-03-30T13:14:33.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.382+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58014fac0 con 0x7fc58015e4c0 2024-03-30T13:14:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc567fff640 1 -- 172.21.15.43:0/3742645243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc57801d070 con 0x7fc58015e4c0 2024-03-30T13:14:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc567fff640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc56c067ff0 0x7fc56c06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc567fff640 1 -- 172.21.15.43:0/3742645243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7fc578086e70 con 0x7fc58015e4c0 2024-03-30T13:14:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc56c067ff0 0x7fc56c06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc56c067ff0 0x7fc56c06a4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc5800ca0d0 tx=0x7fc570008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 Processor -- start 2024-03-30T13:14:33.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 -- start start 2024-03-30T13:14:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a44414150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a44218aa0 con 0x556a44414690 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556a4419faf0 con 0x556a443ca830 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x556a44214a50 con 0x556a44413d70 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33458/0 (socket says 172.21.15.43:33458) 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 -- 172.21.15.43:0/1669514065 learned_addr learned my addr 172.21.15.43:0/1669514065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a44414150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57ffff640 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 -- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 msgr2=0x556a44414150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a44414150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 -- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 msgr2=0x556a443cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 -- 172.21.15.43:0/1669514065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4419dd00 con 0x556a44414690 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.386+0000 7fc57effd640 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443ca2f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc5740ae040 tx=0x7fc574004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5a48b1d14d7cbb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc5657fa640 1 -- 172.21.15.43:0/1669514065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740b0040 con 0x556a44414690 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc5657fa640 1 -- 172.21.15.43:0/1669514065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5740abb00 con 0x556a44414690 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc5657fa640 1 -- 172.21.15.43:0/1669514065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740b7460 con 0x556a44414690 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/1669514065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 msgr2=0x556a443ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443ca2f0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc5740ae040 tx=0x7fc574004740 comp rx=0 tx=0).stop 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/1669514065 shutdown_connections 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443ca2f0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- 172.21.15.43:0/1669514065 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a44414150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/1669514065 >> 172.21.15.43:0/1669514065 conn(0x556a443c67e0 msgr2=0x556a443c8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/1669514065 shutdown_connections 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/1669514065 wait complete. 2024-03-30T13:14:33.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- start start 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 Processor -- start 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a443f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a441e0270 con 0x556a44414690 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x556a441d66a0 con 0x556a443ca830 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x556a441b0880 con 0x556a44413d70 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 10 client.0 _ll_register_callbacks cb 0x556a443b5490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443f92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33460/0 (socket says 172.21.15.43:33460) 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 -- 172.21.15.43:0/2942849547 learned_addr learned my addr 172.21.15.43:0/2942849547 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57effd640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a443f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 -- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 msgr2=0x556a443f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a443f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 -- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 msgr2=0x556a443cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 -- 172.21.15.43:0/2942849547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a441dd730 con 0x556a44414690 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57ffff640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443f92e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc57805f450 tx=0x7fc578056e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a43f824dfa3137a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc55e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc55e7fc640 1 -- 172.21.15.43:0/2942849547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57805a030 con 0x556a44414690 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc55e7fc640 1 -- 172.21.15.43:0/2942849547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7fc57806f070 con 0x556a44414690 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5740abb00 con 0x556a44414690 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc55e7fc640 1 -- 172.21.15.43:0/2942849547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc578053410 con 0x556a44414690 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 10 client.6588 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x556a441d0ca0 con 0x556a44414690 2024-03-30T13:14:33.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc55e7fc640 1 -- 172.21.15.43:0/2942849547 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5740abb00 con 0x556a44414690 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc55e7fc640 1 -- 172.21.15.43:0/2942849547 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fc57805f4f0 con 0x556a44414690 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 10 client.6588 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 10 client.6588 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 20 client.6588 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a4441b5d0 0x556a4441b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 4 client.6588 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a441b3c90 con 0x556a4441b5d0 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.390+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a4441b5d0 0x556a4441b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a4441b5d0 0x556a4441b9b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad77c630ef0d1c4a server_cookie=451ff28b545c14a in_seq=0 out_seq=0 2024-03-30T13:14:33.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc55e7fc640 10 client.6588 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc55e7fc640 1 -- 172.21.15.43:0/2942849547 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x556a441b3c90 con 0x556a4441b5d0 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc55e7fc640 10 client.6588 handle_command_reply: tid=0 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 client.6588 shutdown 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 msgr2=0x556a443f92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443f92e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc57805f450 tx=0x7fc578056e60 comp rx=0 tx=0).stop 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 shutdown_connections 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a4441b5d0 0x556a4441b9b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44414690 0x556a443f92e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44413d70 0x556a443f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc55e7fc640 0 client.6588 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 --2- 172.21.15.43:0/2942849547 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a443cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 >> 172.21.15.43:0/2942849547 conn(0x556a443c67e0 msgr2=0x556a443c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 shutdown_connections 2024-03-30T13:14:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 1 -- 172.21.15.43:0/2942849547 wait complete. 2024-03-30T13:14:33.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.394+0000 7fc589380000 20 client.6588 trim_cache size 0 max 16384 2024-03-30T13:14:33.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.406+0000 7fc589380000 1 Processor -- start 2024-03-30T13:14:33.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.406+0000 7fc589380000 1 -- start start 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a443ca830 0x556a443f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a443c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a44414690 0x556a443c9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc5802046e0 con 0x556a44413d70 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc580203730 con 0x556a44414690 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc580155790 con 0x556a443ca830 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a443c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a443c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33462/0 (socket says 172.21.15.43:33462) 2024-03-30T13:14:33.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/1237786723 learned_addr learned my addr 172.21.15.43:0/1237786723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57ffff640 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a44414690 0x556a443c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a443ca830 0x556a443f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a443ca830 msgr2=0x556a443f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a443ca830 0x556a443f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a44414690 msgr2=0x556a443c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a44414690 0x556a443c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/1237786723 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4419fd00 con 0x556a44413d70 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a443c8bd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc574008010 tx=0x7fc574009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7072ab2eaec00287 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc564ff9640 1 -- 172.21.15.43:0/1237786723 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740c4070 con 0x556a44413d70 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc564ff9640 1 -- 172.21.15.43:0/1237786723 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5740043f0 con 0x556a44413d70 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc564ff9640 1 -- 172.21.15.43:0/1237786723 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740b0040 con 0x556a44413d70 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- 172.21.15.43:0/1237786723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 msgr2=0x556a443c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a443c8bd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc574008010 tx=0x7fc574009240 comp rx=0 tx=0).stop 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- 172.21.15.43:0/1237786723 shutdown_connections 2024-03-30T13:14:33.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a44414690 0x556a443c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a443c8bd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- 172.21.15.43:0/1237786723 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a443ca830 0x556a443f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- 172.21.15.43:0/1237786723 >> 172.21.15.43:0/1237786723 conn(0x556a443c67e0 msgr2=0x556a4442af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- 172.21.15.43:0/1237786723 shutdown_connections 2024-03-30T13:14:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- 172.21.15.43:0/1237786723 wait complete. 2024-03-30T13:14:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- start start 2024-03-30T13:14:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 Processor -- start 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a44481e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a44417ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44414690 0x556a44418010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x556a441e0270 con 0x556a44413d70 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc580155de0 con 0x556a443ca830 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x556a44217db0 con 0x556a44414690 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc589380000 10 client.0 _ll_register_callbacks cb 0x7fc578007a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a44417ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a44417ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33472/0 (socket says 172.21.15.43:33472) 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/230891890 learned_addr learned my addr 172.21.15.43:0/230891890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a44481e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57ffff640 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44414690 0x556a44418010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44414690 msgr2=0x556a44418010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44414690 0x556a44418010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 msgr2=0x556a44481e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a44481e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.410+0000 7fc57effd640 1 -- 172.21.15.43:0/230891890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a44299f70 con 0x556a44413d70 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc57effd640 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a44417ad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc574008010 tx=0x7fc57401aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7722934aa15f757b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 1 -- 172.21.15.43:0/230891890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740b0030 con 0x556a44413d70 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 1 -- 172.21.15.43:0/230891890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7fc5740c9d20 con 0x556a44413d70 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5740043f0 con 0x556a44413d70 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 1 -- 172.21.15.43:0/230891890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc5740c4040 con 0x556a44413d70 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 10 client.6594 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x556a441bf760 con 0x556a44413d70 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 1 -- 172.21.15.43:0/230891890 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5740043f0 con 0x556a44413d70 2024-03-30T13:14:33.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 1 -- 172.21.15.43:0/230891890 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fc574003620 con 0x556a44413d70 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 10 client.6594 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 10 client.6594 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 20 client.6594 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a44418550 0x556a44418960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 4 client.6594 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628419}] 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628419}) -- 0x556a44225fe0 con 0x556a44418550 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a44418550 0x556a44418960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc57f7fe640 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a44418550 0x556a44418960 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16a85f113551c874 server_cookie=a83882f58b5f8ab in_seq=0 out_seq=0 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 10 client.6594 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 1 -- 172.21.15.43:0/230891890 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3808 (crc 0 0 0) 0x556a44225fe0 con 0x556a44418550 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 10 client.6594 handle_command_reply: tid=0 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 client.6594 shutdown 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 msgr2=0x556a44417ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a44417ad0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc574008010 tx=0x7fc57401aaa0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 shutdown_connections 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x556a44418550 0x556a44418960 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x556a44414690 0x556a44418010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a44413d70 0x556a44417ad0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 --2- 172.21.15.43:0/230891890 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x556a443ca830 0x556a44481e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc55e7fc640 0 client.6594 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 >> 172.21.15.43:0/230891890 conn(0x556a443c67e0 msgr2=0x556a4442b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 shutdown_connections 2024-03-30T13:14:33.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 1 -- 172.21.15.43:0/230891890 wait complete. 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc589380000 20 client.6594 trim_cache size 0 max 16384 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debug.h", 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628419, 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.607359+0000", 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.989549+0000", 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:33.425 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:33.426 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 2579, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.989549+0000", 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 4, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:33.427 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2579, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.428 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.607359+0000" 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2579, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.607359+0000" 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 202, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:33.429 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 28, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:33.430 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:33.431 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:33.432 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.433 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:33.434 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.435 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:33.436 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:33.437 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:33.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc56c067ff0 msgr2=0x7fc56c06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc56c067ff0 0x7fc56c06a4b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc5800ca0d0 tx=0x7fc570008040 comp rx=0 tx=0).stop 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58015e4c0 msgr2=0x7fc58014d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58015e4c0 0x7fc58014d800 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fc57800cef0 tx=0x7fc57800a710 comp rx=0 tx=0).stop 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 shutdown_connections 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc56c067ff0 0x7fc56c06a4b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc58015e4c0 0x7fc58014d800 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc5800cf4f0 0x7fc580156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 --2- 172.21.15.43:0/3742645243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc58006c070 0x7fc58015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 >> 172.21.15.43:0/3742645243 conn(0x7fc5800b6410 msgr2=0x7fc5800cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 shutdown_connections 2024-03-30T13:14:33.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.414+0000 7fc5861fe640 1 -- 172.21.15.43:0/3742645243 wait complete. 2024-03-30T13:14:33.439 DEBUG:tasks.cephfs.test_uninlining:getting inode info #16 of 23:1099511628426 2024-03-30T13:14:33.440 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628426 2024-03-30T13:14:33.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628426 2024-03-30T13:14:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.550+0000 7f227c43a640 1 Processor -- start 2024-03-30T13:14:33.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.550+0000 7f227c43a640 1 -- start start 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2274144930 0x7f22740b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f22740b4870 0x7f227406c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f227406cbb0 0x7f22740713f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f22740cdb90 con 0x7f227406cbb0 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f22740b33c0 con 0x7f22740b4870 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f22740b3660 con 0x7f2274144930 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f227406cbb0 0x7f22740713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f227406cbb0 0x7f22740713f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33484/0 (socket says 172.21.15.43:33484) 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 -- 172.21.15.43:0/872879299 learned_addr learned my addr 172.21.15.43:0/872879299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a1d6640 1 --2- 172.21.15.43:0/872879299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2274144930 0x7f22740b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f22799d5640 1 --2- 172.21.15.43:0/872879299 >> v2:172.21.15.113:3300/0 conn(0x7f22740b4870 0x7f227406c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 -- 172.21.15.43:0/872879299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2274144930 msgr2=0x7f22740b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 --2- 172.21.15.43:0/872879299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2274144930 0x7f22740b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 -- 172.21.15.43:0/872879299 >> v2:172.21.15.113:3300/0 conn(0x7f22740b4870 msgr2=0x7f227406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 --2- 172.21.15.43:0/872879299 >> v2:172.21.15.113:3300/0 conn(0x7f22740b4870 0x7f227406c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 -- 172.21.15.43:0/872879299 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22740b3940 con 0x7f227406cbb0 2024-03-30T13:14:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227a9d7640 1 --2- 172.21.15.43:0/872879299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f22740713f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f226c009e50 tx=0x7f226c01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66df96a926bbb99e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f22791d4640 1 -- 172.21.15.43:0/872879299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0ac070 con 0x7f227406cbb0 2024-03-30T13:14:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f22791d4640 1 -- 172.21.15.43:0/872879299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c0a8440 con 0x7f227406cbb0 2024-03-30T13:14:33.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f22791d4640 1 -- 172.21.15.43:0/872879299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0a8810 con 0x7f227406cbb0 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 -- 172.21.15.43:0/872879299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 msgr2=0x7f22740713f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 --2- 172.21.15.43:0/872879299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f22740713f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f226c009e50 tx=0x7f226c01c6a0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 -- 172.21.15.43:0/872879299 shutdown_connections 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 --2- 172.21.15.43:0/872879299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f22740713f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 --2- 172.21.15.43:0/872879299 >> v2:172.21.15.113:3300/0 conn(0x7f22740b4870 0x7f227406c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 --2- 172.21.15.43:0/872879299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2274144930 0x7f22740b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.554+0000 7f227c43a640 1 -- 172.21.15.43:0/872879299 >> 172.21.15.43:0/872879299 conn(0x7f22740d00a0 msgr2=0x7f22740d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- 172.21.15.43:0/872879299 shutdown_connections 2024-03-30T13:14:33.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- 172.21.15.43:0/872879299 wait complete. 2024-03-30T13:14:33.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 Processor -- start 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- start start 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f227415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22740b4870 0x7f22740a9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2274144930 0x7f22740aa200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22740ce180 con 0x7f227406cbb0 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22740ce000 con 0x7f2274144930 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f22740ce300 con 0x7f22740b4870 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f227415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f227415a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33488/0 (socket says 172.21.15.43:33488) 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 -- 172.21.15.43:0/2534552835 learned_addr learned my addr 172.21.15.43:0/2534552835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a9d7640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2274144930 0x7f22740aa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 -- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22740b4870 msgr2=0x7f22740a9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22740b4870 0x7f22740a9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 -- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2274144930 msgr2=0x7f22740aa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2274144930 0x7f22740aa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 -- 172.21.15.43:0/2534552835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2274142f40 con 0x7f227406cbb0 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227a1d6640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f227415a590 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f227000d620 tx=0x7f227000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f226affd640 1 -- 172.21.15.43:0/2534552835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000b620 con 0x7f227406cbb0 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f226affd640 1 -- 172.21.15.43:0/2534552835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2270019070 con 0x7f227406cbb0 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2274143220 con 0x7f227406cbb0 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f226affd640 1 -- 172.21.15.43:0/2534552835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227000ba20 con 0x7f227406cbb0 2024-03-30T13:14:33.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.558+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22741437e0 con 0x7f227406cbb0 2024-03-30T13:14:33.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f226affd640 1 -- 172.21.15.43:0/2534552835 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f227000bbc0 con 0x7f227406cbb0 2024-03-30T13:14:33.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f226affd640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2258067fc0 0x7f225806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f226affd640 1 -- 172.21.15.43:0/2534552835 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f2270086b90 con 0x7f227406cbb0 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f22799d5640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2258067fc0 0x7f225806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 Processor -- start 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- start start 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f22799d5640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2258067fc0 0x7f225806a480 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f227415b320 tx=0x7f2264008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d45a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d410700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d411020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd6d25aaa0 con 0x55fd6d410c40 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fd6d1e1af0 con 0x55fd6d45a8e0 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fd6d256a50 con 0x55fd6d459fc0 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d411020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d411020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33494/0 (socket says 172.21.15.43:33494) 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 -- 172.21.15.43:0/4012137346 learned_addr learned my addr 172.21.15.43:0/4012137346 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a1d6640 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d45a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 -- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 msgr2=0x55fd6d45a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d45a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f22799d5640 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d410700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 -- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 msgr2=0x55fd6d410700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d410700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 -- 172.21.15.43:0/4012137346 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd6d1dfd00 con 0x55fd6d410c40 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f22799d5640 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d410700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227a9d7640 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d411020 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f226c01cf40 tx=0x7f226c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586352ed6026e26a server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f2257fff640 1 -- 172.21.15.43:0/4012137346 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0c1070 con 0x55fd6d410c40 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f2257fff640 1 -- 172.21.15.43:0/4012137346 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c0091d0 con 0x55fd6d410c40 2024-03-30T13:14:33.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f2257fff640 1 -- 172.21.15.43:0/4012137346 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226c0ac040 con 0x55fd6d410c40 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- 172.21.15.43:0/4012137346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 msgr2=0x55fd6d411020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d411020 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f226c01cf40 tx=0x7f226c004ac0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- 172.21.15.43:0/4012137346 shutdown_connections 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d411020 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d410700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 --2- 172.21.15.43:0/4012137346 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d45a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- 172.21.15.43:0/4012137346 >> 172.21.15.43:0/4012137346 conn(0x55fd6d4087e0 msgr2=0x55fd6d40abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- 172.21.15.43:0/4012137346 shutdown_connections 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.562+0000 7f227f5bc000 1 -- 172.21.15.43:0/4012137346 wait complete. 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 -- start start 2024-03-30T13:14:33.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 Processor -- start 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d459890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d416910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d416e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd6d222270 con 0x55fd6d410c40 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55fd6d2186a0 con 0x55fd6d45a8e0 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fd6d1f2880 con 0x55fd6d459fc0 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 10 client.0 _ll_register_callbacks cb 0x55fd6d3f7490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d459890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d459890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33496/0 (socket says 172.21.15.43:33496) 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 -- 172.21.15.43:0/1510533477 learned_addr learned my addr 172.21.15.43:0/1510533477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22799d5640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d416910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 -- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 msgr2=0x55fd6d416910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d416910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 -- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 msgr2=0x55fd6d416e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a9d7640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d416e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d416e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 -- 172.21.15.43:0/1510533477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd6d21f730 con 0x55fd6d410c40 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22799d5640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d416910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a9d7640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d416e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227a1d6640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d459890 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2270052e40 tx=0x7f2270056af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c34283a57aaf596 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22557fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22557fa640 1 -- 172.21.15.43:0/1510533477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227005f610 con 0x55fd6d410c40 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22557fa640 1 -- 172.21.15.43:0/1510533477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f227005d0b0 con 0x55fd6d410c40 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226c0091d0 con 0x55fd6d410c40 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22557fa640 1 -- 172.21.15.43:0/1510533477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f227005d2b0 con 0x55fd6d410c40 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 10 client.6606 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55fd6d212ca0 con 0x55fd6d410c40 2024-03-30T13:14:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22557fa640 1 -- 172.21.15.43:0/1510533477 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c0091d0 con 0x55fd6d410c40 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22557fa640 1 -- 172.21.15.43:0/1510533477 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f227006d070 con 0x55fd6d410c40 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 10 client.6606 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 10 client.6606 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 20 client.6606 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d417b90 0x55fd6d46c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 4 client.6606 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd6d1f5c90 con 0x55fd6d417b90 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22799d5640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d417b90 0x55fd6d46c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22799d5640 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d417b90 0x55fd6d46c280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23a7ee043d578107 server_cookie=2b406d11f8706cde in_seq=0 out_seq=0 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.566+0000 7f22557fa640 10 client.6606 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f22557fa640 1 -- 172.21.15.43:0/1510533477 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55fd6d1f5c90 con 0x55fd6d417b90 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f22557fa640 10 client.6606 handle_command_reply: tid=0 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 client.6606 shutdown 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 msgr2=0x55fd6d459890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d459890 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f2270052e40 tx=0x7f2270056af0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 shutdown_connections 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d417b90 0x55fd6d46c280 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d416e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d416910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f22557fa640 0 client.6606 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 --2- 172.21.15.43:0/1510533477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d459890 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 >> 172.21.15.43:0/1510533477 conn(0x55fd6d4087e0 msgr2=0x55fd6d4118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 shutdown_connections 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 1 -- 172.21.15.43:0/1510533477 wait complete. 2024-03-30T13:14:33.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.570+0000 7f227f5bc000 20 client.6606 trim_cache size 0 max 16384 2024-03-30T13:14:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 Processor -- start 2024-03-30T13:14:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 -- start start 2024-03-30T13:14:33.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d410c40 0x55fd6d46f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d459fc0 0x55fd6d439080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d4395c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f22741ffcb0 con 0x55fd6d459fc0 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f22741404c0 con 0x55fd6d45a8e0 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f22741f5ce0 con 0x55fd6d410c40 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d459fc0 0x55fd6d439080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d459fc0 0x55fd6d439080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33510/0 (socket says 172.21.15.43:33510) 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 -- 172.21.15.43:0/2809236847 learned_addr learned my addr 172.21.15.43:0/2809236847 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227a1d6640 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d410c40 0x55fd6d46f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f227a9d7640 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d4395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 -- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d410c40 msgr2=0x55fd6d46f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d410c40 0x55fd6d46f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 -- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 msgr2=0x55fd6d4395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d4395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.582+0000 7f22799d5640 1 -- 172.21.15.43:0/2809236847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd6d1e1d00 con 0x55fd6d459fc0 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22799d5640 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d459fc0 0x55fd6d439080 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2264006aa0 tx=0x7f2264013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cedac573c0a056 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22577fe640 1 -- 172.21.15.43:0/2809236847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226402b070 con 0x55fd6d459fc0 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22577fe640 1 -- 172.21.15.43:0/2809236847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2264014a80 con 0x55fd6d459fc0 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22577fe640 1 -- 172.21.15.43:0/2809236847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2264014e30 con 0x55fd6d459fc0 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2809236847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d459fc0 msgr2=0x55fd6d439080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d459fc0 0x55fd6d439080 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f2264006aa0 tx=0x7f2264013000 comp rx=0 tx=0).stop 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2809236847 shutdown_connections 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d4395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d459fc0 0x55fd6d439080 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2809236847 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d410c40 0x55fd6d46f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2809236847 >> 172.21.15.43:0/2809236847 conn(0x55fd6d4087e0 msgr2=0x55fd6d439d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2809236847 shutdown_connections 2024-03-30T13:14:33.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2809236847 wait complete. 2024-03-30T13:14:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- start start 2024-03-30T13:14:33.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 Processor -- start 2024-03-30T13:14:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d439370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d46d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d46d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55fd6d222270 con 0x55fd6d410c40 2024-03-30T13:14:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f227413fe70 con 0x55fd6d45a8e0 2024-03-30T13:14:33.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55fd6d25caf0 con 0x55fd6d459fc0 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 10 client.0 _ll_register_callbacks cb 0x7f2270057130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d439370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d439370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33522/0 (socket says 172.21.15.43:33522) 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 -- 172.21.15.43:0/2981531127 learned_addr learned my addr 172.21.15.43:0/2981531127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22799d5640 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d46d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a9d7640 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d46d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 -- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 msgr2=0x55fd6d46d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d46d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 -- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 msgr2=0x55fd6d46d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d46d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 -- 172.21.15.43:0/2981531127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd6d2dbf70 con 0x55fd6d410c40 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227a1d6640 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d439370 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f227005f300 tx=0x7f2270052d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569e4d198b37b551 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22557fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22557fa640 1 -- 172.21.15.43:0/2981531127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227005f610 con 0x55fd6d410c40 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22557fa640 1 -- 172.21.15.43:0/2981531127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f22700530a0 con 0x55fd6d410c40 2024-03-30T13:14:33.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2264014a80 con 0x55fd6d410c40 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22557fa640 1 -- 172.21.15.43:0/2981531127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2270075300 con 0x55fd6d410c40 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 10 client.6612 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55fd6d201760 con 0x55fd6d410c40 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22557fa640 1 -- 172.21.15.43:0/2981531127 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2264014a80 con 0x55fd6d410c40 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22557fa640 1 -- 172.21.15.43:0/2981531127 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f227006d070 con 0x55fd6d410c40 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 10 client.6612 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 10 client.6612 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 20 client.6612 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d4cb760 0x55fd6d4cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 4 client.6612 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628426}] 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628426}) -- 0x55fd6d267fe0 con 0x55fd6d4cb760 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.586+0000 7f22799d5640 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d4cb760 0x55fd6d4cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f22799d5640 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d4cb760 0x55fd6d4cbb70 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1822df73d26d4df1 server_cookie=885dff3f84cb9e47 in_seq=0 out_seq=0 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f22557fa640 10 client.6612 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f22557fa640 1 -- 172.21.15.43:0/2981531127 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3802 (crc 0 0 0) 0x55fd6d267fe0 con 0x55fd6d4cb760 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f22557fa640 10 client.6612 handle_command_reply: tid=0 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 client.6612 shutdown 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 msgr2=0x55fd6d439370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d439370 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f227005f300 tx=0x7f2270052d50 comp rx=0 tx=0).stop 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 shutdown_connections 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55fd6d4cb760 0x55fd6d4cbb70 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55fd6d45a8e0 0x55fd6d46d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55fd6d459fc0 0x55fd6d46d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f22557fa640 0 client.6612 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 --2- 172.21.15.43:0/2981531127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fd6d410c40 0x55fd6d439370 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 >> 172.21.15.43:0/2981531127 conn(0x55fd6d4087e0 msgr2=0x55fd6d43a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 shutdown_connections 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 1 -- 172.21.15.43:0/2981531127 wait complete. 2024-03-30T13:14:33.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227f5bc000 20 client.6612 trim_cache size 0 max 16384 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-sysfs.h", 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628426, 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.608111+0000", 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:57.019831+0000", 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:33.599 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:33.600 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 77, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:57.019831+0000", 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:33.601 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 77, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.608111+0000" 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:33.602 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 77, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.608111+0000" 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 204, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 42, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:33.603 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:33.604 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:33.605 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.606 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:33.607 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:33.608 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.609 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:33.610 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2258067fc0 msgr2=0x7f225806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2258067fc0 0x7f225806a480 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f227415b320 tx=0x7f2264008040 comp rx=0 tx=0).stop 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 msgr2=0x7f227415a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f227415a590 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f227000d620 tx=0x7f227000dae0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 shutdown_connections 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2258067fc0 0x7f225806a480 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2274144930 0x7f22740aa200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f22740b4870 0x7f22740a9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 --2- 172.21.15.43:0/2534552835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f227406cbb0 0x7f227415a590 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 >> 172.21.15.43:0/2534552835 conn(0x7f22740d00a0 msgr2=0x7f2274073080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 shutdown_connections 2024-03-30T13:14:33.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.590+0000 7f227c43a640 1 -- 172.21.15.43:0/2534552835 wait complete. 2024-03-30T13:14:33.613 DEBUG:tasks.cephfs.test_uninlining:getting inode info #17 of 23:1099511628409 2024-03-30T13:14:33.613 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628409 2024-03-30T13:14:33.613 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628409 2024-03-30T13:14:33.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff2a5221640 1 Processor -- start 2024-03-30T13:14:33.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff2a5221640 1 -- start start 2024-03-30T13:14:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff2a5221640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff2a015e4c0 0x7ff2a00b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff2a5221640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a00cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff2a5221640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2a006c850 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff2a5221640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff2a015e8a0 con 0x7ff2a015e4c0 2024-03-30T13:14:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff2a5221640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a006cb30 con 0x7ff2a006c070 2024-03-30T13:14:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29f59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff2a00cf4f0 0x7ff2a0161f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35792/0 (socket says 172.21.15.43:35792) 2024-03-30T13:14:33.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29f59e640 1 -- 172.21.15.43:0/14925528 learned_addr learned my addr 172.21.15.43:0/14925528 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29e59c640 1 --2- 172.21.15.43:0/14925528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a00cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/14925528 >> v2:172.21.15.113:3300/0 conn(0x7ff2a015e4c0 0x7ff2a00b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29dd9b640 1 -- 172.21.15.43:0/14925528 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 87936629 0 0) 0x7ff2a006c850 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29dd9b640 1 -- 172.21.15.43:0/14925528 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff288003530 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29dd9b640 1 -- 172.21.15.43:0/14925528 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3199707386 0 0) 0x7ff288003530 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29e59c640 1 -- 172.21.15.43:0/14925528 >> v2:172.21.15.113:3300/0 conn(0x7ff2a015e4c0 msgr2=0x7ff2a00b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29e59c640 1 --2- 172.21.15.43:0/14925528 >> v2:172.21.15.113:3300/0 conn(0x7ff2a015e4c0 0x7ff2a00b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29e59c640 1 -- 172.21.15.43:0/14925528 >> v1:172.21.15.43:6789/0 conn(0x7ff2a00cf4f0 legacy=0x7ff2a0161f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.722+0000 7ff29e59c640 1 -- 172.21.15.43:0/14925528 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a006ce10 con 0x7ff2a006c070 2024-03-30T13:14:33.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff29e59c640 1 --2- 172.21.15.43:0/14925528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a00cefb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2940097c0 tx=0x7ff29401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af3c50ae84ce9a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff29dd9b640 1 -- 172.21.15.43:0/14925528 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940b0070 con 0x7ff2a006c070 2024-03-30T13:14:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff29dd9b640 1 -- 172.21.15.43:0/14925528 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2940ab400 con 0x7ff2a006c070 2024-03-30T13:14:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff29dd9b640 1 -- 172.21.15.43:0/14925528 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298001a00 con 0x7ff2a006c070 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 -- 172.21.15.43:0/14925528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 msgr2=0x7ff2a00cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 --2- 172.21.15.43:0/14925528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a00cefb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2940097c0 tx=0x7ff29401c3c0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 -- 172.21.15.43:0/14925528 shutdown_connections 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 --2- 172.21.15.43:0/14925528 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a00cefb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 --2- 172.21.15.43:0/14925528 >> v2:172.21.15.113:3300/0 conn(0x7ff2a015e4c0 0x7ff2a00b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 -- 172.21.15.43:0/14925528 >> 172.21.15.43:0/14925528 conn(0x7ff2a00b6410 msgr2=0x7ff2a00d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 -- 172.21.15.43:0/14925528 shutdown_connections 2024-03-30T13:14:33.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 -- 172.21.15.43:0/14925528 wait complete. 2024-03-30T13:14:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 Processor -- start 2024-03-30T13:14:33.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.726+0000 7ff2a5221640 1 -- start start 2024-03-30T13:14:33.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a00cf4f0 0x7ff2a0153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2a015e4c0 0x7ff2a00c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a016b3d0 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a016b250 con 0x7ff2a015e4c0 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a016b550 con 0x7ff2a006c070 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a00cf4f0 0x7ff2a0153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a00cf4f0 0x7ff2a0153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33532/0 (socket says 172.21.15.43:33532) 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 -- 172.21.15.43:0/3468925641 learned_addr learned my addr 172.21.15.43:0/3468925641 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2a015e4c0 0x7ff2a00c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 -- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 msgr2=0x7ff2a015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a015f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 -- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2a015e4c0 msgr2=0x7ff2a00c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2a015e4c0 0x7ff2a00c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 -- 172.21.15.43:0/3468925641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a00ccfa0 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2a015e4c0 0x7ff2a00c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff29e59c640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a00cf4f0 0x7ff2a0153970 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff294000c10 tx=0x7ff294019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2877fe640 1 -- 172.21.15.43:0/3468925641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940ba070 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2877fe640 1 -- 172.21.15.43:0/3468925641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff294003c50 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2877fe640 1 -- 172.21.15.43:0/3468925641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940b0040 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a00cd280 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a00cd840 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2877fe640 1 -- 172.21.15.43:0/3468925641 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff2940bd070 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2877fe640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff28c067fc0 0x7ff28c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.730+0000 7ff2877fe640 1 -- 172.21.15.43:0/3468925641 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7ff29411fb00 con 0x7ff2a00cf4f0 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff28c067fc0 0x7ff28c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 Processor -- start 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- start start 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff28c067fc0 0x7ff28c06a480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff290009a50 tx=0x7ff290008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c10873150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 0x560c108292f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10829c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560c10677aa0 con 0x560c10872d70 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560c105feaf0 con 0x560c10829830 2024-03-30T13:14:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560c10673a50 con 0x560c10873690 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c10873150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c10873150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33536/0 (socket says 172.21.15.43:33536) 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 -- 172.21.15.43:0/4000132101 learned_addr learned my addr 172.21.15.43:0/4000132101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 -- 172.21.15.43:0/4000132101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 msgr2=0x560c108292f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/4000132101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 0x560c108292f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 -- 172.21.15.43:0/4000132101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 msgr2=0x560c10829c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/4000132101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10829c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 -- 172.21.15.43:0/4000132101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c105fcd00 con 0x560c10872d70 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/4000132101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c10873150 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff290011ce0 tx=0x7ff290014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2db5ec554eae82f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff284ff9640 1 -- 172.21.15.43:0/4000132101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290011d10 con 0x560c10872d70 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff284ff9640 1 -- 172.21.15.43:0/4000132101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff290026070 con 0x560c10872d70 2024-03-30T13:14:33.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff284ff9640 1 -- 172.21.15.43:0/4000132101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff290021640 con 0x560c10872d70 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- 172.21.15.43:0/4000132101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 msgr2=0x560c10873150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/4000132101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c10873150 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff290011ce0 tx=0x7ff290014000 comp rx=0 tx=0).stop 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- 172.21.15.43:0/4000132101 shutdown_connections 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/4000132101 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10829c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/4000132101 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 0x560c108292f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/4000132101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c10873150 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- 172.21.15.43:0/4000132101 >> 172.21.15.43:0/4000132101 conn(0x560c108257e0 msgr2=0x560c10827bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- 172.21.15.43:0/4000132101 shutdown_connections 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- 172.21.15.43:0/4000132101 wait complete. 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- start start 2024-03-30T13:14:33.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 Processor -- start 2024-03-30T13:14:33.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c1082ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c108582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560c1063f270 con 0x560c10873690 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x560c106356a0 con 0x560c10829830 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560c1060f880 con 0x560c10872d70 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff2a83a3000 10 client.0 _ll_register_callbacks cb 0x560c10814490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c108582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c108582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33542/0 (socket says 172.21.15.43:33542) 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.734+0000 7ff29f59e640 1 -- 172.21.15.43:0/3194574239 learned_addr learned my addr 172.21.15.43:0/3194574239 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c1082ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29e59c640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29f59e640 1 -- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 msgr2=0x560c10857da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29f59e640 1 -- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 msgr2=0x560c1082ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c1082ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29f59e640 1 -- 172.21.15.43:0/3194574239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c1063c730 con 0x560c10873690 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29e59c640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c108582e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d940 tx=0x7ff29800de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff7892aa8c072f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 1 -- 172.21.15.43:0/3194574239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29800bac0 con 0x560c10873690 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 1 -- 172.21.15.43:0/3194574239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7ff29800bc60 con 0x560c10873690 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff290026070 con 0x560c10873690 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 1 -- 172.21.15.43:0/3194574239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff29801f050 con 0x560c10873690 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 10 client.6624 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x560c1062fca0 con 0x560c10873690 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 1 -- 172.21.15.43:0/3194574239 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff290026070 con 0x560c10873690 2024-03-30T13:14:33.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 1 -- 172.21.15.43:0/3194574239 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7ff298022020 con 0x560c10873690 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 10 client.6624 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 10 client.6624 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 20 client.6624 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c1087a5d0 0x560c1087a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 4 client.6624 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c10612c90 con 0x560c1087a5d0 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c1087a5d0 0x560c1087a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c1087a5d0 0x560c1087a9e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5f9d0080fcff2d server_cookie=7f1d91c2e024b600 in_seq=0 out_seq=0 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 10 client.6624 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 1 -- 172.21.15.43:0/3194574239 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x560c10612c90 con 0x560c1087a5d0 2024-03-30T13:14:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 10 client.6624 handle_command_reply: tid=0 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 client.6624 shutdown 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 msgr2=0x560c108582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c108582e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff29800d940 tx=0x7ff29800de00 comp rx=0 tx=0).stop 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 shutdown_connections 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c1087a5d0 0x560c1087a9e0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c108582e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff27dffb640 0 client.6624 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3194574239 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c1082ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 >> 172.21.15.43:0/3194574239 conn(0x560c108257e0 msgr2=0x560c10828460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 shutdown_connections 2024-03-30T13:14:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3194574239 wait complete. 2024-03-30T13:14:33.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.738+0000 7ff2a83a3000 20 client.6624 trim_cache size 0 max 16384 2024-03-30T13:14:33.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 Processor -- start 2024-03-30T13:14:33.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- start start 2024-03-30T13:14:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10856d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c10857a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a02040c0 con 0x560c10873690 2024-03-30T13:14:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a0203120 con 0x560c10829830 2024-03-30T13:14:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a015cf20 con 0x560c10872d70 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c10857a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c10857a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33556/0 (socket says 172.21.15.43:33556) 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 -- 172.21.15.43:0/3915956607 learned_addr learned my addr 172.21.15.43:0/3915956607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 -- 172.21.15.43:0/3915956607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 msgr2=0x560c10857290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3915956607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 -- 172.21.15.43:0/3915956607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 msgr2=0x560c10856d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3915956607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10856d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 -- 172.21.15.43:0/3915956607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c105fed00 con 0x560c10873690 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff29f59e640 1 --2- 172.21.15.43:0/3915956607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c10857a90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff298001230 tx=0x7ff298007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1149eae1528f9d12 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff27ffff640 1 -- 172.21.15.43:0/3915956607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff29800bac0 con 0x560c10873690 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff27ffff640 1 -- 172.21.15.43:0/3915956607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff298017070 con 0x560c10873690 2024-03-30T13:14:33.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff27ffff640 1 -- 172.21.15.43:0/3915956607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff298012560 con 0x560c10873690 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3915956607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 msgr2=0x560c10857a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3915956607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c10857a90 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff298001230 tx=0x7ff298007000 comp rx=0 tx=0).stop 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3915956607 shutdown_connections 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3915956607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10873690 0x560c10857a90 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3915956607 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10872d70 0x560c10857290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/3915956607 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10856d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3915956607 >> 172.21.15.43:0/3915956607 conn(0x560c108257e0 msgr2=0x560c10889680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3915956607 shutdown_connections 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- 172.21.15.43:0/3915956607 wait complete. 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.754+0000 7ff2a83a3000 1 -- start start 2024-03-30T13:14:33.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 Processor -- start 2024-03-30T13:14:33.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10828fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c108e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 0x560c108e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x560c106356a0 con 0x560c10872d70 2024-03-30T13:14:33.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a015d570 con 0x560c10829830 2024-03-30T13:14:33.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x560c1063f270 con 0x560c10873690 2024-03-30T13:14:33.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 10 client.0 _ll_register_callbacks cb 0x7ff2980020d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c108e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c108e05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33562/0 (socket says 172.21.15.43:33562) 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 -- 172.21.15.43:0/2577779631 learned_addr learned my addr 172.21.15.43:0/2577779631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29f59e640 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 0x560c108e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10828fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 -- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 msgr2=0x560c108e0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 0x560c108e0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 -- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 msgr2=0x560c10828fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10828fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 -- 172.21.15.43:0/2577779631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c106f8f70 con 0x560c10872d70 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29e59c640 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c108e05f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff29401c930 tx=0x7ff294006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3b243f62a5b8e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 1 -- 172.21.15.43:0/2577779631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2940efe00 con 0x560c10872d70 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff298017070 con 0x560c10872d70 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 1 -- 172.21.15.43:0/2577779631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7ff2940f80f0 con 0x560c10872d70 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 1 -- 172.21.15.43:0/2577779631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff2940f3d50 con 0x560c10872d70 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 10 client.6630 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x560c1061e760 con 0x560c10872d70 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 1 -- 172.21.15.43:0/2577779631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff298017070 con 0x560c10872d70 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 1 -- 172.21.15.43:0/2577779631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7ff2940b3070 con 0x560c10872d70 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 10 client.6630 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 10 client.6630 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 20 client.6630 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c10876ad0 0x560c10876eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 4 client.6630 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628409}] 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628409}) -- 0x560c10684fe0 con 0x560c10876ad0 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c10876ad0 0x560c10876eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff29ed9d640 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c10876ad0 0x560c10876eb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a277ec63d4a3f31a server_cookie=823c57a7864e5e0d in_seq=0 out_seq=0 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 10 client.6630 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 1 -- 172.21.15.43:0/2577779631 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3799 (crc 0 0 0) 0x560c10684fe0 con 0x560c10876ad0 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 10 client.6630 handle_command_reply: tid=0 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 client.6630 shutdown 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 msgr2=0x560c108e05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c108e05f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff29401c930 tx=0x7ff294006e20 comp rx=0 tx=0).stop 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 shutdown_connections 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x560c10876ad0 0x560c10876eb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x560c10873690 0x560c108e0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff27dffb640 0 client.6630 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560c10872d70 0x560c108e05f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 --2- 172.21.15.43:0/2577779631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x560c10829830 0x560c10828fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 >> 172.21.15.43:0/2577779631 conn(0x560c108257e0 msgr2=0x560c10889f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 shutdown_connections 2024-03-30T13:14:33.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 1 -- 172.21.15.43:0/2577779631 wait complete. 2024-03-30T13:14:33.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.758+0000 7ff2a83a3000 20 client.6630 trim_cache size 0 max 16384 2024-03-30T13:14:33.771 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:33.771 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/dcache.c", 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628409, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.611150+0000", 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.928846+0000", 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:33.772 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 3670, 2024-03-30T13:14:33.773 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.928846+0000", 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 4, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:33.774 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.775 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3670, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.611150+0000" 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 3670, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.776 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.611150+0000" 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 208, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 8, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:33.777 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:33.778 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:33.779 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.780 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.781 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.782 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.783 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:33.784 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:33.785 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff28c067fc0 msgr2=0x7ff28c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff28c067fc0 0x7ff28c06a480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff290009a50 tx=0x7ff290008040 comp rx=0 tx=0).stop 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a00cf4f0 msgr2=0x7ff2a0153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a00cf4f0 0x7ff2a0153970 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7ff294000c10 tx=0x7ff294019160 comp rx=0 tx=0).stop 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 shutdown_connections 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff28c067fc0 0x7ff28c06a480 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff2a015e4c0 0x7ff2a00c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2a00cf4f0 0x7ff2a0153970 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 --2- 172.21.15.43:0/3468925641 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff2a006c070 0x7ff2a015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 >> 172.21.15.43:0/3468925641 conn(0x7ff2a00b6410 msgr2=0x7ff2a0156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 shutdown_connections 2024-03-30T13:14:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.762+0000 7ff2a5221640 1 -- 172.21.15.43:0/3468925641 wait complete. 2024-03-30T13:14:33.788 DEBUG:tasks.cephfs.test_uninlining:getting inode info #18 of 23:1099511628417 2024-03-30T13:14:33.788 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628417 2024-03-30T13:14:33.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628417 2024-03-30T13:14:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8699af8640 1 Processor -- start 2024-03-30T13:14:33.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8699af8640 1 -- start start 2024-03-30T13:14:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8699af8640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f86940ad8e0 0x7f869406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8699af8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86940b4360 con 0x7f869406ff20 2024-03-30T13:14:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8699af8640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86940b4640 con 0x7f8694147d90 2024-03-30T13:14:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8699af8640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8694070390 con 0x7f86940ad8e0 2024-03-30T13:14:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f869406ff20 0x7f86940b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35808/0 (socket says 172.21.15.43:35808) 2024-03-30T13:14:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 -- 172.21.15.43:0/2818136649 learned_addr learned my addr 172.21.15.43:0/2818136649 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:33.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f868bfff640 1 --2- 172.21.15.43:0/2818136649 >> v2:172.21.15.113:3300/0 conn(0x7f86940ad8e0 0x7f869406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4227957595 0 0) 0x7f86940b4360 con 0x7f869406ff20 2024-03-30T13:14:33.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8680003530 con 0x7f869406ff20 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1272631420 0 0) 0x7f86940b4640 con 0x7f8694147d90 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 --> v1:172.21.15.113:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86940b4360 con 0x7f8694147d90 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 718069251 0 0) 0x7f8680003530 con 0x7f869406ff20 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 >> v2:172.21.15.113:3300/0 conn(0x7f86940ad8e0 msgr2=0x7f869406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 --2- 172.21.15.43:0/2818136649 >> v2:172.21.15.113:3300/0 conn(0x7f86940ad8e0 0x7f869406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 >> v1:172.21.15.113:6789/0 conn(0x7f8694147d90 legacy=0x7f869406b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86940b4920 con 0x7f869406ff20 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f868c001a00 con 0x7f869406ff20 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 >> v1:172.21.15.43:6789/0 conn(0x7f869406ff20 legacy=0x7f86940b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680003b50 0x7f8680003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680004810 0x7f8680004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f868000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8694070390 con 0x7f86800051d0 2024-03-30T13:14:33.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f868000c050 con 0x7f8680003b50 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f868000c250 con 0x7f8680004810 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f868000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f868bfff640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680004810 0x7f8680004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f86937fe640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680003b50 0x7f8680003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 -- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680004810 msgr2=0x7f8680004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8680004810 0x7f8680004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 -- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680003b50 msgr2=0x7f8680003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680003b50 0x7f8680003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f8693fff640 1 -- 172.21.15.43:0/2818136649 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8680003530 con 0x7f86800051d0 2024-03-30T13:14:33.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.898+0000 7f86937fe640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8680003b50 0x7f8680003f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8693fff640 1 -- 172.21.15.43:0/2818136649 reap_dead start 2024-03-30T13:14:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8693fff640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f868000b9d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f868c00a810 tx=0x7f868c01fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f2cf1a8e2a9b442 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c00a430 con 0x7f86800051d0 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f868c0aa6b0 con 0x7f86800051d0 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8692ffd640 1 -- 172.21.15.43:0/2818136649 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0aaaa0 con 0x7f86800051d0 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- 172.21.15.43:0/2818136649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 msgr2=0x7f868000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f868000b9d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f868c00a810 tx=0x7f868c01fa80 comp rx=0 tx=0).stop 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- 172.21.15.43:0/2818136649 shutdown_connections 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 --2- 172.21.15.43:0/2818136649 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f868000b9d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- 172.21.15.43:0/2818136649 >> 172.21.15.43:0/2818136649 conn(0x7f86940d1070 msgr2=0x7f86940d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- 172.21.15.43:0/2818136649 shutdown_connections 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- 172.21.15.43:0/2818136649 wait complete. 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 Processor -- start 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- start start 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f86940af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86941f8280 0x7f86940afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8694151b90 0x7f8694152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f86940b3fe0 con 0x7f86800051d0 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8694148170 con 0x7f86941f8280 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8699af8640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f86940b4160 con 0x7f8694151b90 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f86940af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f86940af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33578/0 (socket says 172.21.15.43:33578) 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 -- 172.21.15.43:0/2364858480 learned_addr learned my addr 172.21.15.43:0/2364858480 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f8693fff640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8694151b90 0x7f8694152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f868bfff640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86941f8280 0x7f86940afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 -- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8694151b90 msgr2=0x7f8694152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8694151b90 0x7f8694152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 -- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86941f8280 msgr2=0x7f86940afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86941f8280 0x7f86940afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 -- 172.21.15.43:0/2364858480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869413c830 con 0x7f86800051d0 2024-03-30T13:14:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.902+0000 7f86937fe640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f86940af530 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f868400d770 tx=0x7f868400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8690ff9640 1 -- 172.21.15.43:0/2364858480 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684016070 con 0x7f86800051d0 2024-03-30T13:14:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8690ff9640 1 -- 172.21.15.43:0/2364858480 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8684007220 con 0x7f86800051d0 2024-03-30T13:14:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8690ff9640 1 -- 172.21.15.43:0/2364858480 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86840075f0 con 0x7f86800051d0 2024-03-30T13:14:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869413cad0 con 0x7f86800051d0 2024-03-30T13:14:33.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86941f7a80 con 0x7f86800051d0 2024-03-30T13:14:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8690ff9640 1 -- 172.21.15.43:0/2364858480 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f86840069f0 con 0x7f86800051d0 2024-03-30T13:14:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8690ff9640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8674067ff0 0x7f867406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f8690ff9640 1 -- 172.21.15.43:0/2364858480 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f8684086750 con 0x7f86800051d0 2024-03-30T13:14:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f868bfff640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8674067ff0 0x7f867406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 Processor -- start 2024-03-30T13:14:33.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f868bfff640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8674067ff0 0x7f867406a4b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f869413fd60 tx=0x7f867c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 -- start start 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f91f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f8d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8d5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a6f723aa0 con 0x557a6f91f690 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557a6f6aaaf0 con 0x557a6f8d5830 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x557a6f71fa50 con 0x557a6f91ed70 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f868bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f8d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f868bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f8d52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33592/0 (socket says 172.21.15.43:33592) 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.906+0000 7f868bfff640 1 -- 172.21.15.43:0/2748637643 learned_addr learned my addr 172.21.15.43:0/2748637643 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8d5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86937fe640 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f91f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 -- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 msgr2=0x557a6f91f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f91f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 -- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 msgr2=0x557a6f8d5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8d5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 -- 172.21.15.43:0/2748637643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6f6a8d00 con 0x557a6f91f690 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f8d52f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f867c004350 tx=0x7f867c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3520b3231758dc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86897fa640 1 -- 172.21.15.43:0/2748637643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c029070 con 0x557a6f91f690 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86897fa640 1 -- 172.21.15.43:0/2748637643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867c01f050 con 0x557a6f91f690 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86897fa640 1 -- 172.21.15.43:0/2748637643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867c003da0 con 0x557a6f91f690 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- 172.21.15.43:0/2748637643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 msgr2=0x557a6f8d52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f8d52f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f867c004350 tx=0x7f867c016000 comp rx=0 tx=0).stop 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- 172.21.15.43:0/2748637643 shutdown_connections 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8d5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f8d52f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2748637643 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f91f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- 172.21.15.43:0/2748637643 >> 172.21.15.43:0/2748637643 conn(0x557a6f8d17e0 msgr2=0x557a6f8d3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- 172.21.15.43:0/2748637643 shutdown_connections 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- 172.21.15.43:0/2748637643 wait complete. 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- start start 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 Processor -- start 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8dac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f903da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f9042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a6f6eb270 con 0x557a6f91f690 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x557a6f6e16a0 con 0x557a6f8d5830 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x557a6f6bb880 con 0x557a6f91ed70 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 10 client.0 _ll_register_callbacks cb 0x557a6f8c0490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f9042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f9042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33606/0 (socket says 172.21.15.43:33606) 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 -- 172.21.15.43:0/1175048220 learned_addr learned my addr 172.21.15.43:0/1175048220 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86937fe640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8dac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f903da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 -- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 msgr2=0x557a6f903da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f903da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 -- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 msgr2=0x557a6f8dac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8dac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 -- 172.21.15.43:0/1175048220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6f6e8730 con 0x557a6f91f690 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f868bfff640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f903da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f8693fff640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f9042e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f868c020320 tx=0x7f868c01e2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2697bd9c4de44dfa server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86727fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86727fc640 1 -- 172.21.15.43:0/1175048220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868c0bc060 con 0x557a6f91f690 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f867c01f050 con 0x557a6f91f690 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86727fc640 1 -- 172.21.15.43:0/1175048220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f868c0aa4b0 con 0x557a6f91f690 2024-03-30T13:14:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f86727fc640 1 -- 172.21.15.43:0/1175048220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f868c0c2050 con 0x557a6f91f690 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 10 client.6642 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.910+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x557a6f6dbca0 con 0x557a6f91f690 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86727fc640 1 -- 172.21.15.43:0/1175048220 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f867c01f050 con 0x557a6f91f690 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86727fc640 1 -- 172.21.15.43:0/1175048220 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f868c0c5020 con 0x557a6f91f690 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 10 client.6642 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 10 client.6642 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 20 client.6642 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f9265d0 0x557a6f9269b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 4 client.6642 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a6f6bec90 con 0x557a6f9265d0 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86937fe640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f9265d0 0x557a6f9269b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86727fc640 10 client.6642 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86937fe640 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f9265d0 0x557a6f9269b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be6b5399f6e75a2 server_cookie=5db257d4588087a2 in_seq=0 out_seq=0 2024-03-30T13:14:33.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86727fc640 1 -- 172.21.15.43:0/1175048220 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x557a6f6bec90 con 0x557a6f9265d0 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86727fc640 10 client.6642 handle_command_reply: tid=0 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 client.6642 shutdown 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 msgr2=0x557a6f9042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f9042e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f868c020320 tx=0x7f868c01e2c0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 shutdown_connections 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f9265d0 0x557a6f9269b0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f91f690 0x557a6f9042e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91ed70 0x557a6f903da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 --2- 172.21.15.43:0/1175048220 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f8d5830 0x557a6f8dac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f86727fc640 0 client.6642 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 >> 172.21.15.43:0/1175048220 conn(0x557a6f8d17e0 msgr2=0x557a6f8d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 shutdown_connections 2024-03-30T13:14:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 1 -- 172.21.15.43:0/1175048220 wait complete. 2024-03-30T13:14:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.914+0000 7f869cc7a000 20 client.6642 trim_cache size 0 max 16384 2024-03-30T13:14:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 Processor -- start 2024-03-30T13:14:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- start start 2024-03-30T13:14:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f8d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f8d4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f8d4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8694201800 con 0x557a6f8d5830 2024-03-30T13:14:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f86942007e0 con 0x557a6f91ed70 2024-03-30T13:14:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f86941ffb40 con 0x557a6f91f690 2024-03-30T13:14:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f8d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f8d3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33614/0 (socket says 172.21.15.43:33614) 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 -- 172.21.15.43:0/2768806021 learned_addr learned my addr 172.21.15.43:0/2768806021 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f868bfff640 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f8d4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 -- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 msgr2=0x557a6f8d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f8693fff640 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f8d4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f8d4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 -- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 msgr2=0x557a6f8d4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f8d4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 -- 172.21.15.43:0/2768806021 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a6f6aad00 con 0x557a6f8d5830 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f868bfff640 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f8d4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f86937fe640 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f8d3bd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f868405ed30 tx=0x7f86840849d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61a12635af5b7c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f8688ff9640 1 -- 172.21.15.43:0/2768806021 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868401f040 con 0x557a6f8d5830 2024-03-30T13:14:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f8688ff9640 1 -- 172.21.15.43:0/2768806021 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8684054980 con 0x557a6f8d5830 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f8688ff9640 1 -- 172.21.15.43:0/2768806021 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8684006ce0 con 0x557a6f8d5830 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- 172.21.15.43:0/2768806021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 msgr2=0x557a6f8d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f8d3bd0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f868405ed30 tx=0x7f86840849d0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- 172.21.15.43:0/2768806021 shutdown_connections 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f8d4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f8d4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2768806021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f8d3bd0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- 172.21.15.43:0/2768806021 >> 172.21.15.43:0/2768806021 conn(0x557a6f8d17e0 msgr2=0x557a6f935f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- 172.21.15.43:0/2768806021 shutdown_connections 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- 172.21.15.43:0/2768806021 wait complete. 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 -- start start 2024-03-30T13:14:33.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.930+0000 7f869cc7a000 1 Processor -- start 2024-03-30T13:14:33.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f9910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f9915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f922ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x557a6f6eb270 con 0x557a6f8d5830 2024-03-30T13:14:33.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8694200130 con 0x557a6f91ed70 2024-03-30T13:14:33.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x557a6f7dfb00 con 0x557a6f91f690 2024-03-30T13:14:33.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 10 client.0 _ll_register_callbacks cb 0x7f8680003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f9910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f9910a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33630/0 (socket says 172.21.15.43:33630) 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 -- 172.21.15.43:0/2323668141 learned_addr learned my addr 172.21.15.43:0/2323668141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f868bfff640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f9915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8693fff640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f922ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 -- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 msgr2=0x557a6f922ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f922ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 -- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 msgr2=0x557a6f9915e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f9915e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 -- 172.21.15.43:0/2323668141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a6f7a4f70 con 0x557a6f8d5830 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8693fff640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f922ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86937fe640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f9910a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8684052ae0 tx=0x7f8684014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3a641a941ac2116 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:33.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 1 -- 172.21.15.43:0/2323668141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f868401f070 con 0x557a6f8d5830 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8684054980 con 0x557a6f8d5830 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 1 -- 172.21.15.43:0/2323668141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f868407fd20 con 0x557a6f8d5830 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 1 -- 172.21.15.43:0/2323668141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8684052cf0 con 0x557a6f8d5830 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 10 client.6648 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x557a6f6ca760 con 0x557a6f8d5830 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 1 -- 172.21.15.43:0/2323668141 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8684054980 con 0x557a6f8d5830 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 1 -- 172.21.15.43:0/2323668141 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f868405bbc0 con 0x557a6f8d5830 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 10 client.6648 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 10 client.6648 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 20 client.6648 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f923430 0x557a6f923840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 4 client.6648 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628417}] 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628417}) -- 0x557a6f730fe0 con 0x557a6f923430 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f868bfff640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f923430 0x557a6f923840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 10 client.6648 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f868bfff640 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f923430 0x557a6f923840 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=508506b758e530bc server_cookie=8d5493aca20651e0 in_seq=0 out_seq=0 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 1 -- 172.21.15.43:0/2323668141 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3806 (crc 0 0 0) 0x557a6f730fe0 con 0x557a6f923430 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 10 client.6648 handle_command_reply: tid=0 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 client.6648 shutdown 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 msgr2=0x557a6f9910a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f9910a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8684052ae0 tx=0x7f8684014040 comp rx=0 tx=0).stop 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 shutdown_connections 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x557a6f923430 0x557a6f923840 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x557a6f91f690 0x557a6f922ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x557a6f91ed70 0x557a6f9915e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f86727fc640 0 client.6648 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 --2- 172.21.15.43:0/2323668141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557a6f8d5830 0x557a6f9910a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 >> 172.21.15.43:0/2323668141 conn(0x557a6f8d17e0 msgr2=0x557a6f936620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 shutdown_connections 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 1 -- 172.21.15.43:0/2323668141 wait complete. 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f869cc7a000 20 client.6648 trim_cache size 0 max 16384 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-bufmap.h", 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628417, 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.612432+0000", 2024-03-30T13:14:33.946 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.981702+0000", 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:33.947 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 925, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:33.948 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.981702+0000", 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:33.949 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 925, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.612432+0000" 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:33.950 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 925, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.612432+0000" 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 212, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 24, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:33.951 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:33.952 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:33.953 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:33.954 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.955 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:33.956 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:33.957 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:33.958 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:33.959 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:33.959 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8674067ff0 msgr2=0x7f867406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8674067ff0 0x7f867406a4b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f869413fd60 tx=0x7f867c008040 comp rx=0 tx=0).stop 2024-03-30T13:14:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 msgr2=0x7f86940af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:33.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f86940af530 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f868400d770 tx=0x7f868400dc30 comp rx=0 tx=0).stop 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 shutdown_connections 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8674067ff0 0x7f867406a4b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8694151b90 0x7f8694152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f86941f8280 0x7f86940afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 --2- 172.21.15.43:0/2364858480 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800051d0 0x7f86940af530 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 >> 172.21.15.43:0/2364858480 conn(0x7f86940d1070 msgr2=0x7f86940ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 shutdown_connections 2024-03-30T13:14:33.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:33.934+0000 7f8699af8640 1 -- 172.21.15.43:0/2364858480 wait complete. 2024-03-30T13:14:33.961 DEBUG:tasks.cephfs.test_uninlining:getting inode info #19 of 23:1099511628421 2024-03-30T13:14:33.961 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628421 2024-03-30T13:14:33.962 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628421 2024-03-30T13:14:34.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1faf163640 1 Processor -- start 2024-03-30T13:14:34.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1faf163640 1 -- start start 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1faf163640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa815f740 0x7f1fa8073000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1faf163640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1fa8073540 0x7f1fa81620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1faf163640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fa80d0b20 con 0x7f1fa80d0740 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1faf163640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f1fa806c490 con 0x7f1fa8073540 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1faf163640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa815f2b0 con 0x7f1fa815f740 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fad700640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1fa8073540 0x7f1fa81620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fad700640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f1fa8073540 0x7f1fa81620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:44076/0 (socket says 172.21.15.43:44076) 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fad700640 1 -- 172.21.15.43:0/4068054706 learned_addr learned my addr 172.21.15.43:0/4068054706 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa815f740 0x7f1fa8073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fad700640 1 -- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa815f740 msgr2=0x7f1fa8073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fad700640 1 --2- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa815f740 0x7f1fa8073000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fad700640 1 -- 172.21.15.43:0/4068054706 >> v1:172.21.15.113:6789/0 conn(0x7f1fa80d0740 legacy=0x7f1fa806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.074+0000 7f1fad700640 1 -- 172.21.15.43:0/4068054706 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa80ca850 con 0x7f1fa8073540 2024-03-30T13:14:34.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1fad700640 1 --2- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa8073540 0x7f1fa81620b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009e50 tx=0x7f1fa001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65fdea0dea2c82e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1fa77fe640 1 -- 172.21.15.43:0/4068054706 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00090d0 con 0x7f1fa8073540 2024-03-30T13:14:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1fa77fe640 1 -- 172.21.15.43:0/4068054706 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa00a5660 con 0x7f1fa8073540 2024-03-30T13:14:34.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1fa77fe640 1 -- 172.21.15.43:0/4068054706 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0009270 con 0x7f1fa8073540 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 -- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa8073540 msgr2=0x7f1fa81620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 --2- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa8073540 0x7f1fa81620b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009e50 tx=0x7f1fa001c9a0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 -- 172.21.15.43:0/4068054706 shutdown_connections 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 --2- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa8073540 0x7f1fa81620b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 --2- 172.21.15.43:0/4068054706 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa815f740 0x7f1fa8073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 -- 172.21.15.43:0/4068054706 >> 172.21.15.43:0/4068054706 conn(0x7f1fa80724e0 msgr2=0x7f1fa80728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 -- 172.21.15.43:0/4068054706 shutdown_connections 2024-03-30T13:14:34.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 -- 172.21.15.43:0/4068054706 wait complete. 2024-03-30T13:14:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.078+0000 7f1faf163640 1 Processor -- start 2024-03-30T13:14:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 -- start start 2024-03-30T13:14:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa8073540 0x7f1fa813f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa80d0740 0x7f1fa8157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa815f740 0x7f1fa80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa806bef0 con 0x7f1fa8073540 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa806bd70 con 0x7f1fa815f740 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa816b450 con 0x7f1fa80d0740 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa8073540 0x7f1fa813f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa8073540 0x7f1fa813f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33646/0 (socket says 172.21.15.43:33646) 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 -- 172.21.15.43:0/204131992 learned_addr learned my addr 172.21.15.43:0/204131992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1fad700640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa815f740 0x7f1fa80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa80d0740 0x7f1fa8157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 -- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa80d0740 msgr2=0x7f1fa8157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa80d0740 0x7f1fa8157cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 -- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa815f740 msgr2=0x7f1fa80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa815f740 0x7f1fa80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 -- 172.21.15.43:0/204131992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa80b4970 con 0x7f1fa8073540 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa80d0740 0x7f1fa8157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faceff640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa8073540 0x7f1fa813f4d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c370 tx=0x7f1f9800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1fa57fa640 1 -- 172.21.15.43:0/204131992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980052d0 con 0x7f1fa8073540 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1fa57fa640 1 -- 172.21.15.43:0/204131992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98005470 con 0x7f1fa8073540 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa813e430 con 0x7f1fa8073540 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1fa57fa640 1 -- 172.21.15.43:0/204131992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98007260 con 0x7f1fa8073540 2024-03-30T13:14:34.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa813e9f0 con 0x7f1fa8073540 2024-03-30T13:14:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.082+0000 7f1fa57fa640 1 -- 172.21.15.43:0/204131992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f1f98007400 con 0x7f1fa8073540 2024-03-30T13:14:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fa57fa640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1f94067fc0 0x7f1f9406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fa57fa640 1 -- 172.21.15.43:0/204131992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f1f98085be0 con 0x7f1fa8073540 2024-03-30T13:14:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1f94067fc0 0x7f1f9406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 Processor -- start 2024-03-30T13:14:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1f94067fc0 0x7f1f9406a480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80b73c0 tx=0x7f1f9c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- start start 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e972150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e972690 0x55e24e9282f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e928c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e24e776aa0 con 0x55e24e971d70 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e24e6fdaf0 con 0x55e24e972690 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e24e772a50 con 0x55e24e928830 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e972150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e972150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33648/0 (socket says 172.21.15.43:33648) 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 -- 172.21.15.43:0/632777631 learned_addr learned my addr 172.21.15.43:0/632777631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e972690 0x55e24e9282f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fad700640 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e928c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 -- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 msgr2=0x55e24e928c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e928c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 -- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e972690 msgr2=0x55e24e9282f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e972690 0x55e24e9282f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 -- 172.21.15.43:0/632777631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24e6fbd00 con 0x55e24e971d70 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1faceff640 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e972150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1f98007fd0 tx=0x7f1f98056240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=220b5228a2511f6d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1f8e7fc640 1 -- 172.21.15.43:0/632777631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9805e660 con 0x55e24e971d70 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1f8e7fc640 1 -- 172.21.15.43:0/632777631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98053d80 con 0x55e24e971d70 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1f8e7fc640 1 -- 172.21.15.43:0/632777631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9805c0c0 con 0x55e24e971d70 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- 172.21.15.43:0/632777631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 msgr2=0x55e24e972150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e972150 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f1f98007fd0 tx=0x7f1f98056240 comp rx=0 tx=0).stop 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- 172.21.15.43:0/632777631 shutdown_connections 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e928c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e972690 0x55e24e9282f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/632777631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e972150 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- 172.21.15.43:0/632777631 >> 172.21.15.43:0/632777631 conn(0x55e24e9247e0 msgr2=0x55e24e926bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- 172.21.15.43:0/632777631 shutdown_connections 2024-03-30T13:14:34.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.086+0000 7f1fb22e5000 1 -- 172.21.15.43:0/632777631 wait complete. 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 -- start start 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 Processor -- start 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e92dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e971d70 0x55e24e956da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e9572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e24e73e270 con 0x55e24e972690 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55e24e7346a0 con 0x55e24e928830 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e24e70e880 con 0x55e24e971d70 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 10 client.0 _ll_register_callbacks cb 0x55e24e913490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e9572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e9572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33662/0 (socket says 172.21.15.43:33662) 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 -- 172.21.15.43:0/2514897353 learned_addr learned my addr 172.21.15.43:0/2514897353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1faceff640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e92dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e971d70 0x55e24e956da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 -- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e971d70 msgr2=0x55e24e956da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e971d70 0x55e24e956da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 -- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 msgr2=0x55e24e92dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e92dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 -- 172.21.15.43:0/2514897353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24e73b730 con 0x55e24e972690 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e971d70 0x55e24e956da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fad700640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e9572e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009e50 tx=0x7f1fa001b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c868eb4c1ebf4607 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 1 -- 172.21.15.43:0/2514897353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00bc070 con 0x55e24e972690 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 1 -- 172.21.15.43:0/2514897353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f1fa00c1cc0 con 0x55e24e972690 2024-03-30T13:14:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f98053d80 con 0x55e24e972690 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 1 -- 172.21.15.43:0/2514897353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1fa00c1ec0 con 0x55e24e972690 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 10 client.6657 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55e24e72eca0 con 0x55e24e972690 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 1 -- 172.21.15.43:0/2514897353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f98053d80 con 0x55e24e972690 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 1 -- 172.21.15.43:0/2514897353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f1fa00a5860 con 0x55e24e972690 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 10 client.6657 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 10 client.6657 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 20 client.6657 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e9795d0 0x55e24e9799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 4 client.6657 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e24e711c90 con 0x55e24e9795d0 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1faceff640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e9795d0 0x55e24e9799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1faceff640 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e9795d0 0x55e24e9799e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745b00c6b3b5ce1f server_cookie=ca81a2ce5fcc1b67 in_seq=0 out_seq=0 2024-03-30T13:14:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 10 client.6657 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 1 -- 172.21.15.43:0/2514897353 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55e24e711c90 con 0x55e24e9795d0 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1f877fe640 10 client.6657 handle_command_reply: tid=0 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.090+0000 7f1fb22e5000 1 client.6657 shutdown 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 msgr2=0x55e24e9572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e9572e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0009e50 tx=0x7f1fa001b1e0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 shutdown_connections 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e9795d0 0x55e24e9799e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e9572e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e971d70 0x55e24e956da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2514897353 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e92dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1f877fe640 0 client.6657 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 >> 172.21.15.43:0/2514897353 conn(0x55e24e9247e0 msgr2=0x55e24e927460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 shutdown_connections 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2514897353 wait complete. 2024-03-30T13:14:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.094+0000 7f1fb22e5000 20 client.6657 trim_cache size 0 max 16384 2024-03-30T13:14:34.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 Processor -- start 2024-03-30T13:14:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 -- start start 2024-03-30T13:14:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e957180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e926bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e972690 0x55e24e927110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8204150 con 0x55e24e971d70 2024-03-30T13:14:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8203130 con 0x55e24e928830 2024-03-30T13:14:34.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f1fa80b6ef0 con 0x55e24e972690 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e926bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e926bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33668/0 (socket says 172.21.15.43:33668) 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 -- 172.21.15.43:0/2458757094 learned_addr learned my addr 172.21.15.43:0/2458757094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1faceff640 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e957180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fad700640 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e972690 0x55e24e927110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 -- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e972690 msgr2=0x55e24e927110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e972690 0x55e24e927110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 -- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 msgr2=0x55e24e957180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e957180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 -- 172.21.15.43:0/2458757094 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e24e6fdd00 con 0x55e24e971d70 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e926bd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c010e00 tx=0x7f1f9c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257dea55454fe553 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1f8dffb640 1 -- 172.21.15.43:0/2458757094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c01c040 con 0x55e24e971d70 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1f8dffb640 1 -- 172.21.15.43:0/2458757094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c01d040 con 0x55e24e971d70 2024-03-30T13:14:34.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.106+0000 7f1f8dffb640 1 -- 172.21.15.43:0/2458757094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c00f040 con 0x55e24e971d70 2024-03-30T13:14:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2458757094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 msgr2=0x55e24e926bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e926bd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c010e00 tx=0x7f1f9c016000 comp rx=0 tx=0).stop 2024-03-30T13:14:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2458757094 shutdown_connections 2024-03-30T13:14:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e972690 0x55e24e927110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e971d70 0x55e24e926bd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/2458757094 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e928830 0x55e24e957180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2458757094 >> 172.21.15.43:0/2458757094 conn(0x55e24e9247e0 msgr2=0x55e24e988f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2458757094 shutdown_connections 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/2458757094 wait complete. 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- start start 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 Processor -- start 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e9e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e971d70 0x55e24e9e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e975ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55e24e73e270 con 0x55e24e972690 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa8202ab0 con 0x55e24e971d70 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55e24e832b00 con 0x55e24e928830 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 10 client.0 _ll_register_callbacks cb 0x7f1f9c009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e975ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e975ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33680/0 (socket says 172.21.15.43:33680) 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 -- 172.21.15.43:0/3669153225 learned_addr learned my addr 172.21.15.43:0/3669153225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1faceff640 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e9e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fa7fff640 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e971d70 0x55e24e9e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 -- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 msgr2=0x55e24e9e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e9e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 -- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e971d70 msgr2=0x55e24e9e45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e971d70 0x55e24e9e45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 -- 172.21.15.43:0/3669153225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e24e7f7f70 con 0x55e24e972690 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fad700640 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e975ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1fa001d240 tx=0x7f1fa0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e16cd10dd9b5b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1f877fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1f877fe640 1 -- 172.21.15.43:0/3669153225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0009510 con 0x55e24e972690 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1f877fe640 1 -- 172.21.15.43:0/3669153225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f1fa00af930 con 0x55e24e972690 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f9c01d040 con 0x55e24e972690 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1f877fe640 1 -- 172.21.15.43:0/3669153225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1fa00d9050 con 0x55e24e972690 2024-03-30T13:14:34.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 10 client.6663 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55e24e71d760 con 0x55e24e972690 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1f877fe640 1 -- 172.21.15.43:0/3669153225 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f9c01d040 con 0x55e24e972690 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1f877fe640 1 -- 172.21.15.43:0/3669153225 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f1fa00dc020 con 0x55e24e972690 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 10 client.6663 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 10 client.6663 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 20 client.6663 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e976920 0x55e24e976d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 4 client.6663 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628421}] 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628421}) -- 0x55e24e783fe0 con 0x55e24e976920 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1faceff640 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e976920 0x55e24e976d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1faceff640 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e976920 0x55e24e976d30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e557ae46298c3e8 server_cookie=cf7d702d2aacd954 in_seq=0 out_seq=0 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.110+0000 7f1f877fe640 10 client.6663 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1f877fe640 1 -- 172.21.15.43:0/3669153225 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3807 (crc 0 0 0) 0x55e24e783fe0 con 0x55e24e976920 2024-03-30T13:14:34.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1f877fe640 10 client.6663 handle_command_reply: tid=0 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 client.6663 shutdown 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 msgr2=0x55e24e975ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e975ad0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1fa001d240 tx=0x7f1fa0009810 comp rx=0 tx=0).stop 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 shutdown_connections 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55e24e976920 0x55e24e976d30 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e24e972690 0x55e24e975ad0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55e24e971d70 0x55e24e9e45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1f877fe640 0 client.6663 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 --2- 172.21.15.43:0/3669153225 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55e24e928830 0x55e24e9e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 >> 172.21.15.43:0/3669153225 conn(0x55e24e9247e0 msgr2=0x55e24e989620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 shutdown_connections 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 1 -- 172.21.15.43:0/3669153225 wait complete. 2024-03-30T13:14:34.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1fb22e5000 20 client.6663 trim_cache size 0 max 16384 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-debugfs.h", 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628421, 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.637191+0000", 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.997648+0000", 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:34.123 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 311, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:34.124 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.997648+0000", 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:34.125 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 311, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.637191+0000" 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:34.126 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 311, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.637191+0000" 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 232, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 32, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:34.127 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:34.128 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:34.129 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:34.130 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:34.131 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:34.132 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:34.133 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:34.134 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:34.135 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:34.135 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1f94067fc0 msgr2=0x7f1f9406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1f94067fc0 0x7f1f9406a480 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80b73c0 tx=0x7f1f9c008040 comp rx=0 tx=0).stop 2024-03-30T13:14:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa8073540 msgr2=0x7f1fa813f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa8073540 0x7f1fa813f4d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800c370 tx=0x7f1f9800c830 comp rx=0 tx=0).stop 2024-03-30T13:14:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 shutdown_connections 2024-03-30T13:14:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f1f94067fc0 0x7f1f9406a480 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f1fa815f740 0x7f1fa80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f1fa80d0740 0x7f1fa8157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 --2- 172.21.15.43:0/204131992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fa8073540 0x7f1fa813f4d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 >> 172.21.15.43:0/204131992 conn(0x7f1fa80724e0 msgr2=0x7f1fa81611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 shutdown_connections 2024-03-30T13:14:34.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.114+0000 7f1faf163640 1 -- 172.21.15.43:0/204131992 wait complete. 2024-03-30T13:14:34.137 DEBUG:tasks.cephfs.test_uninlining:getting inode info #20 of 23:1099511628422 2024-03-30T13:14:34.137 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628422 2024-03-30T13:14:34.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628422 2024-03-30T13:14:34.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 Processor -- start 2024-03-30T13:14:34.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 -- start start 2024-03-30T13:14:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406db00 0x7fa9f40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fa9f406dee0 0x7fa9f415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9f406f060 0x7fa9f415f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f406f5d0 con 0x7fa9f406f060 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fa9f406f8b0 con 0x7fa9f406dee0 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f9084640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f406fb90 con 0x7fa9f406db00 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9f406f060 0x7fa9f415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9f406f060 0x7fa9f415f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33696/0 (socket says 172.21.15.43:33696) 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 -- 172.21.15.43:0/2062340507 learned_addr learned my addr 172.21.15.43:0/2062340507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2062340507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406db00 0x7fa9f40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/2062340507 >> v2:172.21.15.113:3300/0 conn(0x7fa9f406dee0 0x7fa9f415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 -- 172.21.15.43:0/2062340507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406db00 msgr2=0x7fa9f40cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 --2- 172.21.15.43:0/2062340507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406db00 0x7fa9f40cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 -- 172.21.15.43:0/2062340507 >> v2:172.21.15.113:3300/0 conn(0x7fa9f406dee0 msgr2=0x7fa9f415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 --2- 172.21.15.43:0/2062340507 >> v2:172.21.15.113:3300/0 conn(0x7fa9f406dee0 0x7fa9f415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 -- 172.21.15.43:0/2062340507 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f406fe70 con 0x7fa9f406f060 2024-03-30T13:14:34.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.246+0000 7fa9f359e640 1 --2- 172.21.15.43:0/2062340507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406f060 0x7fa9f415f200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40683f0 tx=0x7fa9e401cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6dfa2f5cd415e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f259c640 1 -- 172.21.15.43:0/2062340507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40ab5f0 con 0x7fa9f406f060 2024-03-30T13:14:34.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f259c640 1 -- 172.21.15.43:0/2062340507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9e40b0070 con 0x7fa9f406f060 2024-03-30T13:14:34.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f259c640 1 -- 172.21.15.43:0/2062340507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40ab9b0 con 0x7fa9f406f060 2024-03-30T13:14:34.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- 172.21.15.43:0/2062340507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406f060 msgr2=0x7fa9f415f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2062340507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406f060 0x7fa9f415f200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40683f0 tx=0x7fa9e401cf80 comp rx=0 tx=0).stop 2024-03-30T13:14:34.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- 172.21.15.43:0/2062340507 shutdown_connections 2024-03-30T13:14:34.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2062340507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406f060 0x7fa9f415f200 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2062340507 >> v2:172.21.15.113:3300/0 conn(0x7fa9f406dee0 0x7fa9f415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2062340507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406db00 0x7fa9f40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- 172.21.15.43:0/2062340507 >> 172.21.15.43:0/2062340507 conn(0x7fa9f40b2c70 msgr2=0x7fa9f40b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- 172.21.15.43:0/2062340507 shutdown_connections 2024-03-30T13:14:34.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- 172.21.15.43:0/2062340507 wait complete. 2024-03-30T13:14:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 Processor -- start 2024-03-30T13:14:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- start start 2024-03-30T13:14:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9f406db00 0x7fa9f415d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406dee0 0x7fa9f414f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406f060 0x7fa9f413c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f415c270 con 0x7fa9f406dee0 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f415c0f0 con 0x7fa9f406db00 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f415c3f0 con 0x7fa9f406f060 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406dee0 0x7fa9f414f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406dee0 0x7fa9f414f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33700/0 (socket says 172.21.15.43:33700) 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 -- 172.21.15.43:0/2555355361 learned_addr learned my addr 172.21.15.43:0/2555355361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9f406db00 0x7fa9f415d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f359e640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406f060 0x7fa9f413c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 -- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406f060 msgr2=0x7fa9f413c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406f060 0x7fa9f413c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 -- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9f406db00 msgr2=0x7fa9f415d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9f406db00 0x7fa9f415d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 -- 172.21.15.43:0/2555355361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f41f8d50 con 0x7fa9f406dee0 2024-03-30T13:14:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406dee0 0x7fa9f414f130 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00d5d0 tx=0x7fa9dc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ebfff640 1 -- 172.21.15.43:0/2555355361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004240 con 0x7fa9f406dee0 2024-03-30T13:14:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ebfff640 1 -- 172.21.15.43:0/2555355361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc0043e0 con 0x7fa9f406dee0 2024-03-30T13:14:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f41f9030 con 0x7fa9f406dee0 2024-03-30T13:14:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9ebfff640 1 -- 172.21.15.43:0/2555355361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc003840 con 0x7fa9f406dee0 2024-03-30T13:14:34.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.250+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41f95f0 con 0x7fa9f406dee0 2024-03-30T13:14:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ebfff640 1 -- 172.21.15.43:0/2555355361 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fa9dc0039e0 con 0x7fa9f406dee0 2024-03-30T13:14:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ebfff640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa9d4067fc0 0x7fa9d406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ebfff640 1 -- 172.21.15.43:0/2555355361 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7fa9dc0868a0 con 0x7fa9f406dee0 2024-03-30T13:14:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa9d4067fc0 0x7fa9d406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 Processor -- start 2024-03-30T13:14:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 -- start start 2024-03-30T13:14:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa9d4067fc0 0x7fa9d406a480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009fd0 tx=0x7fa9ec009800 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 0x5555eb5403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4f6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb4f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5555eb340aa0 con 0x5555eb5408e0 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5555eb2c7af0 con 0x5555eb4f6c40 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5555eb33ca50 con 0x5555eb53ffc0 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33716/0 (socket says 172.21.15.43:33716) 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 -- 172.21.15.43:0/3451321151 learned_addr learned my addr 172.21.15.43:0/3451321151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9f359e640 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb4f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 -- 172.21.15.43:0/3451321151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 msgr2=0x5555eb5403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 0x5555eb5403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 -- 172.21.15.43:0/3451321151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 msgr2=0x5555eb4f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb4f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 -- 172.21.15.43:0/3451321151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eb2c5d00 con 0x5555eb5408e0 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4f6700 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc05ede0 tx=0x7fa9dc056430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d138de759449d590 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9e8d99640 1 -- 172.21.15.43:0/3451321151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc083070 con 0x5555eb5408e0 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9e8d99640 1 -- 172.21.15.43:0/3451321151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc05f310 con 0x5555eb5408e0 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.254+0000 7fa9e8d99640 1 -- 172.21.15.43:0/3451321151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc003cd0 con 0x5555eb5408e0 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/3451321151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 msgr2=0x5555eb4f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4f6700 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc05ede0 tx=0x7fa9dc056430 comp rx=0 tx=0).stop 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/3451321151 shutdown_connections 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb4f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4f6700 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3451321151 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 0x5555eb5403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/3451321151 >> 172.21.15.43:0/3451321151 conn(0x5555eb4ee7e0 msgr2=0x5555eb4f0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/3451321151 shutdown_connections 2024-03-30T13:14:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/3451321151 wait complete. 2024-03-30T13:14:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- start start 2024-03-30T13:14:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 Processor -- start 2024-03-30T13:14:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb53f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 0x5555eb4fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4fce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5555eb308270 con 0x5555eb5408e0 2024-03-30T13:14:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5555eb2fe6a0 con 0x5555eb4f6c40 2024-03-30T13:14:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5555eb2d8880 con 0x5555eb53ffc0 2024-03-30T13:14:34.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 10 client.0 _ll_register_callbacks cb 0x5555eb4dd490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4fce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33724/0 (socket says 172.21.15.43:33724) 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 -- 172.21.15.43:0/2074057084 learned_addr learned my addr 172.21.15.43:0/2074057084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb53f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 0x5555eb4fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 -- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 msgr2=0x5555eb4fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 0x5555eb4fc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 -- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 msgr2=0x5555eb53f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb53f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 -- 172.21.15.43:0/2074057084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eb305730 con 0x5555eb5408e0 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb53f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f359e640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4fce50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40ad040 tx=0x7fa9e401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6554009f469b3b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/2074057084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40b4a70 con 0x5555eb5408e0 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/2074057084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7fa9e40c0ce0 con 0x5555eb5408e0 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9dc05f310 con 0x5555eb5408e0 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/2074057084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa9e40b4c10 con 0x5555eb5408e0 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 10 client.6675 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5555eb2f8ca0 con 0x5555eb5408e0 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/2074057084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc05f310 con 0x5555eb5408e0 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/2074057084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fa9e40c7050 con 0x5555eb5408e0 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 10 client.6675 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 10 client.6675 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 20 client.6675 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb4fdbc0 0x5555eb552280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 4 client.6675 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555eb2dbc90 con 0x5555eb4fdbc0 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.258+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb4fdbc0 0x5555eb552280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb4fdbc0 0x5555eb552280 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0cb69d5b1f8a158 server_cookie=393b840dffc8140d in_seq=0 out_seq=0 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9d1ffb640 10 client.6675 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/2074057084 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5555eb2dbc90 con 0x5555eb4fdbc0 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9d1ffb640 10 client.6675 handle_command_reply: tid=0 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 client.6675 shutdown 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 msgr2=0x5555eb4fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4fce50 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa9e40ad040 tx=0x7fa9e401ad60 comp rx=0 tx=0).stop 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 shutdown_connections 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb4fdbc0 0x5555eb552280 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb5408e0 0x5555eb4fce50 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb53ffc0 0x5555eb4fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9d1ffb640 0 client.6675 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 --2- 172.21.15.43:0/2074057084 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb53f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 >> 172.21.15.43:0/2074057084 conn(0x5555eb4ee7e0 msgr2=0x5555eb4f78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 shutdown_connections 2024-03-30T13:14:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 1 -- 172.21.15.43:0/2074057084 wait complete. 2024-03-30T13:14:34.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.262+0000 7fa9fc206000 20 client.6675 trim_cache size 0 max 16384 2024-03-30T13:14:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.274+0000 7fa9fc206000 1 Processor -- start 2024-03-30T13:14:34.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.274+0000 7fa9fc206000 1 -- start start 2024-03-30T13:14:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb4f6c40 0x5555eb5552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb53ffc0 0x5555eb5557f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 0x5555eb51e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f41ffe30 con 0x5555eb4f6c40 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f4153950 con 0x5555eb53ffc0 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f4151a00 con 0x5555eb5408e0 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb4f6c40 0x5555eb5552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb4f6c40 0x5555eb5552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33730/0 (socket says 172.21.15.43:33730) 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 -- 172.21.15.43:0/3641059493 learned_addr learned my addr 172.21.15.43:0/3641059493 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb53ffc0 0x5555eb5557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 -- 172.21.15.43:0/3641059493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 msgr2=0x5555eb51e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 0x5555eb51e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 -- 172.21.15.43:0/3641059493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb53ffc0 msgr2=0x5555eb5557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb53ffc0 0x5555eb5557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 -- 172.21.15.43:0/3641059493 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555eb2c7d00 con 0x5555eb4f6c40 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb4f6c40 0x5555eb5552b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009ea0 tx=0x7fa9ec013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baea9556923d3b6c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9d3fff640 1 -- 172.21.15.43:0/3641059493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec017c10 con 0x5555eb4f6c40 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9d3fff640 1 -- 172.21.15.43:0/3641059493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ec017db0 con 0x5555eb4f6c40 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9d3fff640 1 -- 172.21.15.43:0/3641059493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec028160 con 0x5555eb4f6c40 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- 172.21.15.43:0/3641059493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb4f6c40 msgr2=0x5555eb5552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb4f6c40 0x5555eb5552b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009ea0 tx=0x7fa9ec013000 comp rx=0 tx=0).stop 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- 172.21.15.43:0/3641059493 shutdown_connections 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 0x5555eb51e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb53ffc0 0x5555eb5557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- 172.21.15.43:0/3641059493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb4f6c40 0x5555eb5552b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- 172.21.15.43:0/3641059493 >> 172.21.15.43:0/3641059493 conn(0x5555eb4ee7e0 msgr2=0x5555eb5205e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- 172.21.15.43:0/3641059493 shutdown_connections 2024-03-30T13:14:34.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- 172.21.15.43:0/3641059493 wait complete. 2024-03-30T13:14:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- start start 2024-03-30T13:14:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 Processor -- start 2024-03-30T13:14:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb5b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb53ffc0 0x5555eb5b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 0x5555eb5532d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5555eb2d8880 con 0x5555eb53ffc0 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f414e3b0 con 0x5555eb4f6c40 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5555eb308270 con 0x5555eb5408e0 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9fc206000 10 client.0 _ll_register_callbacks cb 0x7fa9e40096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb53ffc0 0x5555eb5b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb53ffc0 0x5555eb5b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33744/0 (socket says 172.21.15.43:33744) 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 -- 172.21.15.43:0/156193713 learned_addr learned my addr 172.21.15.43:0/156193713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb5b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 -- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 msgr2=0x5555eb5532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 0x5555eb5532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 -- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 msgr2=0x5555eb5b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb5b2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 -- 172.21.15.43:0/156193713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555eb3c1f70 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9ea59c640 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb53ffc0 0x5555eb5b2b60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc05ede0 tx=0x7fa9dc086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edc2196f0d35223 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.278+0000 7fa9d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/156193713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc083040 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/156193713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7fa9dc054ae0 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ec017db0 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/156193713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa9dc07c050 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 10 client.6681 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5555eb2e7760 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/156193713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9ec017db0 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/156193713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7fa9dc013070 con 0x5555eb53ffc0 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 10 client.6681 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 10 client.6681 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 20 client.6681 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb553810 0x5555eb553c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 4 client.6681 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628422}] 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628422}) -- 0x5555eb34dfe0 con 0x5555eb553810 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb553810 0x5555eb553c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f2d9d640 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb553810 0x5555eb553c20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcb429133d31b6b server_cookie=7cfca19867b98808 in_seq=0 out_seq=0 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 10 client.6681 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 1 -- 172.21.15.43:0/156193713 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3812 (crc 0 0 0) 0x5555eb34dfe0 con 0x5555eb553810 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 10 client.6681 handle_command_reply: tid=0 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 client.6681 shutdown 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb53ffc0 msgr2=0x5555eb5b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb53ffc0 0x5555eb5b2b60 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc05ede0 tx=0x7fa9dc086040 comp rx=0 tx=0).stop 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 shutdown_connections 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5555eb553810 0x5555eb553c20 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5555eb5408e0 0x5555eb5532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9d1ffb640 0 client.6681 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5555eb53ffc0 0x5555eb5b2b60 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 --2- 172.21.15.43:0/156193713 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5555eb4f6c40 0x5555eb5b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 >> 172.21.15.43:0/156193713 conn(0x5555eb4ee7e0 msgr2=0x5555eb4f7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 shutdown_connections 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 1 -- 172.21.15.43:0/156193713 wait complete. 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9fc206000 20 client.6681 trim_cache size 0 max 16384 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/orangefs-dev-proto.h", 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628422, 2024-03-30T13:14:34.292 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.646257+0000", 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:57.000182+0000", 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:34.293 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 2111, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:34.294 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:57.000182+0000", 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 4, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.295 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2111, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.646257+0000" 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2111, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.646257+0000" 2024-03-30T13:14:34.296 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 238, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 34, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:34.297 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:34.298 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:34.299 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:34.300 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:34.301 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:34.302 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:34.303 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa9d4067fc0 msgr2=0x7fa9d406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa9d4067fc0 0x7fa9d406a480 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009fd0 tx=0x7fa9ec009800 comp rx=0 tx=0).stop 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406dee0 msgr2=0x7fa9f414f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406dee0 0x7fa9f414f130 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00d5d0 tx=0x7fa9dc00da90 comp rx=0 tx=0).stop 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 shutdown_connections 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fa9d4067fc0 0x7fa9d406a480 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fa9f406f060 0x7fa9f413c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f406dee0 0x7fa9f414f130 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 --2- 172.21.15.43:0/2555355361 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fa9f406db00 0x7fa9f415d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 >> 172.21.15.43:0/2555355361 conn(0x7fa9f40b2c70 msgr2=0x7fa9f4150160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 shutdown_connections 2024-03-30T13:14:34.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.282+0000 7fa9f9084640 1 -- 172.21.15.43:0/2555355361 wait complete. 2024-03-30T13:14:34.306 DEBUG:tasks.cephfs.test_uninlining:getting inode info #21 of 23:1099511628407 2024-03-30T13:14:34.306 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628407 2024-03-30T13:14:34.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628407 2024-03-30T13:14:34.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0499387640 1 Processor -- start 2024-03-30T13:14:34.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0499387640 1 -- start start 2024-03-30T13:14:34.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0499387640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 0x7f0494166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0499387640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049406d710 con 0x7f049413f140 2024-03-30T13:14:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0499387640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f049406d9f0 con 0x7f049415ff40 2024-03-30T13:14:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0499387640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f049406e320 con 0x7f049406cd80 2024-03-30T13:14:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0492ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f049413f140 0x7f04941606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35814/0 (socket says 172.21.15.43:35814) 2024-03-30T13:14:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0492ffd640 1 -- 172.21.15.43:0/2479618519 learned_addr learned my addr 172.21.15.43:0/2479618519 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f04937fe640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 0x7f0494166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2431912780 0 0) 0x7f049406d710 con 0x7f049413f140 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f047c003530 con 0x7f049413f140 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1755914952 0 0) 0x7f047c003530 con 0x7f049413f140 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 msgr2=0x7f0494166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 0x7f0494166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 >> v1:172.21.15.113:6789/0 conn(0x7f049415ff40 legacy=0x7f049406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049406dcd0 con 0x7f049413f140 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.418+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f0484001680 con 0x7f049413f140 2024-03-30T13:14:34.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 >> v1:172.21.15.43:6789/0 conn(0x7f049413f140 legacy=0x7f04941606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 0x7f047c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c003f30 0x7f047c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f049406e320 con 0x7f047c004d50 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f047c00ba30 con 0x7f047c003f30 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f047c00bc30 con 0x7f049406cd80 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0492ffd640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 0x7f047c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04927fc640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c003f30 0x7f047c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 -- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 msgr2=0x7f047c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f049406cd80 0x7f047c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 -- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c003f30 msgr2=0x7f047c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c003f30 0x7f047c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 -- 172.21.15.43:0/2479618519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047c003530 con 0x7f047c004d50 2024-03-30T13:14:34.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04927fc640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c003f30 0x7f047c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f048c009f90 tx=0x7f048c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a463e734a513cb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0b2070 con 0x7f047c004d50 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0a5b50 con 0x7f047c004d50 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0491ffb640 1 -- 172.21.15.43:0/2479618519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c009c10 con 0x7f047c004d50 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0499387640 1 -- 172.21.15.43:0/2479618519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f047c004d50 msgr2=0x7f047c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0499387640 1 --2- 172.21.15.43:0/2479618519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f047c004d50 0x7f047c0051d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f048c009f90 tx=0x7f048c01cc40 comp rx=0 tx=0).stop 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f04937fe640 1 -- 172.21.15.43:0/2479618519 reap_dead start 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0499387640 1 -- 172.21.15.43:0/2479618519 shutdown_connections 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0499387640 1 -- 172.21.15.43:0/2479618519 >> 172.21.15.43:0/2479618519 conn(0x7f04940d00a0 msgr2=0x7f04940d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0499387640 1 -- 172.21.15.43:0/2479618519 shutdown_connections 2024-03-30T13:14:34.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.422+0000 7f0499387640 1 -- 172.21.15.43:0/2479618519 wait complete. 2024-03-30T13:14:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 Processor -- start 2024-03-30T13:14:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 -- start start 2024-03-30T13:14:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c004d50 0x7f04941367c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0494072440 0x7f04940728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0494154e40 0x7f04941552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f049413f520 con 0x7f0494154e40 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0494160de0 con 0x7f047c004d50 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f049413f6a0 con 0x7f0494072440 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0494154e40 0x7f04941552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0494154e40 0x7f04941552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33770/0 (socket says 172.21.15.43:33770) 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 -- 172.21.15.43:0/2291787459 learned_addr learned my addr 172.21.15.43:0/2291787459 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0492ffd640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c004d50 0x7f04941367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04927fc640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0494072440 0x7f04940728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 -- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0494072440 msgr2=0x7f04940728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0494072440 0x7f04940728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 -- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c004d50 msgr2=0x7f04941367c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c004d50 0x7f04941367c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 -- 172.21.15.43:0/2291787459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0494154370 con 0x7f0494154e40 2024-03-30T13:14:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f04937fe640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0494154e40 0x7f04941552c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f048c009f90 tx=0x7f048c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f047b7fe640 1 -- 172.21.15.43:0/2291787459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0bd070 con 0x7f0494154e40 2024-03-30T13:14:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f047b7fe640 1 -- 172.21.15.43:0/2291787459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0a5950 con 0x7f0494154e40 2024-03-30T13:14:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f047b7fe640 1 -- 172.21.15.43:0/2291787459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0b2040 con 0x7f0494154e40 2024-03-30T13:14:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0494073180 con 0x7f0494154e40 2024-03-30T13:14:34.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.426+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0494073740 con 0x7f0494154e40 2024-03-30T13:14:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f047b7fe640 1 -- 172.21.15.43:0/2291787459 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f048c0c0070 con 0x7f0494154e40 2024-03-30T13:14:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f047b7fe640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0480067fc0 0x7f048006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f047b7fe640 1 -- 172.21.15.43:0/2291787459 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f048c1203f0 con 0x7f0494154e40 2024-03-30T13:14:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f0492ffd640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0480067fc0 0x7f048006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 Processor -- start 2024-03-30T13:14:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- start start 2024-03-30T13:14:34.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f0492ffd640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0480067fc0 0x7f048006a480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0484005260 tx=0x7f0484009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1bae690 0x5577e1b642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1b64830 0x5577e1b64c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5577e19b2aa0 con 0x5577e1b64830 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5577e1939af0 con 0x5577e1bae690 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5577e19aea50 con 0x5577e1badd70 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1b64830 0x5577e1b64c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1b64830 0x5577e1b64c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33772/0 (socket says 172.21.15.43:33772) 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 -- 172.21.15.43:0/448825848 learned_addr learned my addr 172.21.15.43:0/448825848 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f0492ffd640 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04927fc640 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1bae690 0x5577e1b642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 -- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 msgr2=0x5577e1bae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 -- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1bae690 msgr2=0x5577e1b642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1bae690 0x5577e1b642f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 -- 172.21.15.43:0/448825848 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e1937d00 con 0x5577e1b64830 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f04937fe640 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1b64830 0x5577e1b64c10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f048c11e710 tx=0x7f048c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eb6cd54cbf21d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f0478ff9640 1 -- 172.21.15.43:0/448825848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0f4640 con 0x5577e1b64830 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f0478ff9640 1 -- 172.21.15.43:0/448825848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0f8e70 con 0x5577e1b64830 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f0478ff9640 1 -- 172.21.15.43:0/448825848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0ec9b0 con 0x5577e1b64830 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- 172.21.15.43:0/448825848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1b64830 msgr2=0x5577e1b64c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1b64830 0x5577e1b64c10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f048c11e710 tx=0x7f048c00b040 comp rx=0 tx=0).stop 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- 172.21.15.43:0/448825848 shutdown_connections 2024-03-30T13:14:34.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1b64830 0x5577e1b64c10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1bae690 0x5577e1b642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 --2- 172.21.15.43:0/448825848 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- 172.21.15.43:0/448825848 >> 172.21.15.43:0/448825848 conn(0x5577e1b607e0 msgr2=0x5577e1b62bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- 172.21.15.43:0/448825848 shutdown_connections 2024-03-30T13:14:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.430+0000 7f049c509000 1 -- 172.21.15.43:0/448825848 wait complete. 2024-03-30T13:14:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 -- start start 2024-03-30T13:14:34.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 Processor -- start 2024-03-30T13:14:34.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1b64830 0x5577e1b69c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1badd70 0x5577e1b92da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1b932e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5577e197a270 con 0x5577e1bae690 2024-03-30T13:14:34.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x5577e19706a0 con 0x5577e1badd70 2024-03-30T13:14:34.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5577e194a880 con 0x5577e1b64830 2024-03-30T13:14:34.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 10 client.0 _ll_register_callbacks cb 0x5577e1b4f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1b932e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1b932e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33776/0 (socket says 172.21.15.43:33776) 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 -- 172.21.15.43:0/1952115256 learned_addr learned my addr 172.21.15.43:0/1952115256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04927fc640 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1badd70 0x5577e1b92da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0492ffd640 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1b64830 0x5577e1b69c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 -- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1b64830 msgr2=0x5577e1b69c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1b64830 0x5577e1b69c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 -- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1badd70 msgr2=0x5577e1b92da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1badd70 0x5577e1b92da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 -- 172.21.15.43:0/1952115256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e1977730 con 0x5577e1bae690 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f04937fe640 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1b932e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f048c098770 tx=0x7f048c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9ee7d50c91c306 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0471ffb640 1 -- 172.21.15.43:0/1952115256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c120220 con 0x5577e1bae690 2024-03-30T13:14:34.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0471ffb640 1 -- 172.21.15.43:0/1952115256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f048c0f89e0 con 0x5577e1bae690 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f048c0f8e70 con 0x5577e1bae690 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0471ffb640 1 -- 172.21.15.43:0/1952115256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f048c0f8be0 con 0x5577e1bae690 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 10 client.6693 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5577e196aca0 con 0x5577e1bae690 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0471ffb640 1 -- 172.21.15.43:0/1952115256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0f8e70 con 0x5577e1bae690 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0471ffb640 1 -- 172.21.15.43:0/1952115256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f048c0b5050 con 0x5577e1bae690 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 10 client.6693 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 10 client.6693 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 20 client.6693 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bb55d0 0x5577e1bb59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 4 client.6693 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e194dc90 con 0x5577e1bb55d0 2024-03-30T13:14:34.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0492ffd640 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bb55d0 0x5577e1bb59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0471ffb640 10 client.6693 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.434+0000 7f0492ffd640 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bb55d0 0x5577e1bb59b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b389269914815a2 server_cookie=ed119f6f8bb1ea0a in_seq=0 out_seq=0 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f0471ffb640 1 -- 172.21.15.43:0/1952115256 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x5577e194dc90 con 0x5577e1bb55d0 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f0471ffb640 10 client.6693 handle_command_reply: tid=0 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 client.6693 shutdown 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 msgr2=0x5577e1b932e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1b932e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f048c098770 tx=0x7f048c01cc40 comp rx=0 tx=0).stop 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 shutdown_connections 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bb55d0 0x5577e1bb59b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1b932e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1badd70 0x5577e1b92da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f0471ffb640 0 client.6693 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 --2- 172.21.15.43:0/1952115256 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1b64830 0x5577e1b69c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 >> 172.21.15.43:0/1952115256 conn(0x5577e1b607e0 msgr2=0x5577e1b63460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 shutdown_connections 2024-03-30T13:14:34.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 1 -- 172.21.15.43:0/1952115256 wait complete. 2024-03-30T13:14:34.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.438+0000 7f049c509000 20 client.6693 trim_cache size 0 max 16384 2024-03-30T13:14:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 Processor -- start 2024-03-30T13:14:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 -- start start 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1bddcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1badd70 0x5577e1bde1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1bae690 0x5577e1bde730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f04941ff260 con 0x5577e1badd70 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04941fe260 con 0x5577e1b64830 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f049413d750 con 0x5577e1bae690 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1badd70 0x5577e1bde1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1badd70 0x5577e1bde1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33788/0 (socket says 172.21.15.43:33788) 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 -- 172.21.15.43:0/3697101675 learned_addr learned my addr 172.21.15.43:0/3697101675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 -- 172.21.15.43:0/3697101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1bae690 msgr2=0x5577e1bde730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f0492ffd640 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1bddcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1bae690 0x5577e1bde730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 -- 172.21.15.43:0/3697101675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 msgr2=0x5577e1bddcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1bddcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 -- 172.21.15.43:0/3697101675 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e1939d00 con 0x5577e1badd70 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.450+0000 7f04927fc640 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1badd70 0x5577e1bde1f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0488001720 tx=0x7f0488001be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff16df9c7111aaf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0473fff640 1 -- 172.21.15.43:0/3697101675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488010040 con 0x5577e1badd70 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0473fff640 1 -- 172.21.15.43:0/3697101675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048801b070 con 0x5577e1badd70 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0473fff640 1 -- 172.21.15.43:0/3697101675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0488012840 con 0x5577e1badd70 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3697101675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1badd70 msgr2=0x5577e1bde1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1badd70 0x5577e1bde1f0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f0488001720 tx=0x7f0488001be0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3697101675 shutdown_connections 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1bae690 0x5577e1bde730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1badd70 0x5577e1bde1f0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- 172.21.15.43:0/3697101675 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1bddcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3697101675 >> 172.21.15.43:0/3697101675 conn(0x5577e1b607e0 msgr2=0x5577e1bc4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3697101675 shutdown_connections 2024-03-30T13:14:34.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3697101675 wait complete. 2024-03-30T13:14:34.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- start start 2024-03-30T13:14:34.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 Processor -- start 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1c22e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1bdf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x5577e1a6eb00 con 0x5577e1bae690 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f04941fdc10 con 0x5577e1b64830 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x5577e194a880 con 0x5577e1badd70 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 10 client.0 _ll_register_callbacks cb 0x7f048c0efca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1bdf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1bdf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33794/0 (socket says 172.21.15.43:33794) 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 -- 172.21.15.43:0/3801301181 learned_addr learned my addr 172.21.15.43:0/3801301181 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0492ffd640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1c22e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 -- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 msgr2=0x5577e1bded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04927fc640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bded00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 -- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 msgr2=0x5577e1c22e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1c22e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 -- 172.21.15.43:0/3801301181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e1a33f70 con 0x5577e1bae690 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0492ffd640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1c22e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f04937fe640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1bdf240 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f048c0efff0 tx=0x7f048c11e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb7ff0914b204c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0471ffb640 1 -- 172.21.15.43:0/3801301181 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0ec5e0 con 0x5577e1bae690 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0471ffb640 1 -- 172.21.15.43:0/3801301181 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f048c11e3e0 con 0x5577e1bae690 2024-03-30T13:14:34.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f048801b070 con 0x5577e1bae690 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0471ffb640 1 -- 172.21.15.43:0/3801301181 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f048c0f8e70 con 0x5577e1bae690 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 10 client.6699 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5577e1959760 con 0x5577e1bae690 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0471ffb640 1 -- 172.21.15.43:0/3801301181 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048801b070 con 0x5577e1bae690 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0471ffb640 1 -- 172.21.15.43:0/3801301181 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f048c0b5020 con 0x5577e1bae690 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 10 client.6699 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 10 client.6699 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 20 client.6699 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bdf780 0x5577e1bdfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 4 client.6699 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628407}] 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628407}) -- 0x5577e19bffe0 con 0x5577e1bdf780 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.454+0000 7f0492ffd640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bdf780 0x5577e1bdfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0492ffd640 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bdf780 0x5577e1bdfb60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da3eca2337a5100 server_cookie=dad72d12de473175 in_seq=0 out_seq=0 2024-03-30T13:14:34.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0471ffb640 10 client.6699 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0471ffb640 1 -- 172.21.15.43:0/3801301181 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3796 (crc 0 0 0) 0x5577e19bffe0 con 0x5577e1bdf780 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0471ffb640 10 client.6699 handle_command_reply: tid=0 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 client.6699 shutdown 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 msgr2=0x5577e1bdf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1bdf240 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f048c0efff0 tx=0x7f048c11e5e0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 shutdown_connections 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x5577e1bdf780 0x5577e1bdfb60 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5577e1bae690 0x5577e1bdf240 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x5577e1badd70 0x5577e1bded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0471ffb640 0 client.6699 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 --2- 172.21.15.43:0/3801301181 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x5577e1b64830 0x5577e1c22e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 >> 172.21.15.43:0/3801301181 conn(0x5577e1b607e0 msgr2=0x5577e1bc5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 shutdown_connections 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 1 -- 172.21.15.43:0/3801301181 wait complete. 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f049c509000 20 client.6699 trim_cache size 0 max 16384 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/Makefile", 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628407, 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.647271+0000", 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.920451+0000", 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:34.468 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.469 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 356, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.920451+0000", 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:34.470 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.647271+0000" 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.471 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 356, 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.647271+0000" 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.472 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 240, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 4, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:34.473 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:34.474 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:34.475 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:34.476 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.477 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.478 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:34.479 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:34.480 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0480067fc0 msgr2=0x7f048006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0480067fc0 0x7f048006a480 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0484005260 tx=0x7f0484009040 comp rx=0 tx=0).stop 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0494154e40 msgr2=0x7f04941552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0494154e40 0x7f04941552c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f048c009f90 tx=0x7f048c009750 comp rx=0 tx=0).stop 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 shutdown_connections 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0480067fc0 0x7f048006a480 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0494154e40 0x7f04941552c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0494072440 0x7f04940728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 --2- 172.21.15.43:0/2291787459 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f047c004d50 0x7f04941367c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 >> 172.21.15.43:0/2291787459 conn(0x7f04940d00a0 msgr2=0x7f04940b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 shutdown_connections 2024-03-30T13:14:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.458+0000 7f0499387640 1 -- 172.21.15.43:0/2291787459 wait complete. 2024-03-30T13:14:34.483 DEBUG:tasks.cephfs.test_uninlining:getting inode info #22 of 23:1099511628430 2024-03-30T13:14:34.483 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628430 2024-03-30T13:14:34.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628430 2024-03-30T13:14:34.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da4558640 1 Processor -- start 2024-03-30T13:14:34.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da4558640 1 -- start start 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da4558640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d9c13f140 0x7f9d9c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da4558640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9d9c06cd80 0x7f9d9c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da4558640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d9c06d710 con 0x7f9d9c15ff40 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da4558640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9c06e320 con 0x7f9d9c13f140 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da4558640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9d9c06d9f0 con 0x7f9d9c06cd80 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da22f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d9c13f140 0x7f9d9c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da2af5640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9d9c06cd80 0x7f9d9c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da22f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d9c13f140 0x7f9d9c1606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33800/0 (socket says 172.21.15.43:33800) 2024-03-30T13:14:34.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da22f4640 1 -- 172.21.15.43:0/1632198815 learned_addr learned my addr 172.21.15.43:0/1632198815 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da22f4640 1 -- 172.21.15.43:0/1632198815 >> v2:172.21.15.113:3300/0 conn(0x7f9d9c06cd80 msgr2=0x7f9d9c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da22f4640 1 --2- 172.21.15.43:0/1632198815 >> v2:172.21.15.113:3300/0 conn(0x7f9d9c06cd80 0x7f9d9c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da22f4640 1 -- 172.21.15.43:0/1632198815 >> v1:172.21.15.113:6789/0 conn(0x7f9d9c15ff40 legacy=0x7f9d9c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.594+0000 7f9da22f4640 1 -- 172.21.15.43:0/1632198815 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c06dcd0 con 0x7f9d9c13f140 2024-03-30T13:14:34.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.598+0000 7f9da22f4640 1 --2- 172.21.15.43:0/1632198815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c13f140 0x7f9d9c1606d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c15c870 tx=0x7f9d9801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b59d0dcbdd98fb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.598+0000 7f9da12f2640 1 -- 172.21.15.43:0/1632198815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980aa670 con 0x7f9d9c13f140 2024-03-30T13:14:34.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.598+0000 7f9da12f2640 1 -- 172.21.15.43:0/1632198815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d980af070 con 0x7f9d9c13f140 2024-03-30T13:14:34.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.598+0000 7f9da12f2640 1 -- 172.21.15.43:0/1632198815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980aaa30 con 0x7f9d9c13f140 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- 172.21.15.43:0/1632198815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c13f140 msgr2=0x7f9d9c1606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 --2- 172.21.15.43:0/1632198815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c13f140 0x7f9d9c1606d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c15c870 tx=0x7f9d9801c4f0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- 172.21.15.43:0/1632198815 shutdown_connections 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 --2- 172.21.15.43:0/1632198815 >> v2:172.21.15.113:3300/0 conn(0x7f9d9c06cd80 0x7f9d9c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 --2- 172.21.15.43:0/1632198815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c13f140 0x7f9d9c1606d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- 172.21.15.43:0/1632198815 >> 172.21.15.43:0/1632198815 conn(0x7f9d9c0d00a0 msgr2=0x7f9d9c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- 172.21.15.43:0/1632198815 shutdown_connections 2024-03-30T13:14:34.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- 172.21.15.43:0/1632198815 wait complete. 2024-03-30T13:14:34.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 Processor -- start 2024-03-30T13:14:34.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- start start 2024-03-30T13:14:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c06cd80 0x7f9d9c1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d9c13f140 0x7f9d9c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d9c15ff40 0x7f9d9c151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0cf810 con 0x7f9d9c06cd80 2024-03-30T13:14:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0cf690 con 0x7f9d9c15ff40 2024-03-30T13:14:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c0cf990 con 0x7f9d9c13f140 2024-03-30T13:14:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c06cd80 0x7f9d9c1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c06cd80 0x7f9d9c1532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33816/0 (socket says 172.21.15.43:33816) 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 -- 172.21.15.43:0/4162624786 learned_addr learned my addr 172.21.15.43:0/4162624786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da2af5640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d9c15ff40 0x7f9d9c151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da1af3640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d9c13f140 0x7f9d9c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 -- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d9c13f140 msgr2=0x7f9d9c14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d9c13f140 0x7f9d9c14f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 -- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d9c15ff40 msgr2=0x7f9d9c151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d9c15ff40 0x7f9d9c151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 -- 172.21.15.43:0/4162624786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c14fe50 con 0x7f9d9c06cd80 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da2af5640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d9c15ff40 0x7f9d9c151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da22f4640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c06cd80 0x7f9d9c1532e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9d9801a120 tx=0x7f9d9801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9d92ffd640 1 -- 172.21.15.43:0/4162624786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980bd5f0 con 0x7f9d9c06cd80 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9d92ffd640 1 -- 172.21.15.43:0/4162624786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d980bd790 con 0x7f9d9c06cd80 2024-03-30T13:14:34.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c138da0 con 0x7f9d9c06cd80 2024-03-30T13:14:34.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9d92ffd640 1 -- 172.21.15.43:0/4162624786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980bdb60 con 0x7f9d9c06cd80 2024-03-30T13:14:34.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.602+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c137850 con 0x7f9d9c06cd80 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9d92ffd640 1 -- 172.21.15.43:0/4162624786 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9d980b33f0 con 0x7f9d9c06cd80 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9d92ffd640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d80067fc0 0x7f9d8006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9d92ffd640 1 -- 172.21.15.43:0/4162624786 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f9d98120ea0 con 0x7f9d9c06cd80 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da1af3640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d80067fc0 0x7f9d8006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 Processor -- start 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- start start 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da1af3640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d80067fc0 0x7f9d8006a480 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007fd0 tx=0x7f9d8c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0ab320 0x55d27a0ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b1910 0x55d27a0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b2230 0x55d27a0f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d279ef5aa0 con 0x55d27a0ab320 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d279e7caf0 con 0x55d27a0b2230 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55d279ef1a50 con 0x55d27a0b1910 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0ab320 0x55d27a0ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0ab320 0x55d27a0ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33832/0 (socket says 172.21.15.43:33832) 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 -- 172.21.15.43:0/1642280510 learned_addr learned my addr 172.21.15.43:0/1642280510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da2af5640 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b2230 0x55d27a0f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da1af3640 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b1910 0x55d27a0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 -- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b1910 msgr2=0x55d27a0b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b1910 0x55d27a0b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 -- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b2230 msgr2=0x55d27a0f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b2230 0x55d27a0f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 -- 172.21.15.43:0/1642280510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d279e7ad00 con 0x55d27a0ab320 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da1af3640 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b1910 0x55d27a0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da22f4640 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0ab320 0x55d27a0ab700 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9d980ed1a0 tx=0x7f9d980f09b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0a00852528d374 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9d7ffff640 1 -- 172.21.15.43:0/1642280510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980f9920 con 0x55d27a0ab320 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9d7ffff640 1 -- 172.21.15.43:0/1642280510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d980af040 con 0x55d27a0ab320 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9d7ffff640 1 -- 172.21.15.43:0/1642280510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d980ed3b0 con 0x55d27a0ab320 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- 172.21.15.43:0/1642280510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0ab320 msgr2=0x55d27a0ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0ab320 0x55d27a0ab700 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9d980ed1a0 tx=0x7f9d980f09b0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- 172.21.15.43:0/1642280510 shutdown_connections 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b2230 0x55d27a0f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b1910 0x55d27a0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 --2- 172.21.15.43:0/1642280510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0ab320 0x55d27a0ab700 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- 172.21.15.43:0/1642280510 >> 172.21.15.43:0/1642280510 conn(0x55d27a0acb80 msgr2=0x55d27a0a9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- 172.21.15.43:0/1642280510 shutdown_connections 2024-03-30T13:14:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.606+0000 7f9da76da000 1 -- 172.21.15.43:0/1642280510 wait complete. 2024-03-30T13:14:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 -- start start 2024-03-30T13:14:34.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 Processor -- start 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0ab320 0x55d27a0fbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b1910 0x55d27a0fc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b2230 0x55d27a0dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d279eb1510 con 0x55d27a0b1910 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d279eb36a0 con 0x55d27a0ab320 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55d279fb1b00 con 0x55d27a0b2230 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b1910 0x55d27a0fc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 10 client.0 _ll_register_callbacks cb 0x55d27a092490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b1910 0x55d27a0fc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33836/0 (socket says 172.21.15.43:33836) 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 -- 172.21.15.43:0/3396277482 learned_addr learned my addr 172.21.15.43:0/3396277482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da22f4640 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0ab320 0x55d27a0fbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da2af5640 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b2230 0x55d27a0dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 -- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b2230 msgr2=0x55d27a0dfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b2230 0x55d27a0dfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 -- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0ab320 msgr2=0x55d27a0fbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0ab320 0x55d27a0fbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 -- 172.21.15.43:0/3396277482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d279eba730 con 0x55d27a0b1910 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da1af3640 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b1910 0x55d27a0fc490 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c014bd0 tx=0x7f9d8c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf8ce8e91b4bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/3396277482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c01f410 con 0x55d27a0b1910 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/3396277482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f9d8c01f5b0 con 0x55d27a0b1910 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d980af040 con 0x55d27a0b1910 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/3396277482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c02c050 con 0x55d27a0b1910 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 10 client.6711 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55d279eadca0 con 0x55d27a0b1910 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/3396277482 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d980af040 con 0x55d27a0b1910 2024-03-30T13:14:34.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/3396277482 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f9d8c02f020 con 0x55d27a0b1910 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 10 client.6711 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 10 client.6711 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 20 client.6711 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a0e06c0 0x55d27a0e0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 4 client.6711 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d279e90c90 con 0x55d27a0e06c0 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da22f4640 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a0e06c0 0x55d27a0e0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9d7d7fa640 10 client.6711 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.610+0000 7f9da22f4640 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a0e06c0 0x55d27a0e0ad0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be4b576b0f8617f6 server_cookie=7434c24e96b015a2 in_seq=0 out_seq=0 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/3396277482 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x55d279e90c90 con 0x55d27a0e06c0 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9d7d7fa640 10 client.6711 handle_command_reply: tid=0 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 client.6711 shutdown 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b1910 msgr2=0x55d27a0fc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b1910 0x55d27a0fc490 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c014bd0 tx=0x7f9d8c000bd0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 shutdown_connections 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a0e06c0 0x55d27a0e0ad0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0b2230 0x55d27a0dfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b1910 0x55d27a0fc490 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9d7d7fa640 0 client.6711 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 --2- 172.21.15.43:0/3396277482 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0ab320 0x55d27a0fbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 >> 172.21.15.43:0/3396277482 conn(0x55d27a0acb80 msgr2=0x55d27a0abf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 shutdown_connections 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 1 -- 172.21.15.43:0/3396277482 wait complete. 2024-03-30T13:14:34.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.614+0000 7f9da76da000 20 client.6711 trim_cache size 0 max 16384 2024-03-30T13:14:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.626+0000 7f9da76da000 1 Processor -- start 2024-03-30T13:14:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.626+0000 7f9da76da000 1 -- start start 2024-03-30T13:14:34.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0e0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0b4e30 con 0x55d27a0b2230 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c0b4190 con 0x55d27a0b1910 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9d9c0b47e0 con 0x55d27a0ab320 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0e0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33846/0 (socket says 172.21.15.43:33846) 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/3830101939 learned_addr learned my addr 172.21.15.43:0/3830101939 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da22f4640 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 msgr2=0x55d27a0e20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0e20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da1af3640 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0e0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 msgr2=0x55d27a0e0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0e0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/3830101939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27a0b2060 con 0x55d27a0b2230 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0e0fe0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9d94007680 tx=0x7f9d94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e27c650f2153ec server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9d7f7fe640 1 -- 172.21.15.43:0/3830101939 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9401a070 con 0x55d27a0b2230 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9d7f7fe640 1 -- 172.21.15.43:0/3830101939 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94016830 con 0x55d27a0b2230 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9d7f7fe640 1 -- 172.21.15.43:0/3830101939 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94016c20 con 0x55d27a0b2230 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- 172.21.15.43:0/3830101939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 msgr2=0x55d27a0e0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0e0fe0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9d94007680 tx=0x7f9d94009750 comp rx=0 tx=0).stop 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- 172.21.15.43:0/3830101939 shutdown_connections 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0e0fe0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0e0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- 172.21.15.43:0/3830101939 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0e20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- 172.21.15.43:0/3830101939 >> 172.21.15.43:0/3830101939 conn(0x55d27a0acb80 msgr2=0x55d27a0aa490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- 172.21.15.43:0/3830101939 shutdown_connections 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- 172.21.15.43:0/3830101939 wait complete. 2024-03-30T13:14:34.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- start start 2024-03-30T13:14:34.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 Processor -- start 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0fdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0fe2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0fe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x55d279e8d880 con 0x55d27a0b2230 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x55d279ef4db0 con 0x55d27a0b1910 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x55d279ebd270 con 0x55d27a0ab320 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 10 client.0 _ll_register_callbacks cb 0x7f9d8c0011b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33848/0 (socket says 172.21.15.43:33848) 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/569348117 learned_addr learned my addr 172.21.15.43:0/569348117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da22f4640 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0fdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 msgr2=0x55d27a0fdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0fdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da1af3640 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0fe2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 msgr2=0x55d27a0fe2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0fe2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 -- 172.21.15.43:0/569348117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d279e7cd00 con 0x55d27a0b2230 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da2af5640 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0fe800 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009790 tx=0x7f9d94004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8165461bba28a6b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9d7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/569348117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9401a070 con 0x55d27a0b2230 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/569348117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7f9d9402ad00 con 0x55d27a0b2230 2024-03-30T13:14:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d94016830 con 0x55d27a0b2230 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.630+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/569348117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f9d94031050 con 0x55d27a0b2230 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 10 client.6717 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55d279f76f70 con 0x55d27a0b2230 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/569348117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94016830 con 0x55d27a0b2230 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/569348117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7f9d94034020 con 0x55d27a0b2230 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 10 client.6717 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 10 client.6717 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 20 client.6717 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a165dd0 0x55d27a0fed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 4 client.6717 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628430}] 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628430}) -- 0x55d279f02fe0 con 0x55d27a165dd0 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da22f4640 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a165dd0 0x55d27a0fed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da22f4640 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a165dd0 0x55d27a0fed40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1a19c2a4bc5600 server_cookie=d2fc74b9acb81307 in_seq=0 out_seq=0 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9d7d7fa640 10 client.6717 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9d7d7fa640 1 -- 172.21.15.43:0/569348117 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3798 (crc 0 0 0) 0x55d279f02fe0 con 0x55d27a165dd0 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9d7d7fa640 10 client.6717 handle_command_reply: tid=0 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 client.6717 shutdown 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 msgr2=0x55d27a0fe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0fe800 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9d94009790 tx=0x7f9d94004ac0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 shutdown_connections 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x55d27a165dd0 0x55d27a0fed40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d27a0b2230 0x55d27a0fe800 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x55d27a0b1910 0x55d27a0fe2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9d7d7fa640 0 client.6717 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 --2- 172.21.15.43:0/569348117 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x55d27a0ab320 0x55d27a0fdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 >> 172.21.15.43:0/569348117 conn(0x55d27a0acb80 msgr2=0x55d27a0aaba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 shutdown_connections 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 1 -- 172.21.15.43:0/569348117 wait complete. 2024-03-30T13:14:34.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da76da000 20 client.6717 trim_cache size 0 max 16384 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/symlink.c", 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628430, 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.647599+0000", 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:57.035501+0000", 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:34.645 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.646 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 497, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:57.035501+0000", 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:34.647 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 497, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.647599+0000" 2024-03-30T13:14:34.648 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 497, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.647599+0000" 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 242, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 50, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:34.649 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.650 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.652 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:34.653 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:34.654 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:34.655 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:34.656 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d80067fc0 msgr2=0x7f9d8006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d80067fc0 0x7f9d8006a480 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c007fd0 tx=0x7f9d8c009040 comp rx=0 tx=0).stop 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c06cd80 msgr2=0x7f9d9c1532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c06cd80 0x7f9d9c1532e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9d9801a120 tx=0x7f9d9801ad30 comp rx=0 tx=0).stop 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 shutdown_connections 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9d80067fc0 0x7f9d8006a480 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9d9c15ff40 0x7f9d9c151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9d9c13f140 0x7f9d9c14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 --2- 172.21.15.43:0/4162624786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d9c06cd80 0x7f9d9c1532e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 >> 172.21.15.43:0/4162624786 conn(0x7f9d9c0d00a0 msgr2=0x7f9d9c0cd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 shutdown_connections 2024-03-30T13:14:34.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.634+0000 7f9da4558640 1 -- 172.21.15.43:0/4162624786 wait complete. 2024-03-30T13:14:34.659 DEBUG:tasks.cephfs.test_uninlining:getting inode info #23 of 23:1099511628412 2024-03-30T13:14:34.659 DEBUG:tasks.cephfs.test_uninlining:try #1 - dump inode 1099511628412 2024-03-30T13:14:34.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 dump inode 1099511628412 2024-03-30T13:14:34.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff726168640 1 Processor -- start 2024-03-30T13:14:34.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff726168640 1 -- start start 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff726168640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72006caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff726168640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7200b40b0 0x7ff7200b4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff726168640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff72014c680 con 0x7ff72014c2a0 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff726168640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7ff7200b4a60 con 0x7ff7200b40b0 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff726168640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff720164750 con 0x7ff72006c670 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7200b40b0 0x7ff7200b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7200b40b0 0x7ff7200b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33854/0 (socket says 172.21.15.43:33854) 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71ffff640 1 -- 172.21.15.43:0/2770485413 learned_addr learned my addr 172.21.15.43:0/2770485413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71effd640 1 --2- 172.21.15.43:0/2770485413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72006caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71ffff640 1 -- 172.21.15.43:0/2770485413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 msgr2=0x7ff72006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71ffff640 1 --2- 172.21.15.43:0/2770485413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72006caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71ffff640 1 -- 172.21.15.43:0/2770485413 >> v1:172.21.15.113:6789/0 conn(0x7ff72014c2a0 legacy=0x7ff720164040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.774+0000 7ff71ffff640 1 -- 172.21.15.43:0/2770485413 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7200b3310 con 0x7ff7200b40b0 2024-03-30T13:14:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 --2- 172.21.15.43:0/2770485413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7200b40b0 0x7ff7200b4520 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2571a88d7c1bcb1c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71e7fc640 1 -- 172.21.15.43:0/2770485413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100a7070 con 0x7ff7200b40b0 2024-03-30T13:14:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71e7fc640 1 -- 172.21.15.43:0/2770485413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7100b0070 con 0x7ff7200b40b0 2024-03-30T13:14:34.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71e7fc640 1 -- 172.21.15.43:0/2770485413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100ab890 con 0x7ff7200b40b0 2024-03-30T13:14:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- 172.21.15.43:0/2770485413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7200b40b0 msgr2=0x7ff7200b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 --2- 172.21.15.43:0/2770485413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7200b40b0 0x7ff7200b4520 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff710009870 tx=0x7ff71001c560 comp rx=0 tx=0).stop 2024-03-30T13:14:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- 172.21.15.43:0/2770485413 shutdown_connections 2024-03-30T13:14:34.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 --2- 172.21.15.43:0/2770485413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7200b40b0 0x7ff7200b4520 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 --2- 172.21.15.43:0/2770485413 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72006caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- 172.21.15.43:0/2770485413 >> 172.21.15.43:0/2770485413 conn(0x7ff7200d00a0 msgr2=0x7ff7200d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- 172.21.15.43:0/2770485413 shutdown_connections 2024-03-30T13:14:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- 172.21.15.43:0/2770485413 wait complete. 2024-03-30T13:14:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 Processor -- start 2024-03-30T13:14:34.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- start start 2024-03-30T13:14:34.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72015bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7200b40b0 0x7ff7201484b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72014c2a0 0x7ff7201489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff720163b20 con 0x7ff72014c2a0 2024-03-30T13:14:34.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201639a0 con 0x7ff7200b40b0 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff726168640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff720163ca0 con 0x7ff72006c670 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72014c2a0 0x7ff7201489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72014c2a0 0x7ff7201489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33870/0 (socket says 172.21.15.43:33870) 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 -- 172.21.15.43:0/1176885878 learned_addr learned my addr 172.21.15.43:0/1176885878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72015bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71effd640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7200b40b0 0x7ff7201484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 -- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 msgr2=0x7ff72015bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72015bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 -- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7200b40b0 msgr2=0x7ff7201484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.778+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7200b40b0 0x7ff7201484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff71ffff640 1 -- 172.21.15.43:0/1176885878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7201405c0 con 0x7ff72014c2a0 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff71effd640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7200b40b0 0x7ff7201484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72014c2a0 0x7ff7201489f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff71001a190 tx=0x7ff71001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff707fff640 1 -- 172.21.15.43:0/1176885878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100be5f0 con 0x7ff72014c2a0 2024-03-30T13:14:34.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff707fff640 1 -- 172.21.15.43:0/1176885878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7100be790 con 0x7ff72014c2a0 2024-03-30T13:14:34.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201408a0 con 0x7ff72014c2a0 2024-03-30T13:14:34.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff707fff640 1 -- 172.21.15.43:0/1176885878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100beb40 con 0x7ff72014c2a0 2024-03-30T13:14:34.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff720160c00 con 0x7ff72014c2a0 2024-03-30T13:14:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff707fff640 1 -- 172.21.15.43:0/1176885878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff7100b43f0 con 0x7ff72014c2a0 2024-03-30T13:14:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff707fff640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff70c067fc0 0x7ff70c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff707fff640 1 -- 172.21.15.43:0/1176885878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7ff710120df0 con 0x7ff72014c2a0 2024-03-30T13:14:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff70c067fc0 0x7ff70c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff7292ea000 1 Processor -- start 2024-03-30T13:14:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff7292ea000 1 -- start start 2024-03-30T13:14:34.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.782+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff70c067fc0 0x7ff70c06a480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7180012f0 tx=0x7ff718009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c025150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3bfdb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfdbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d3be29aa0 con 0x563d3c025690 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563d3bdb0af0 con 0x563d3bfdb830 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563d3be25a50 con 0x563d3c024d70 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3bfdb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3bfdb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33882/0 (socket says 172.21.15.43:33882) 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1753328299 learned_addr learned my addr 172.21.15.43:0/1753328299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c025150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfdbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 msgr2=0x563d3c025150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c025150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 msgr2=0x563d3bfdbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfdbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1753328299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d3bdaed00 con 0x563d3c025690 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c025150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3bfdb2f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff714009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb30096fb0cc4e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7057fa640 1 -- 172.21.15.43:0/1753328299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71400c070 con 0x563d3c025690 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7057fa640 1 -- 172.21.15.43:0/1753328299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff714018070 con 0x563d3c025690 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7057fa640 1 -- 172.21.15.43:0/1753328299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71400f040 con 0x563d3c025690 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- 172.21.15.43:0/1753328299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 msgr2=0x563d3bfdb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3bfdb2f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff714009900 tx=0x7ff714009dc0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- 172.21.15.43:0/1753328299 shutdown_connections 2024-03-30T13:14:34.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfdbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3bfdb2f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1753328299 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c025150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- 172.21.15.43:0/1753328299 >> 172.21.15.43:0/1753328299 conn(0x563d3bfd77e0 msgr2=0x563d3bfd9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- 172.21.15.43:0/1753328299 shutdown_connections 2024-03-30T13:14:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- 172.21.15.43:0/1753328299 wait complete. 2024-03-30T13:14:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- start start 2024-03-30T13:14:34.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 Processor -- start 2024-03-30T13:14:34.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfe0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c024d70 0x563d3c009da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3c00a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d3bdf1270 con 0x563d3c024d70 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x563d3bde76a0 con 0x563d3bfdb830 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563d3bdc1880 con 0x563d3c025690 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff7292ea000 10 client.0 _ll_register_callbacks cb 0x563d3bfc6490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c024d70 0x563d3c009da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c024d70 0x563d3c009da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33894/0 (socket says 172.21.15.43:33894) 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1601878121 learned_addr learned my addr 172.21.15.43:0/1601878121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3c00a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfe0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 msgr2=0x563d3c00a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3c00a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 msgr2=0x563d3bfe0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfe0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 -- 172.21.15.43:0/1601878121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d3bdee730 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3c00a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff71effd640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c024d70 0x563d3c009da0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x563d3bee9c80 tx=0x7ff714004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6283f76d81c43b73 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.786+0000 7ff6fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1601878121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71400fdd0 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1601878121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7ff714027cf0 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff714018070 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1601878121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff7140044e0 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 10 client.6729 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x563d3bde1ca0 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1601878121 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff714018070 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1601878121 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7ff71402e050 con 0x563d3c024d70 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 10 client.6729 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 10 client.6729 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 20 client.6729 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c02c5d0 0x563d3c02c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 4 client.6729 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d3bdc4c90 con 0x563d3c02c5d0 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c02c5d0 0x563d3c02c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 10 client.6729 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c02c5d0 0x563d3c02c9e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7022640983238283 server_cookie=fa088cdaa6909aea in_seq=0 out_seq=0 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1601878121 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23239 (crc 0 0 0) 0x563d3bdc4c90 con 0x563d3c02c5d0 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 10 client.6729 handle_command_reply: tid=0 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 client.6729 shutdown 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c024d70 msgr2=0x563d3c009da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c024d70 0x563d3c009da0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x563d3bee9c80 tx=0x7ff714004740 comp rx=0 tx=0).stop 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 shutdown_connections 2024-03-30T13:14:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c02c5d0 0x563d3c02c9e0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3c00a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c024d70 0x563d3c009da0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1601878121 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3bfe0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff6fe7fc640 0 client.6729 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 >> 172.21.15.43:0/1601878121 conn(0x563d3bfd77e0 msgr2=0x563d3bfda460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 shutdown_connections 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 1 -- 172.21.15.43:0/1601878121 wait complete. 2024-03-30T13:14:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.790+0000 7ff7292ea000 20 client.6729 trim_cache size 0 max 16384 2024-03-30T13:14:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 Processor -- start 2024-03-30T13:14:34.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- start start 2024-03-30T13:14:34.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3bfdb830 0x563d3bfdb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3c024d70 0x563d3bfd9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3bfda110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff7202083b0 con 0x563d3bfdb830 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff720207410 con 0x563d3c024d70 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff720161db0 con 0x563d3c025690 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3bfdb830 0x563d3bfdb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3bfdb830 0x563d3bfdb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33906/0 (socket says 172.21.15.43:33906) 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 -- 172.21.15.43:0/861784305 learned_addr learned my addr 172.21.15.43:0/861784305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71ffff640 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3bfda110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71effd640 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3c024d70 0x563d3bfd9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 -- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 msgr2=0x563d3bfda110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3bfda110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 -- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3c024d70 msgr2=0x563d3bfd9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3c024d70 0x563d3bfd9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 -- 172.21.15.43:0/861784305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d3bdb0d00 con 0x563d3bfdb830 2024-03-30T13:14:34.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71effd640 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3c024d70 0x563d3bfd9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:34.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3bfdb830 0x563d3bfdb000 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff718003f50 tx=0x7ff718012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e5e88650737879 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff704ff9640 1 -- 172.21.15.43:0/861784305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71800ed10 con 0x563d3bfdb830 2024-03-30T13:14:34.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff704ff9640 1 -- 172.21.15.43:0/861784305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff718005550 con 0x563d3bfdb830 2024-03-30T13:14:34.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff704ff9640 1 -- 172.21.15.43:0/861784305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718005700 con 0x563d3bfdb830 2024-03-30T13:14:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- 172.21.15.43:0/861784305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3bfdb830 msgr2=0x563d3bfdb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3bfdb830 0x563d3bfdb000 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff718003f50 tx=0x7ff718012000 comp rx=0 tx=0).stop 2024-03-30T13:14:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- 172.21.15.43:0/861784305 shutdown_connections 2024-03-30T13:14:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c025690 0x563d3bfda110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3c024d70 0x563d3bfd9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 --2- 172.21.15.43:0/861784305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3bfdb830 0x563d3bfdb000 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- 172.21.15.43:0/861784305 >> 172.21.15.43:0/861784305 conn(0x563d3bfd77e0 msgr2=0x563d3c0082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- 172.21.15.43:0/861784305 shutdown_connections 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.806+0000 7ff7292ea000 1 -- 172.21.15.43:0/861784305 wait complete. 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- start start 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 Processor -- start 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3c009c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c00a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3c091fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x563d3bdf1270 con 0x563d3c025690 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff720206e20 con 0x563d3bfdb830 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x563d3bfdb540 con 0x563d3c024d70 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 10 client.0 _ll_register_callbacks cb 0x7ff714009750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-30T13:14:34.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3c091fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3c091fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33922/0 (socket says 172.21.15.43:33922) 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 -- 172.21.15.43:0/1474338720 learned_addr learned my addr 172.21.15.43:0/1474338720 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71effd640 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c00a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3c009c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 -- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 msgr2=0x563d3c00a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c00a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 -- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 msgr2=0x563d3c009c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3c009c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 -- 172.21.15.43:0/1474338720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d3beaaf70 con 0x563d3c025690 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71ffff640 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3c091fa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff71001ca40 tx=0x7ff7100f0e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88abcf4a6cbf62c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1474338720 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100f3040 con 0x563d3c025690 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1474338720 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7ff7100ed4b0 con 0x563d3c025690 2024-03-30T13:14:34.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff718005550 con 0x563d3c025690 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1474338720 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff710101050 con 0x563d3c025690 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 10 client.6735 fetch_fsmap learned FSMap version 21 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x563d3bdd0760 con 0x563d3c025690 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1474338720 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff718005550 con 0x563d3c025690 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1474338720 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) ==== 3494+0+0 (secure 0 0 0) 0x7ff710104020 con 0x563d3c025690 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 10 client.6735 fetch_fsmap finished waiting for FSMap version 21 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 10 client.6735 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.d 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 20 client.6735 populate_metadata read hostname 'smithi043' 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c092930 0x563d3c092d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 4 client.6735 mds_command: new command op to 4880 tid=0 multi_id=0 [{"prefix": "dump inode", "number": 1099511628412}] 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 --> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] -- command(tid 0: {"prefix": "dump inode", "number": 1099511628412}) -- 0x563d3be36fe0 con 0x563d3c092930 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c092930 0x563d3c092d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff71f7fe640 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c092930 0x563d3c092d10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ee890c5d15bcb5 server_cookie=733a2e12146645a4 in_seq=0 out_seq=0 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 10 client.6735 ms_handle_connect on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 1 -- 172.21.15.43:0/1474338720 <== mds.0 v2:172.21.15.113:6836/1104336007 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3802 (crc 0 0 0) 0x563d3be36fe0 con 0x563d3c092930 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 10 client.6735 handle_command_reply: tid=0 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 client.6735 shutdown 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 msgr2=0x563d3c091fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3c091fa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7ff71001ca40 tx=0x7ff7100f0e40 comp rx=0 tx=0).stop 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 shutdown_connections 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:6836/1104336007,v1:172.21.15.113:6837/1104336007] conn(0x563d3c092930 0x563d3c092d10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563d3c025690 0x563d3c091fa0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x563d3c024d70 0x563d3c00a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff6fe7fc640 0 client.6735 ms_handle_reset on v2:172.21.15.113:6836/1104336007 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.810+0000 7ff7292ea000 1 --2- 172.21.15.43:0/1474338720 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x563d3bfdb830 0x563d3c009c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 >> 172.21.15.43:0/1474338720 conn(0x563d3bfd77e0 msgr2=0x563d3bfd9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 shutdown_connections 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff7292ea000 1 -- 172.21.15.43:0/1474338720 wait complete. 2024-03-30T13:14:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff7292ea000 20 client.6735 trim_cache size 0 max 16384 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/linux-5.4/fs/orangefs/downcall.h", 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511628412, 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "rdev": 0, 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "ctime": "2024-03-30T13:13:58.651794+0000", 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "btime": "2024-03-30T13:13:56.944931+0000", 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": 33204, 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "uid": 1000, 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "gid": 1271, 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "nlink": 1, 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_layout": { 2024-03-30T13:14:34.822 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_hash": 0, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "unused1": 0, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "unused2": 0, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "unused3": 0 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "layout": { 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_unit": 4194304, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "stripe_count": 1, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "object_size": 4194304, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_id": 6, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "pool_ns": "" 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "old_pools": [], 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "size": 2953, 2024-03-30T13:14:34.823 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_seq": 1, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_size": 18446744073709551615, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_from": 0, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "truncate_pending": 0, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "2019-11-25T00:32:01.000000+0000", 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "atime": "2024-03-30T13:13:56.944931+0000", 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "time_warp_seq": 1, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 3, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "export_pin": -1, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_random_pin": 0, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "export_ephemeral_distributed_pin": false, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "quiesce_block": false, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "client_ranges": [], 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "dirstat": { 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.824 INFO:teuthology.orchestra.run.smithi043.stdout: "mtime": "0.000000", 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "num_files": 0, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "num_subdirs": 0, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "change_attr": 0 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "rstat": { 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2953, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.651794+0000" 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "accounted_rstat": { 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 0, 2024-03-30T13:14:34.825 INFO:teuthology.orchestra.run.smithi043.stdout: "rbytes": 2953, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "rfiles": 1, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "rsubdirs": 0, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "rsnaps": 0, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "rctime": "2024-03-30T13:13:58.651794+0000" 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "version": 244, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "file_data_version": 0, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "xattr_version": 1, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "backtrace_version": 14, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_version": 18446744073709551615, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "inline_data_length": 0, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "stray_prior_path": "", 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max_size_ever": 0, 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "quota": { 2024-03-30T13:14:34.826 INFO:teuthology.orchestra.run.smithi043.stdout: "max_bytes": 0, 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "max_files": 0 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_stamp": "0.000000", 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "last_scrub_version": 0, 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "symlink": "", 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrs": [], 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtree": { 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "splits": [] 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "old_inodes": [], 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "oldest_snap": 6, 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_flags": 0, 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "is_auth": true, 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_state": { 2024-03-30T13:14:34.827 INFO:teuthology.orchestra.run.smithi043.stdout: "replicas": {} 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_state": { 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "authority": [ 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: 0, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: -2 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "replica_nonce": 0 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_pins": 0, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "is_frozen": false, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "is_freezing": false, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "pins": { 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "ptrwaiter": 0, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 0, 2024-03-30T13:14:34.828 INFO:teuthology.orchestra.run.smithi043.stdout: "lock": 0, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 1, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent": 1, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty": 1, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "waiter": 0, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "authpin": 0, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubqueue": 0 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "nref": 3, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "versionlock": { 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iversion", 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.829 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "authlock": {}, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "linklock": {}, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "dirfragtreelock": {}, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "filelock": {}, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "xattrlock": {}, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "snaplock": {}, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "nestlock": { 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "inest", 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.830 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "flocklock": {}, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "policylock": {}, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "quiescelock": { 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "gather_set": [], 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "lock", 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "iquiesce", 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "is_leased": false, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "num_rdlocks": 0, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "num_wrlocks": 0, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "num_xlocks": 0, 2024-03-30T13:14:34.831 INFO:teuthology.orchestra.run.smithi043.stdout: "xlock_by": {} 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "states": [ 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "auth", 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "dirty", 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtyparent", 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "dirtypool" 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "client_caps": [ 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "client_id": 5757, 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "pending": "pAsLsXsFscr", 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "issued": "pAsLsXsFscr", 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "wanted": "-", 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: "last_sent": 6 2024-03-30T13:14:34.832 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-03-30T13:14:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: ], 2024-03-30T13:14:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "loner": -1, 2024-03-30T13:14:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "want_loner": -1, 2024-03-30T13:14:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_caps_wanted": [], 2024-03-30T13:14:34.833 INFO:teuthology.orchestra.run.smithi043.stdout: "is_subvolume": false 2024-03-30T13:14:34.833 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:34.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff70c067fc0 msgr2=0x7ff70c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff70c067fc0 0x7ff70c06a480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff7180012f0 tx=0x7ff718009040 comp rx=0 tx=0).stop 2024-03-30T13:14:34.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72014c2a0 msgr2=0x7ff7201489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:34.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72014c2a0 0x7ff7201489f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff71001a190 tx=0x7ff71001ada0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 shutdown_connections 2024-03-30T13:14:34.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff70c067fc0 0x7ff70c06a480 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72014c2a0 0x7ff7201489f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff7200b40b0 0x7ff7201484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 --2- 172.21.15.43:0/1176885878 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff72006c670 0x7ff72015bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:34.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 >> 172.21.15.43:0/1176885878 conn(0x7ff7200d00a0 msgr2=0x7ff7200d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:34.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 shutdown_connections 2024-03-30T13:14:34.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:34.814+0000 7ff726168640 1 -- 172.21.15.43:0/1176885878 wait complete. 2024-03-30T13:14:34.835 DEBUG:tasks.cephfs.test_uninlining:removing dir linux-5.4 2024-03-30T13:14:34.835 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && rm -rf linux-5.4/ 2024-03-30T13:14:35.416 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_length == 0 2024-03-30T13:14:35.416 INFO:tasks.cephfs.test_uninlining:Found 23 files with inline_data_version == CEPH_INLINE_NONE 2024-03-30T13:14:35.416 INFO:tasks.cephfs.test_uninlining:Running Data Uninlining test with: clear cache before scrub, without unmount client, without mount.open_background, without multimds, without snapshots 2024-03-30T13:14:35.416 DEBUG:tasks.cephfs.test_uninlining:setting max_mds:1 2024-03-30T13:14:35.416 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-03-30T13:14:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.550+0000 7fcdcc7cb640 1 Processor -- start 2024-03-30T13:14:35.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.550+0000 7fcdcc7cb640 1 -- start start 2024-03-30T13:14:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdcc7cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdc40b9e30 0x7fcdc40cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdcc7cb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc40cf4f0 0x7fcdc40d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdcc7cb640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdc406c850 con 0x7fcdc415e4c0 2024-03-30T13:14:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdcc7cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fcdc40ba290 con 0x7fcdc40b9e30 2024-03-30T13:14:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdcc7cb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc415e8a0 con 0x7fcdc40cf4f0 2024-03-30T13:14:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdc9d66640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdc40b9e30 0x7fcdc40cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:35.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdc9d66640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdc40b9e30 0x7fcdc40cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33926/0 (socket says 172.21.15.43:33926) 2024-03-30T13:14:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdc9d66640 1 -- 172.21.15.43:0/1023506437 learned_addr learned my addr 172.21.15.43:0/1023506437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdcad68640 1 --2- 172.21.15.43:0/1023506437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc40cf4f0 0x7fcdc40d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdc9d66640 1 -- 172.21.15.43:0/1023506437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc40cf4f0 msgr2=0x7fcdc40d2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdc9d66640 1 --2- 172.21.15.43:0/1023506437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc40cf4f0 0x7fcdc40d2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdc9d66640 1 -- 172.21.15.43:0/1023506437 >> v1:172.21.15.113:6789/0 conn(0x7fcdc415e4c0 legacy=0x7fcdc406c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:35.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.554+0000 7fcdc9d66640 1 -- 172.21.15.43:0/1023506437 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc406cb30 con 0x7fcdc40b9e30 2024-03-30T13:14:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdc9d66640 1 --2- 172.21.15.43:0/1023506437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc40cefb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009870 tx=0x7fcdb801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c95dfca14d8c604 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdc9565640 1 -- 172.21.15.43:0/1023506437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80b2070 con 0x7fcdc40b9e30 2024-03-30T13:14:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdc9565640 1 -- 172.21.15.43:0/1023506437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb80a9070 con 0x7fcdc40b9e30 2024-03-30T13:14:35.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdc9565640 1 -- 172.21.15.43:0/1023506437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb80ad880 con 0x7fcdc40b9e30 2024-03-30T13:14:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/1023506437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 msgr2=0x7fcdc40cefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/1023506437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc40cefb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009870 tx=0x7fcdb801c560 comp rx=0 tx=0).stop 2024-03-30T13:14:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/1023506437 shutdown_connections 2024-03-30T13:14:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/1023506437 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc40cf4f0 0x7fcdc40d2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/1023506437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc40cefb0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/1023506437 >> 172.21.15.43:0/1023506437 conn(0x7fcdc40b6410 msgr2=0x7fcdc40d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/1023506437 shutdown_connections 2024-03-30T13:14:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/1023506437 wait complete. 2024-03-30T13:14:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 Processor -- start 2024-03-30T13:14:35.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- start start 2024-03-30T13:14:35.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc40cf4f0 0x7fcdc415d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc415e4c0 0x7fcdc4155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40d3ed0 con 0x7fcdc40b9e30 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fcdc40d3d50 con 0x7fcdc40cf4f0 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdcc7cb640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fcdc40d4050 con 0x7fcdc415e4c0 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdca567640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdca567640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc4156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33932/0 (socket says 172.21.15.43:33932) 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.558+0000 7fcdca567640 1 -- 172.21.15.43:0/3407766295 learned_addr learned my addr 172.21.15.43:0/3407766295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdcad68640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc415e4c0 0x7fcdc4155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdc9d66640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc40cf4f0 0x7fcdc415d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdca567640 1 -- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc415e4c0 msgr2=0x7fcdc4155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdca567640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc415e4c0 0x7fcdc4155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:35.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdca567640 1 -- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc40cf4f0 msgr2=0x7fcdc415d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdca567640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc40cf4f0 0x7fcdc415d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdca567640 1 -- 172.21.15.43:0/3407766295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdc4144e10 con 0x7fcdc40b9e30 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdcad68640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc415e4c0 0x7fcdc4155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdc9d66640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc40cf4f0 0x7fcdc415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdca567640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc4156de0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c730 tx=0x7fcdc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdb6ffd640 1 -- 172.21.15.43:0/3407766295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0016020 con 0x7fcdc40b9e30 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdb6ffd640 1 -- 172.21.15.43:0/3407766295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc0006d60 con 0x7fcdc40b9e30 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdc4156300 con 0x7fcdc40b9e30 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdb6ffd640 1 -- 172.21.15.43:0/3407766295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0003bc0 con 0x7fcdc40b9e30 2024-03-30T13:14:35.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdc4157320 con 0x7fcdc40b9e30 2024-03-30T13:14:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdc4156100 con 0x7fcdc40b9e30 2024-03-30T13:14:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdb6ffd640 1 -- 172.21.15.43:0/3407766295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fcdc000a850 con 0x7fcdc40b9e30 2024-03-30T13:14:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdb6ffd640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcda8067fc0 0x7fcda806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdb6ffd640 1 -- 172.21.15.43:0/3407766295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 6231+0+0 (secure 0 0 0) 0x7fcdc0087100 con 0x7fcdc40b9e30 2024-03-30T13:14:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.562+0000 7fcdc9d66640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcda8067fc0 0x7fcda806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:35.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.566+0000 7fcdc9d66640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcda8067fc0 0x7fcda806a480 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80af000 tx=0x7fcdb8008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:35.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.566+0000 7fcdb6ffd640 1 -- 172.21.15.43:0/3407766295 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fcdc0020020 con 0x7fcdc40b9e30 2024-03-30T13:14:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:35.766+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) -- 0x7fcdc4068900 con 0x7fcdc40b9e30 2024-03-30T13:14:36.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.094+0000 7fcdb6ffd640 1 -- 172.21.15.43:0/3407766295 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v22) ==== 105+0+0 (secure 0 0 0) 0x7fcdc0023e40 con 0x7fcdc40b9e30 2024-03-30T13:14:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcda8067fc0 msgr2=0x7fcda806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcda8067fc0 0x7fcda806a480 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80af000 tx=0x7fcdb8008040 comp rx=0 tx=0).stop 2024-03-30T13:14:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 msgr2=0x7fcdc4156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc4156de0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c730 tx=0x7fcdc000cbf0 comp rx=0 tx=0).stop 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 shutdown_connections 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fcda8067fc0 0x7fcda806a480 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fcdc415e4c0 0x7fcdc4155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fcdc40cf4f0 0x7fcdc415d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 --2- 172.21.15.43:0/3407766295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdc40b9e30 0x7fcdc4156de0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 >> 172.21.15.43:0/3407766295 conn(0x7fcdc40b6410 msgr2=0x7fcdc4157fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 shutdown_connections 2024-03-30T13:14:36.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:36.098+0000 7fcdcc7cb640 1 -- 172.21.15.43:0/3407766295 wait complete. 2024-03-30T13:14:36.126 DEBUG:tasks.cephfs.test_uninlining:getting client_max_inline_size 2024-03-30T13:14:36.126 DEBUG:tasks.cephfs.test_uninlining:got client_max_inline_size:4096 2024-03-30T13:14:36.126 DEBUG:tasks.cephfs.test_uninlining:extracting tarball 2024-03-30T13:14:36.126 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && tar -x -z -f linux-5.4.tar.gz linux-5.4/fs/ceph linux-5.4/fs/orangefs linux-5.4/fs/ext2 2024-03-30T13:14:40.985 DEBUG:tasks.cephfs.test_uninlining:fetching dir list 2024-03-30T13:14:40.985 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -mindepth 2 -maxdepth 2 -type d 2024-03-30T13:14:41.053 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph 2024-03-30T13:14:41.053 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2 2024-03-30T13:14:41.053 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs 2024-03-30T13:14:41.054 DEBUG:tasks.cephfs.test_uninlining:distributing dir pins 2024-03-30T13:14:41.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:41.054 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ceph) 2024-03-30T13:14:41.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:41.064 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/ext2) 2024-03-30T13:14:41.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:41.112 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 linux-5.4/fs/orangefs) 2024-03-30T13:14:41.161 DEBUG:tasks.cephfs.test_uninlining:listing files with size <= 4096 2024-03-30T13:14:41.162 DEBUG:teuthology.orchestra.run.smithi043:> cd /home/ubuntu/cephtest/mnt.0 && find linux-5.4/ -type f -size -4097c 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/Kconfig 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/Makefile 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/ceph_frag.c 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/io.h 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ceph/ioctl.h 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/Kconfig 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_user.c 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_security.c 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/Makefile 2024-03-30T13:14:41.270 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/symlink.c 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/acl.h 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr_trusted.c 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/ext2/xattr.h 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/Kconfig 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-debug.h 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-sysfs.h 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/dcache.c 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-bufmap.h 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-debugfs.h 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/orangefs-dev-proto.h 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/Makefile 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/symlink.c 2024-03-30T13:14:41.271 INFO:teuthology.orchestra.run.smithi043.stdout:linux-5.4/fs/orangefs/downcall.h 2024-03-30T13:14:41.272 INFO:tasks.cephfs.test_uninlining:Found 23 inlined files 2024-03-30T13:14:41.272 DEBUG:tasks.cephfs.test_uninlining:clearing cache 2024-03-30T13:14:41.272 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-30T13:14:41.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:14:41.326 DEBUG:teuthology.orchestra.run.smithi113:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-30T13:14:41.378 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-30T13:14:41.379 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:14:41.379 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:14:41.379 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:41.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:14:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.553+0000 7f5d86dec640 1 Processor -- start 2024-03-30T13:14:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.553+0000 7f5d86dec640 1 -- start start 2024-03-30T13:14:41.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d86dec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5d8006c670 0x7f5d800cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d86dec640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f5d800b41f0 0x7f5d8006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d86dec640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d8006e5c0 con 0x7f5d8006ca50 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d86dec640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5d800b34c0 con 0x7f5d8006c670 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d86dec640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f5d800ccef0 con 0x7f5d800b41f0 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d84b88640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5d8006c670 0x7f5d800cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d85389640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5d8006ca50 0x7f5d8015fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33954/0 (socket says 172.21.15.43:33954) 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d85389640 1 -- 172.21.15.43:0/1580767522 learned_addr learned my addr 172.21.15.43:0/1580767522 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/1580767522 >> v2:172.21.15.113:3300/0 conn(0x7f5d800b41f0 0x7f5d8006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d84b88640 1 -- 172.21.15.43:0/1580767522 >> v2:172.21.15.113:3300/0 conn(0x7f5d800b41f0 msgr2=0x7f5d8006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d84b88640 1 --2- 172.21.15.43:0/1580767522 >> v2:172.21.15.113:3300/0 conn(0x7f5d800b41f0 0x7f5d8006e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d84b88640 1 -- 172.21.15.43:0/1580767522 >> v1:172.21.15.43:6789/0 conn(0x7f5d8006ca50 legacy=0x7f5d8015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d84b88640 1 -- 172.21.15.43:0/1580767522 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d800b3710 con 0x7f5d8006c670 2024-03-30T13:14:41.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.557+0000 7f5d7f7fe640 1 -- 172.21.15.43:0/1580767522 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2482346406 0 0) 0x7f5d8006e5c0 con 0x7f5d8006ca50 2024-03-30T13:14:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d84b88640 1 --2- 172.21.15.43:0/1580767522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d800cc910 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5d8006ebe0 tx=0x7f5d7801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578ae5c361bad568 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d7f7fe640 1 -- 172.21.15.43:0/1580767522 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780b0070 con 0x7f5d8006c670 2024-03-30T13:14:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d7f7fe640 1 -- 172.21.15.43:0/1580767522 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d780ab600 con 0x7f5d8006c670 2024-03-30T13:14:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d7f7fe640 1 -- 172.21.15.43:0/1580767522 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780ab9d0 con 0x7f5d8006c670 2024-03-30T13:14:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- 172.21.15.43:0/1580767522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 msgr2=0x7f5d800cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 --2- 172.21.15.43:0/1580767522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d800cc910 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5d8006ebe0 tx=0x7f5d7801c480 comp rx=0 tx=0).stop 2024-03-30T13:14:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- 172.21.15.43:0/1580767522 shutdown_connections 2024-03-30T13:14:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 --2- 172.21.15.43:0/1580767522 >> v2:172.21.15.113:3300/0 conn(0x7f5d800b41f0 0x7f5d8006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 --2- 172.21.15.43:0/1580767522 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d800cc910 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- 172.21.15.43:0/1580767522 >> 172.21.15.43:0/1580767522 conn(0x7f5d800cfee0 msgr2=0x7f5d800d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- 172.21.15.43:0/1580767522 shutdown_connections 2024-03-30T13:14:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- 172.21.15.43:0/1580767522 wait complete. 2024-03-30T13:14:41.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 Processor -- start 2024-03-30T13:14:41.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- start start 2024-03-30T13:14:41.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d80134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d8006ca50 0x7f5d8013dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d800b41f0 0x7f5d801333c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8006cfe0 con 0x7f5d8006c670 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8006ce60 con 0x7f5d8006ca50 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d86dec640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5d80168ef0 con 0x7f5d800b41f0 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d84b88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d80134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d84b88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d80134f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44034/0 (socket says 172.21.15.43:44034) 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d84b88640 1 -- 172.21.15.43:0/4194862537 learned_addr learned my addr 172.21.15.43:0/4194862537 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d85389640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d800b41f0 0x7f5d801333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.561+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d8006ca50 0x7f5d8013dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d84b88640 1 -- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d800b41f0 msgr2=0x7f5d801333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d84b88640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d800b41f0 0x7f5d801333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d84b88640 1 -- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d8006ca50 msgr2=0x7f5d8013dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d84b88640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d8006ca50 0x7f5d8013dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d84b88640 1 -- 172.21.15.43:0/4194862537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d80136600 con 0x7f5d8006c670 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d84b88640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d80134f40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5d7801a0b0 tx=0x7f5d78019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7d7fa640 1 -- 172.21.15.43:0/4194862537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780b8020 con 0x7f5d8006c670 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7d7fa640 1 -- 172.21.15.43:0/4194862537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d780b4630 con 0x7f5d8006c670 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8013ccb0 con 0x7f5d8006c670 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7d7fa640 1 -- 172.21.15.43:0/4194862537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d780b0040 con 0x7f5d8006c670 2024-03-30T13:14:41.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8013d4f0 con 0x7f5d8006c670 2024-03-30T13:14:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d80068890 con 0x7f5d8006c670 2024-03-30T13:14:41.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7d7fa640 1 -- 172.21.15.43:0/4194862537 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5d780bb020 con 0x7f5d8006c670 2024-03-30T13:14:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7d7fa640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5d6c067fc0 0x7f5d6c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7d7fa640 1 -- 172.21.15.43:0/4194862537 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f5d781202d0 con 0x7f5d8006c670 2024-03-30T13:14:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5d6c067fc0 0x7f5d6c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.565+0000 7f5d7ffff640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5d6c067fc0 0x7f5d6c06a480 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d74003fd0 tx=0x7f5d740039a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:41.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.569+0000 7f5d7d7fa640 1 -- 172.21.15.43:0/4194862537 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f5d780c2030 con 0x7f5d8006c670 2024-03-30T13:14:41.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.733+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d800b32a0 con 0x7f5d8006c670 2024-03-30T13:14:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d7d7fa640 1 -- 172.21.15.43:0/4194862537 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+17021 (secure 0 0 0) 0x7f5d8013ccb0 con 0x7f5d8006c670 2024-03-30T13:14:41.741 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:41.741 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:14:40.895915+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3c","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:14:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5d6c067fc0 msgr2=0x7f5d6c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5d6c067fc0 0x7f5d6c06a480 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d74003fd0 tx=0x7f5d740039a0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 msgr2=0x7f5d80134f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d80134f40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5d7801a0b0 tx=0x7f5d78019220 comp rx=0 tx=0).stop 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 shutdown_connections 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5d6c067fc0 0x7f5d6c06a480 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5d800b41f0 0x7f5d801333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5d8006ca50 0x7f5d8013dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 --2- 172.21.15.43:0/4194862537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5d8006c670 0x7f5d80134f40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 >> 172.21.15.43:0/4194862537 conn(0x7f5d800cfee0 msgr2=0x7f5d800b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 shutdown_connections 2024-03-30T13:14:41.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.737+0000 7f5d86dec640 1 -- 172.21.15.43:0/4194862537 wait complete. 2024-03-30T13:14:41.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:12:25.924561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:12:26.302922+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 45, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:14:41.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:14:41.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6f39b640 1 Processor -- start 2024-03-30T13:14:41.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6f39b640 1 -- start start 2024-03-30T13:14:41.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6f39b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3f6815e540 0x7f3f680d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6f39b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f680cf840 0x7f3f6806bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6f39b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f680d3ab0 con 0x7f3f680ba090 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6f39b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3f680d4070 con 0x7f3f680cf840 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6f39b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3f680d3d90 con 0x7f3f6815e540 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6d938640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3f680cf840 0x7f3f6806bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6c936640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3f680ba090 0x7f3f680cf130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33970/0 (socket says 172.21.15.43:33970) 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6c936640 1 -- 172.21.15.43:0/3069038932 learned_addr learned my addr 172.21.15.43:0/3069038932 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6d137640 1 --2- 172.21.15.43:0/3069038932 >> v2:172.21.15.113:3300/0 conn(0x7f3f6815e540 0x7f3f680d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f5ffff640 1 -- 172.21.15.43:0/3069038932 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718066447 0 0) 0x7f3f680d3ab0 con 0x7f3f680ba090 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6d938640 1 -- 172.21.15.43:0/3069038932 >> v2:172.21.15.113:3300/0 conn(0x7f3f6815e540 msgr2=0x7f3f680d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6d938640 1 --2- 172.21.15.43:0/3069038932 >> v2:172.21.15.113:3300/0 conn(0x7f3f6815e540 0x7f3f680d3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6d938640 1 -- 172.21.15.43:0/3069038932 >> v1:172.21.15.43:6789/0 conn(0x7f3f680ba090 legacy=0x7f3f680cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.885+0000 7f3f6d938640 1 -- 172.21.15.43:0/3069038932 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6806b790 con 0x7f3f680cf840 2024-03-30T13:14:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6d938640 1 --2- 172.21.15.43:0/3069038932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f680cf840 0x7f3f6806bf90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3f680d2700 tx=0x7f3f6001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4793dae759f8a60 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f5ffff640 1 -- 172.21.15.43:0/3069038932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600b2070 con 0x7f3f680cf840 2024-03-30T13:14:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f5ffff640 1 -- 172.21.15.43:0/3069038932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f600a5870 con 0x7f3f680cf840 2024-03-30T13:14:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f5ffff640 1 -- 172.21.15.43:0/3069038932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600a5c40 con 0x7f3f680cf840 2024-03-30T13:14:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 -- 172.21.15.43:0/3069038932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f680cf840 msgr2=0x7f3f6806bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/3069038932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f680cf840 0x7f3f6806bf90 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3f680d2700 tx=0x7f3f6001c8a0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 -- 172.21.15.43:0/3069038932 shutdown_connections 2024-03-30T13:14:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/3069038932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f680cf840 0x7f3f6806bf90 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/3069038932 >> v2:172.21.15.113:3300/0 conn(0x7f3f6815e540 0x7f3f680d3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 -- 172.21.15.43:0/3069038932 >> 172.21.15.43:0/3069038932 conn(0x7f3f680d4360 msgr2=0x7f3f680d4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 -- 172.21.15.43:0/3069038932 shutdown_connections 2024-03-30T13:14:41.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 -- 172.21.15.43:0/3069038932 wait complete. 2024-03-30T13:14:41.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 Processor -- start 2024-03-30T13:14:41.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.889+0000 7f3f6f39b640 1 -- start start 2024-03-30T13:14:41.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f680ba090 0x7f3f680cd720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680cf840 0x7f3f6815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f6815e540 0x7f3f68166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6806ce60 con 0x7f3f6815e540 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6806cce0 con 0x7f3f680ba090 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6806cfe0 con 0x7f3f680cf840 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f6815e540 0x7f3f68166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f6815e540 0x7f3f68166ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44058/0 (socket says 172.21.15.43:44058) 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 -- 172.21.15.43:0/2835807377 learned_addr learned my addr 172.21.15.43:0/2835807377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d137640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f680ba090 0x7f3f680cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6c936640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680cf840 0x7f3f6815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 -- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680cf840 msgr2=0x7f3f6815a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680cf840 0x7f3f6815a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 -- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f680ba090 msgr2=0x7f3f680cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f680ba090 0x7f3f680cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:41.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 -- 172.21.15.43:0/2835807377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f68163e60 con 0x7f3f6815e540 2024-03-30T13:14:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6c936640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680cf840 0x7f3f6815a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d938640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f6815e540 0x7f3f68166ff0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3f600b0040 tx=0x7f3f60019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f5dffb640 1 -- 172.21.15.43:0/2835807377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600bd020 con 0x7f3f6815e540 2024-03-30T13:14:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f5dffb640 1 -- 172.21.15.43:0/2835807377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f600b3040 con 0x7f3f6815e540 2024-03-30T13:14:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f5dffb640 1 -- 172.21.15.43:0/2835807377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f600b2050 con 0x7f3f6815e540 2024-03-30T13:14:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f68164140 con 0x7f3f6815e540 2024-03-30T13:14:41.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6815adf0 con 0x7f3f6815e540 2024-03-30T13:14:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68068900 con 0x7f3f6815e540 2024-03-30T13:14:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f5dffb640 1 -- 172.21.15.43:0/2835807377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3f60003ba0 con 0x7f3f6815e540 2024-03-30T13:14:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f5dffb640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f4c067fc0 0x7f3f4c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f5dffb640 1 -- 172.21.15.43:0/2835807377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f3f60121210 con 0x7f3f6815e540 2024-03-30T13:14:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.893+0000 7f3f6d137640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f4c067fc0 0x7f3f4c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:41.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.897+0000 7f3f6d137640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f4c067fc0 0x7f3f4c06a480 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009a50 tx=0x7f3f64008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:41.897+0000 7f3f5dffb640 1 -- 172.21.15.43:0/2835807377 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3f600c7020 con 0x7f3f6815e540 2024-03-30T13:14:42.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.061+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f6815ec00 con 0x7f3f6815e540 2024-03-30T13:14:42.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.061+0000 7f3f5dffb640 1 -- 172.21.15.43:0/2835807377 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7f3f60016520 con 0x7f3f6815e540 2024-03-30T13:14:42.068 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:14:42.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f4c067fc0 msgr2=0x7f3f4c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f4c067fc0 0x7f3f4c06a480 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3f64009a50 tx=0x7f3f64008040 comp rx=0 tx=0).stop 2024-03-30T13:14:42.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f6815e540 msgr2=0x7f3f68166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f6815e540 0x7f3f68166ff0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3f600b0040 tx=0x7f3f60019640 comp rx=0 tx=0).stop 2024-03-30T13:14:42.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 shutdown_connections 2024-03-30T13:14:42.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3f4c067fc0 0x7f3f4c06a480 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f6815e540 0x7f3f68166ff0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3f680cf840 0x7f3f6815a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 --2- 172.21.15.43:0/2835807377 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3f680ba090 0x7f3f680cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 >> 172.21.15.43:0/2835807377 conn(0x7f3f680d4360 msgr2=0x7f3f6815f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:42.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 shutdown_connections 2024-03-30T13:14:42.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.065+0000 7f3f6f39b640 1 -- 172.21.15.43:0/2835807377 wait complete. 2024-03-30T13:14:42.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:14:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc037e21640 1 Processor -- start 2024-03-30T13:14:42.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc037e21640 1 -- start start 2024-03-30T13:14:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc037e21640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03013f140 0x7fc0301606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc037e21640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc03015ff40 0x7fc03006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc037e21640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc03006d710 con 0x7fc03006cbb0 2024-03-30T13:14:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc037e21640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc03006d000 con 0x7fc03015ff40 2024-03-30T13:14:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc037e21640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc03006e320 con 0x7fc03013f140 2024-03-30T13:14:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc035bbd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03013f140 0x7fc0301606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc0353bc640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc03015ff40 0x7fc03006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc035bbd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03013f140 0x7fc0301606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:41762/0 (socket says 172.21.15.43:41762) 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc035bbd640 1 -- 172.21.15.43:0/4061141813 learned_addr learned my addr 172.21.15.43:0/4061141813 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc035bbd640 1 -- 172.21.15.43:0/4061141813 >> v2:172.21.15.113:3300/0 conn(0x7fc03015ff40 msgr2=0x7fc03006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc035bbd640 1 --2- 172.21.15.43:0/4061141813 >> v2:172.21.15.113:3300/0 conn(0x7fc03015ff40 0x7fc03006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc035bbd640 1 -- 172.21.15.43:0/4061141813 >> v1:172.21.15.113:6789/0 conn(0x7fc03006cbb0 legacy=0x7fc030141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc035bbd640 1 -- 172.21.15.43:0/4061141813 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03006d9f0 con 0x7fc03013f140 2024-03-30T13:14:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.201+0000 7fc0353bc640 1 --2- 172.21.15.43:0/4061141813 >> v2:172.21.15.113:3300/0 conn(0x7fc03015ff40 0x7fc03006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:14:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc035bbd640 1 --2- 172.21.15.43:0/4061141813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03013f140 0x7fc0301606d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009870 tx=0x7fc02c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13fcebd233c93881 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc034bbb640 1 -- 172.21.15.43:0/4061141813 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0a95f0 con 0x7fc03013f140 2024-03-30T13:14:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc034bbb640 1 -- 172.21.15.43:0/4061141813 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02c0a9790 con 0x7fc03013f140 2024-03-30T13:14:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc034bbb640 1 -- 172.21.15.43:0/4061141813 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0a9b60 con 0x7fc03013f140 2024-03-30T13:14:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- 172.21.15.43:0/4061141813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03013f140 msgr2=0x7fc0301606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 --2- 172.21.15.43:0/4061141813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03013f140 0x7fc0301606d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009870 tx=0x7fc02c01c530 comp rx=0 tx=0).stop 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- 172.21.15.43:0/4061141813 shutdown_connections 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 --2- 172.21.15.43:0/4061141813 >> v2:172.21.15.113:3300/0 conn(0x7fc03015ff40 0x7fc03006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 --2- 172.21.15.43:0/4061141813 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03013f140 0x7fc0301606d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- 172.21.15.43:0/4061141813 >> 172.21.15.43:0/4061141813 conn(0x7fc0300d00a0 msgr2=0x7fc0300d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- 172.21.15.43:0/4061141813 shutdown_connections 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- 172.21.15.43:0/4061141813 wait complete. 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 Processor -- start 2024-03-30T13:14:42.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- start start 2024-03-30T13:14:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03006cbb0 0x7fc030139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03013f140 0x7fc0300b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc03015ff40 0x7fc030156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc030160d90 con 0x7fc03013f140 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc030160c10 con 0x7fc03015ff40 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc037e21640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc030168ff0 con 0x7fc03006cbb0 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc0353bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03013f140 0x7fc0300b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc0353bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03013f140 0x7fc0300b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44066/0 (socket says 172.21.15.43:44066) 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc0353bc640 1 -- 172.21.15.43:0/1723031639 learned_addr learned my addr 172.21.15.43:0/1723031639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.205+0000 7fc035bbd640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03006cbb0 0x7fc030139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0363be640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc03015ff40 0x7fc030156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0353bc640 1 -- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03006cbb0 msgr2=0x7fc030139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0353bc640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03006cbb0 0x7fc030139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0353bc640 1 -- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc03015ff40 msgr2=0x7fc030156070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0353bc640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc03015ff40 0x7fc030156070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0353bc640 1 -- 172.21.15.43:0/1723031639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0301fae80 con 0x7fc03013f140 2024-03-30T13:14:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0353bc640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03013f140 0x7fc0300b2b90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc02000e9b0 tx=0x7fc02000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0267fc640 1 -- 172.21.15.43:0/1723031639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020017070 con 0x7fc03013f140 2024-03-30T13:14:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0267fc640 1 -- 172.21.15.43:0/1723031639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02000c9a0 con 0x7fc03013f140 2024-03-30T13:14:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0301fb160 con 0x7fc03013f140 2024-03-30T13:14:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0267fc640 1 -- 172.21.15.43:0/1723031639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02000cd70 con 0x7fc03013f140 2024-03-30T13:14:42.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0301fb720 con 0x7fc03013f140 2024-03-30T13:14:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc030068900 con 0x7fc03013f140 2024-03-30T13:14:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0267fc640 1 -- 172.21.15.43:0/1723031639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc02001a070 con 0x7fc03013f140 2024-03-30T13:14:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0267fc640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc014067fc0 0x7fc01406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc0267fc640 1 -- 172.21.15.43:0/1723031639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fc020086850 con 0x7fc03013f140 2024-03-30T13:14:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc035bbd640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc014067fc0 0x7fc01406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.209+0000 7fc035bbd640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc014067fc0 0x7fc01406a480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0ae000 tx=0x7fc02c0b3040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.213+0000 7fc0267fc640 1 -- 172.21.15.43:0/1723031639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc0200567e0 con 0x7fc03013f140 2024-03-30T13:14:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc0300737b0 con 0x7fc03013f140 2024-03-30T13:14:42.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc0267fc640 1 -- 172.21.15.43:0/1723031639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7fc02005aae0 con 0x7fc03013f140 2024-03-30T13:14:42.375 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:14:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc014067fc0 msgr2=0x7fc01406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc014067fc0 0x7fc01406a480 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc02c0ae000 tx=0x7fc02c0b3040 comp rx=0 tx=0).stop 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03013f140 msgr2=0x7fc0300b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03013f140 0x7fc0300b2b90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fc02000e9b0 tx=0x7fc02000ee70 comp rx=0 tx=0).stop 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 shutdown_connections 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc014067fc0 0x7fc01406a480 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc03015ff40 0x7fc030156070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc03013f140 0x7fc0300b2b90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 --2- 172.21.15.43:0/1723031639 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc03006cbb0 0x7fc030139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 >> 172.21.15.43:0/1723031639 conn(0x7fc0300d00a0 msgr2=0x7fc03015f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 shutdown_connections 2024-03-30T13:14:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.369+0000 7fc037e21640 1 -- 172.21.15.43:0/1723031639 wait complete. 2024-03-30T13:14:42.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:14:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e5daf3640 1 Processor -- start 2024-03-30T13:14:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e5daf3640 1 -- start start 2024-03-30T13:14:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e5daf3640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f9e58159c90 0x7f9e58160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e5daf3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5815c970 con 0x7f9e5815d6d0 2024-03-30T13:14:42.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e5daf3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e580ccbe0 con 0x7f9e58070800 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e5daf3640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5815d240 con 0x7f9e58159c90 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e56ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9e5815d6d0 0x7f9e580700f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33984/0 (socket says 172.21.15.43:33984) 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e56ffd640 1 -- 172.21.15.43:0/620670946 learned_addr learned my addr 172.21.15.43:0/620670946 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 --2- 172.21.15.43:0/620670946 >> v2:172.21.15.113:3300/0 conn(0x7f9e58159c90 0x7f9e58160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2425374130 0 0) 0x7f9e4c001120 con 0x7f9e5815d6d0 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e40003530 con 0x7f9e5815d6d0 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3348921340 0 0) 0x7f9e40003530 con 0x7f9e5815d6d0 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 >> v2:172.21.15.113:3300/0 conn(0x7f9e58159c90 msgr2=0x7f9e58160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 --2- 172.21.15.43:0/620670946 >> v2:172.21.15.113:3300/0 conn(0x7f9e58159c90 0x7f9e58160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 >> v1:172.21.15.113:6789/0 conn(0x7f9e58070800 legacy=0x7f9e580b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5815cbf0 con 0x7f9e5815d6d0 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f9e4c001850 con 0x7f9e5815d6d0 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 >> v1:172.21.15.43:6789/0 conn(0x7f9e5815d6d0 legacy=0x7f9e580700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400039c0 0x7f9e40003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e40004810 0x7f9e40004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e400051d0 0x7f9e4000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5815d240 con 0x7f9e400039c0 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4000c070 con 0x7f9e400051d0 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4000c270 con 0x7f9e40004810 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 --2- 172.21.15.43:0/620670946 >> v2:172.21.15.113:3300/0 conn(0x7f9e58159c90 0x7f9e58160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400039c0 0x7f9e40003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e56ffd640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e40004810 0x7f9e40004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e57fff640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e400051d0 0x7f9e4000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 -- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e40004810 msgr2=0x7f9e40004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e40004810 0x7f9e40004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 -- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e400051d0 msgr2=0x7f9e4000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:42.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e400051d0 0x7f9e4000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e577fe640 1 -- 172.21.15.43:0/620670946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e40003530 con 0x7f9e400039c0 2024-03-30T13:14:42.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.513+0000 7f9e57fff640 1 -- 172.21.15.43:0/620670946 reap_dead start 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e577fe640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400039c0 0x7f9e40003de0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009c90 tx=0x7f9e4801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22ee4d260a954d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48009860 con 0x7f9e400039c0 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e48004020 con 0x7f9e400039c0 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e567fc640 1 -- 172.21.15.43:0/620670946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48004450 con 0x7f9e400039c0 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 -- 172.21.15.43:0/620670946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400039c0 msgr2=0x7f9e40003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400039c0 0x7f9e40003de0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009c90 tx=0x7f9e4801cbf0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 -- 172.21.15.43:0/620670946 shutdown_connections 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/620670946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e400039c0 0x7f9e40003de0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 -- 172.21.15.43:0/620670946 >> 172.21.15.43:0/620670946 conn(0x7f9e580ceae0 msgr2=0x7f9e580ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 -- 172.21.15.43:0/620670946 shutdown_connections 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 -- 172.21.15.43:0/620670946 wait complete. 2024-03-30T13:14:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 Processor -- start 2024-03-30T13:14:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.517+0000 7f9e5daf3640 1 -- start start 2024-03-30T13:14:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e400039c0 0x7f9e580c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e58154ad0 0x7f9e581e8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5815fd60 0x7f9e581601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58160d00 con 0x7f9e58154ad0 2024-03-30T13:14:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58160b80 con 0x7f9e5815fd60 2024-03-30T13:14:42.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9e58160e80 con 0x7f9e400039c0 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e58154ad0 0x7f9e581e8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e58154ad0 0x7f9e581e8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44090/0 (socket says 172.21.15.43:44090) 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 -- 172.21.15.43:0/3951775402 learned_addr learned my addr 172.21.15.43:0/3951775402 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e577fe640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e400039c0 0x7f9e580c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e57fff640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5815fd60 0x7f9e581601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 -- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e400039c0 msgr2=0x7f9e580c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e400039c0 0x7f9e580c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 -- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5815fd60 msgr2=0x7f9e581601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5815fd60 0x7f9e581601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 -- 172.21.15.43:0/3951775402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e581e9cd0 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e56ffd640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e58154ad0 0x7f9e581e8ff0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0019c0 tx=0x7f9e4c007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e3ffff640 1 -- 172.21.15.43:0/3951775402 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00f9c0 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e3ffff640 1 -- 172.21.15.43:0/3951775402 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c018070 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e58156900 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e3ffff640 1 -- 172.21.15.43:0/3951775402 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00fde0 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e58156ec0 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e581570c0 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e3ffff640 1 -- 172.21.15.43:0/3951775402 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9e4c01c410 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e3ffff640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9e44067ff0 0x7f9e4406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e3ffff640 1 -- 172.21.15.43:0/3951775402 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f9e4c086210 con 0x7f9e58154ad0 2024-03-30T13:14:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.521+0000 7f9e577fe640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9e44067ff0 0x7f9e4406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.525+0000 7f9e577fe640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9e44067ff0 0x7f9e4406a4b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9e480ae000 tx=0x7f9e48008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.525+0000 7f9e3ffff640 1 -- 172.21.15.43:0/3951775402 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9e581570c0 con 0x7f9e58154ad0 2024-03-30T13:14:42.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.689+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e58068900 con 0x7f9e58154ad0 2024-03-30T13:14:42.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.689+0000 7f9e3ffff640 1 -- 172.21.15.43:0/3951775402 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7f9e5815c970 con 0x7f9e58154ad0 2024-03-30T13:14:42.695 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:14:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9e44067ff0 msgr2=0x7f9e4406a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9e44067ff0 0x7f9e4406a4b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9e480ae000 tx=0x7f9e48008040 comp rx=0 tx=0).stop 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e58154ad0 msgr2=0x7f9e581e8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e58154ad0 0x7f9e581e8ff0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0019c0 tx=0x7f9e4c007720 comp rx=0 tx=0).stop 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 shutdown_connections 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9e44067ff0 0x7f9e4406a4b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9e5815fd60 0x7f9e581601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e58154ad0 0x7f9e581e8ff0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 --2- 172.21.15.43:0/3951775402 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9e400039c0 0x7f9e580c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 >> 172.21.15.43:0/3951775402 conn(0x7f9e580ceae0 msgr2=0x7f9e5815ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 shutdown_connections 2024-03-30T13:14:42.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.693+0000 7f9e5daf3640 1 -- 172.21.15.43:0/3951775402 wait complete. 2024-03-30T13:14:42.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:14:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde10649640 1 Processor -- start 2024-03-30T13:14:42.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde10649640 1 -- start start 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde10649640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde0815e600 0x7fde080b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde10649640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080d1560 0x7fde080d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde10649640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde080d3e60 con 0x7fde080d1100 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde10649640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fde080b7450 con 0x7fde0815e600 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde10649640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fde080d4140 con 0x7fde080d1560 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0dbe4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fde080d1100 0x7fde080b8160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33998/0 (socket says 172.21.15.43:33998) 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0dbe4640 1 -- 172.21.15.43:0/3766809534 learned_addr learned my addr 172.21.15.43:0/3766809534 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0e3e5640 1 --2- 172.21.15.43:0/3766809534 >> v2:172.21.15.43:3300/0 conn(0x7fde0815e600 0x7fde080b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0ebe6640 1 --2- 172.21.15.43:0/3766809534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080d1560 0x7fde080d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0d3e3640 1 -- 172.21.15.43:0/3766809534 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597810544 0 0) 0x7fde080d3e60 con 0x7fde080d1100 2024-03-30T13:14:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0d3e3640 1 -- 172.21.15.43:0/3766809534 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddf4003530 con 0x7fde080d1100 2024-03-30T13:14:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0e3e5640 1 -- 172.21.15.43:0/3766809534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080d1560 msgr2=0x7fde080d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0e3e5640 1 --2- 172.21.15.43:0/3766809534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080d1560 0x7fde080d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0e3e5640 1 -- 172.21.15.43:0/3766809534 >> v1:172.21.15.43:6789/0 conn(0x7fde080d1100 legacy=0x7fde080b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0d3e3640 1 -- 172.21.15.43:0/3766809534 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2210713790 0 0) 0x7fddf4003530 con 0x7fde080d1100 2024-03-30T13:14:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0e3e5640 1 -- 172.21.15.43:0/3766809534 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde080b7680 con 0x7fde0815e600 2024-03-30T13:14:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.829+0000 7fde0ebe6640 1 --2- 172.21.15.43:0/3766809534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080d1560 0x7fde080d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:14:42.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde0e3e5640 1 --2- 172.21.15.43:0/3766809534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0815e600 0x7fde080b9f00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fde040097c0 tx=0x7fde0401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bea87bc5558458 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde0d3e3640 1 -- 172.21.15.43:0/3766809534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde040b0070 con 0x7fde0815e600 2024-03-30T13:14:42.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde0d3e3640 1 -- 172.21.15.43:0/3766809534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde040ab400 con 0x7fde0815e600 2024-03-30T13:14:42.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde0d3e3640 1 -- 172.21.15.43:0/3766809534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8001680 con 0x7fde0815e600 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- 172.21.15.43:0/3766809534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0815e600 msgr2=0x7fde080b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 --2- 172.21.15.43:0/3766809534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0815e600 0x7fde080b9f00 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fde040097c0 tx=0x7fde0401c390 comp rx=0 tx=0).stop 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- 172.21.15.43:0/3766809534 shutdown_connections 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 --2- 172.21.15.43:0/3766809534 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde080d1560 0x7fde080d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 --2- 172.21.15.43:0/3766809534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde0815e600 0x7fde080b9f00 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- 172.21.15.43:0/3766809534 >> 172.21.15.43:0/3766809534 conn(0x7fde080b6230 msgr2=0x7fde080b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- 172.21.15.43:0/3766809534 shutdown_connections 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- 172.21.15.43:0/3766809534 wait complete. 2024-03-30T13:14:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 Processor -- start 2024-03-30T13:14:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- start start 2024-03-30T13:14:42.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080d1100 0x7fde080c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde080d1560 0x7fde080c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde0815e600 0x7fde080c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fde080b9080 con 0x7fde080d1560 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fde080d1970 con 0x7fde080d1100 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde10649640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fde080b9200 con 0x7fde0815e600 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde0dbe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde080d1560 0x7fde080c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde0dbe4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde080d1560 0x7fde080c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44108/0 (socket says 172.21.15.43:44108) 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.833+0000 7fde0dbe4640 1 -- 172.21.15.43:0/684138901 learned_addr learned my addr 172.21.15.43:0/684138901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0ebe6640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde0815e600 0x7fde080c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0e3e5640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080d1100 0x7fde080c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0dbe4640 1 -- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde0815e600 msgr2=0x7fde080c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0dbe4640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde0815e600 0x7fde080c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0dbe4640 1 -- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080d1100 msgr2=0x7fde080c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0dbe4640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080d1100 0x7fde080c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0dbe4640 1 -- 172.21.15.43:0/684138901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde08152540 con 0x7fde080d1560 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0e3e5640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080d1100 0x7fde080c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0dbe4640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde080d1560 0x7fde080c7080 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fddf80017f0 tx=0x7fddf800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fddfeffd640 1 -- 172.21.15.43:0/684138901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8005ae0 con 0x7fde080d1560 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fddfeffd640 1 -- 172.21.15.43:0/684138901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf8005ca0 con 0x7fde080d1560 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde08152820 con 0x7fde080d1560 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fddfeffd640 1 -- 172.21.15.43:0/684138901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf80072b0 con 0x7fde080d1560 2024-03-30T13:14:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde080c75c0 con 0x7fde080d1560 2024-03-30T13:14:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde08068870 con 0x7fde080d1560 2024-03-30T13:14:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fddfeffd640 1 -- 172.21.15.43:0/684138901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fddf8007450 con 0x7fde080d1560 2024-03-30T13:14:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fddfeffd640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fddec067fc0 0x7fddec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fddfeffd640 1 -- 172.21.15.43:0/684138901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fddf8086470 con 0x7fde080d1560 2024-03-30T13:14:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0e3e5640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fddec067fc0 0x7fddec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.837+0000 7fde0e3e5640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fddec067fc0 0x7fddec06a480 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde040ad000 tx=0x7fde04008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:42.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:42.841+0000 7fddfeffd640 1 -- 172.21.15.43:0/684138901 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fddf8010300 con 0x7fde080d1560 2024-03-30T13:14:43.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.013+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde0815d560 con 0x7fde080d1560 2024-03-30T13:14:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.013+0000 7fddfeffd640 1 -- 172.21.15.43:0/684138901 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7fddf8056ad0 con 0x7fde080d1560 2024-03-30T13:14:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:14:43.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fddec067fc0 msgr2=0x7fddec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fddec067fc0 0x7fddec06a480 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fde040ad000 tx=0x7fde04008040 comp rx=0 tx=0).stop 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde080d1560 msgr2=0x7fde080c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde080d1560 0x7fde080c7080 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fddf80017f0 tx=0x7fddf800cbe0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 shutdown_connections 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fddec067fc0 0x7fddec06a480 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fde0815e600 0x7fde080c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde080d1560 0x7fde080c7080 secure :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fddf80017f0 tx=0x7fddf800cbe0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 --2- 172.21.15.43:0/684138901 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fde080d1100 0x7fde080c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 >> 172.21.15.43:0/684138901 conn(0x7fde080b6230 msgr2=0x7fde0815cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 shutdown_connections 2024-03-30T13:14:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.017+0000 7fde10649640 1 -- 172.21.15.43:0/684138901 wait complete. 2024-03-30T13:14:43.037 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-03-30T13:14:43.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:43.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-30T13:14:43.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[38949]: fuse finished with error 0 and tester_r 0 2024-03-30T13:14:43.067 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:14:43.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:43.067 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-30T13:14:43.240 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:14:43.240 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:14:43.240 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:43.245 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:14:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.397+0000 7f3e3ed90640 1 Processor -- start 2024-03-30T13:14:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.397+0000 7f3e3ed90640 1 -- start start 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3ed90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3815a840 0x7f3e3815e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3ed90640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f3e3815e940 0x7f3e3806da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3ed90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e3806df90 0x7f3e3806e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3ed90640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3e3815eda0 con 0x7f3e3806df90 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3ed90640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f3e38072cb0 con 0x7f3e3815e940 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3ed90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3e38072f90 con 0x7f3e3815a840 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e3806df90 0x7f3e3806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e3806df90 0x7f3e3806e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44124/0 (socket says 172.21.15.43:44124) 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 -- 172.21.15.43:0/989609243 learned_addr learned my addr 172.21.15.43:0/989609243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e37fff640 1 --2- 172.21.15.43:0/989609243 >> v2:172.21.15.113:3300/0 conn(0x7f3e3815e940 0x7f3e3806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3cb2c640 1 --2- 172.21.15.43:0/989609243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3815a840 0x7f3e3815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 -- 172.21.15.43:0/989609243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3815a840 msgr2=0x7f3e3815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 --2- 172.21.15.43:0/989609243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3815a840 0x7f3e3815e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 -- 172.21.15.43:0/989609243 >> v2:172.21.15.113:3300/0 conn(0x7f3e3815e940 msgr2=0x7f3e3806da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 --2- 172.21.15.43:0/989609243 >> v2:172.21.15.113:3300/0 conn(0x7f3e3815e940 0x7f3e3806da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 -- 172.21.15.43:0/989609243 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e38073270 con 0x7f3e3806df90 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e37fff640 1 --2- 172.21.15.43:0/989609243 >> v2:172.21.15.113:3300/0 conn(0x7f3e3815e940 0x7f3e3806da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:43.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3cb2c640 1 --2- 172.21.15.43:0/989609243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3815a840 0x7f3e3815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e3d32d640 1 --2- 172.21.15.43:0/989609243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3806df90 0x7f3e3806e420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3e380d0fa0 tx=0x7f3e3001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe860e33ac088e3f server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e377fe640 1 -- 172.21.15.43:0/989609243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300a56c0 con 0x7f3e3806df90 2024-03-30T13:14:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e377fe640 1 -- 172.21.15.43:0/989609243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e300a5860 con 0x7f3e3806df90 2024-03-30T13:14:43.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.401+0000 7f3e377fe640 1 -- 172.21.15.43:0/989609243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300a5c30 con 0x7f3e3806df90 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- 172.21.15.43:0/989609243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3806df90 msgr2=0x7f3e3806e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/989609243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3806df90 0x7f3e3806e420 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3e380d0fa0 tx=0x7f3e3001c930 comp rx=0 tx=0).stop 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- 172.21.15.43:0/989609243 shutdown_connections 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/989609243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3806df90 0x7f3e3806e420 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/989609243 >> v2:172.21.15.113:3300/0 conn(0x7f3e3815e940 0x7f3e3806da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/989609243 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3815a840 0x7f3e3815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- 172.21.15.43:0/989609243 >> 172.21.15.43:0/989609243 conn(0x7f3e380cfee0 msgr2=0x7f3e380d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:43.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- 172.21.15.43:0/989609243 shutdown_connections 2024-03-30T13:14:43.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- 172.21.15.43:0/989609243 wait complete. 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 Processor -- start 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- start start 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3806df90 0x7f3e38142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3815a840 0x7f3e380ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e3815e940 0x7f3e381592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3815f270 con 0x7f3e3815a840 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3815f0f0 con 0x7f3e3815e940 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3e380728d0 con 0x7f3e3806df90 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3815a840 0x7f3e380ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3815a840 0x7f3e380ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44138/0 (socket says 172.21.15.43:44138) 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 -- 172.21.15.43:0/222801507 learned_addr learned my addr 172.21.15.43:0/222801507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3cb2c640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3806df90 0x7f3e38142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3d32d640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e3815e940 0x7f3e381592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 -- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3806df90 msgr2=0x7f3e38142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3806df90 0x7f3e38142b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 -- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e3815e940 msgr2=0x7f3e381592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e3815e940 0x7f3e381592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 -- 172.21.15.43:0/222801507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e38143070 con 0x7f3e3815a840 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e37fff640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3815a840 0x7f3e380ce540 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e810 tx=0x7f3e2c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e357fa640 1 -- 172.21.15.43:0/222801507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c017070 con 0x7f3e3815a840 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e357fa640 1 -- 172.21.15.43:0/222801507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e2c00c8d0 con 0x7f3e3815a840 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e380ccfa0 con 0x7f3e3815a840 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e357fa640 1 -- 172.21.15.43:0/222801507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00f040 con 0x7f3e3815a840 2024-03-30T13:14:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.405+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e380cd560 con 0x7f3e3815a840 2024-03-30T13:14:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.409+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e38068890 con 0x7f3e3815a840 2024-03-30T13:14:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.409+0000 7f3e357fa640 1 -- 172.21.15.43:0/222801507 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3e2c01a070 con 0x7f3e3815a840 2024-03-30T13:14:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.409+0000 7f3e357fa640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3e24067fc0 0x7f3e2406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.409+0000 7f3e357fa640 1 -- 172.21.15.43:0/222801507 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f3e2c086360 con 0x7f3e3815a840 2024-03-30T13:14:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.409+0000 7f3e3cb2c640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3e24067fc0 0x7f3e2406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.409+0000 7f3e3cb2c640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3e24067fc0 0x7f3e2406a480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800a990 tx=0x7f3e28008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.413+0000 7f3e357fa640 1 -- 172.21.15.43:0/222801507 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3e2c0568b0 con 0x7f3e3815a840 2024-03-30T13:14:43.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.577+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e380719d0 con 0x7f3e3815a840 2024-03-30T13:14:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.577+0000 7f3e357fa640 1 -- 172.21.15.43:0/222801507 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+17021 (secure 0 0 0) 0x7f3e2c0561b0 con 0x7f3e3815a840 2024-03-30T13:14:43.584 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:43.585 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:14:40.895915+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3c","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:14:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3e24067fc0 msgr2=0x7f3e2406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3e24067fc0 0x7f3e2406a480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800a990 tx=0x7f3e28008040 comp rx=0 tx=0).stop 2024-03-30T13:14:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3815a840 msgr2=0x7f3e380ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3815a840 0x7f3e380ce540 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e810 tx=0x7f3e2c00ecd0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 shutdown_connections 2024-03-30T13:14:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3e24067fc0 0x7f3e2406a480 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3e3815e940 0x7f3e381592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e3815a840 0x7f3e380ce540 secure :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c00e810 tx=0x7f3e2c00ecd0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 --2- 172.21.15.43:0/222801507 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3e3806df90 0x7f3e38142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 >> 172.21.15.43:0/222801507 conn(0x7f3e380cfee0 msgr2=0x7f3e380ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 shutdown_connections 2024-03-30T13:14:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.581+0000 7f3e3ed90640 1 -- 172.21.15.43:0/222801507 wait complete. 2024-03-30T13:14:43.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:12:25.924561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:12:26.302922+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 45, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:14:43.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:14:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf945b6640 1 Processor -- start 2024-03-30T13:14:43.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf945b6640 1 -- start start 2024-03-30T13:14:43.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf945b6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fdf8c06f9e0 0x7fdf8c06fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf945b6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c15e9e0 con 0x7fdf8c0b42c0 2024-03-30T13:14:43.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf945b6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c15eca0 con 0x7fdf8c06c790 2024-03-30T13:14:43.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf945b6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fdf8c06bd70 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf92b53640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdf8c0b42c0 0x7fdf8c0b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34000/0 (socket says 172.21.15.43:34000) 2024-03-30T13:14:43.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf92b53640 1 -- 172.21.15.43:0/2999747629 learned_addr learned my addr 172.21.15.43:0/2999747629 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:43.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf91b51640 1 --2- 172.21.15.43:0/2999747629 >> v2:172.21.15.113:3300/0 conn(0x7fdf8c06f9e0 0x7fdf8c06fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf91350640 1 -- 172.21.15.43:0/2999747629 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1995343348 0 0) 0x7fdf8c15e9e0 con 0x7fdf8c0b42c0 2024-03-30T13:14:43.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf91350640 1 -- 172.21.15.43:0/2999747629 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf78003530 con 0x7fdf8c0b42c0 2024-03-30T13:14:43.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf91b51640 1 -- 172.21.15.43:0/2999747629 >> v1:172.21.15.113:6789/0 conn(0x7fdf8c06c790 legacy=0x7fdf8c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf91b51640 1 -- 172.21.15.43:0/2999747629 >> v1:172.21.15.43:6789/0 conn(0x7fdf8c0b42c0 legacy=0x7fdf8c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.721+0000 7fdf91b51640 1 -- 172.21.15.43:0/2999747629 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c15ef80 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf91b51640 1 --2- 172.21.15.43:0/2999747629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c06fe30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097c0 tx=0x7fdf7c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0625b9fd1566d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:43.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf91350640 1 -- 172.21.15.43:0/2999747629 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0b0070 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf91350640 1 -- 172.21.15.43:0/2999747629 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c0ab600 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf91350640 1 -- 172.21.15.43:0/2999747629 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0b55f0 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf945b6640 1 -- 172.21.15.43:0/2999747629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06f9e0 msgr2=0x7fdf8c06fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf945b6640 1 --2- 172.21.15.43:0/2999747629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c06fe30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097c0 tx=0x7fdf7c01c4a0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf945b6640 1 -- 172.21.15.43:0/2999747629 shutdown_connections 2024-03-30T13:14:43.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf945b6640 1 --2- 172.21.15.43:0/2999747629 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c06fe30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf945b6640 1 -- 172.21.15.43:0/2999747629 >> 172.21.15.43:0/2999747629 conn(0x7fdf8c0d1070 msgr2=0x7fdf8c0d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:43.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf945b6640 1 -- 172.21.15.43:0/2999747629 shutdown_connections 2024-03-30T13:14:43.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.725+0000 7fdf945b6640 1 -- 172.21.15.43:0/2999747629 wait complete. 2024-03-30T13:14:43.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 Processor -- start 2024-03-30T13:14:43.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 -- start start 2024-03-30T13:14:43.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06c790 0x7fdf8c15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf8c0b42c0 0x7fdf8c161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c06ccf0 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c06cb70 con 0x7fdf8c06c790 2024-03-30T13:14:43.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8c16c380 con 0x7fdf8c0b42c0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44144/0 (socket says 172.21.15.43:44144) 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 -- 172.21.15.43:0/961033374 learned_addr learned my addr 172.21.15.43:0/961033374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf92b53640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf8c0b42c0 0x7fdf8c161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf92352640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06c790 0x7fdf8c15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 -- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf8c0b42c0 msgr2=0x7fdf8c161b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf8c0b42c0 0x7fdf8c161b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 -- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06c790 msgr2=0x7fdf8c15fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06c790 0x7fdf8c15fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 -- 172.21.15.43:0/961033374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c14adf0 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf91b51640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c160050 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097c0 tx=0x7fdf7c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf82ffd640 1 -- 172.21.15.43:0/961033374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0b0050 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c14ff10 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf82ffd640 1 -- 172.21.15.43:0/961033374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c0b1040 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf82ffd640 1 -- 172.21.15.43:0/961033374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c006e20 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8c148810 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.729+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf8c068900 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.733+0000 7fdf82ffd640 1 -- 172.21.15.43:0/961033374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fdf7c0be060 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.733+0000 7fdf82ffd640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdf70067fc0 0x7fdf7006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.733+0000 7fdf82ffd640 1 -- 172.21.15.43:0/961033374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fdf7c120020 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.733+0000 7fdf92352640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdf70067fc0 0x7fdf7006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:43.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.733+0000 7fdf92352640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdf70067fc0 0x7fdf7006a480 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf880015b0 tx=0x7fdf88009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:43.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.737+0000 7fdf82ffd640 1 -- 172.21.15.43:0/961033374 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fdf7c016090 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.897+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf8c0706c0 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.897+0000 7fdf82ffd640 1 -- 172.21.15.43:0/961033374 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fdf7c0f05f0 con 0x7fdf8c06f9e0 2024-03-30T13:14:43.904 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:14:43.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdf70067fc0 msgr2=0x7fdf7006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdf70067fc0 0x7fdf7006a480 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdf880015b0 tx=0x7fdf88009040 comp rx=0 tx=0).stop 2024-03-30T13:14:43.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf8c06f9e0 msgr2=0x7fdf8c160050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c160050 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0097c0 tx=0x7fdf7c01ace0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 shutdown_connections 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fdf70067fc0 0x7fdf7006a480 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fdf8c0b42c0 0x7fdf8c161b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf8c06f9e0 0x7fdf8c160050 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 --2- 172.21.15.43:0/961033374 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fdf8c06c790 0x7fdf8c15fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 >> 172.21.15.43:0/961033374 conn(0x7fdf8c0d1070 msgr2=0x7fdf8c0d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 shutdown_connections 2024-03-30T13:14:43.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:43.901+0000 7fdf945b6640 1 -- 172.21.15.43:0/961033374 wait complete. 2024-03-30T13:14:43.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:14:44.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06f5788640 1 Processor -- start 2024-03-30T13:14:44.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06f5788640 1 -- start start 2024-03-30T13:14:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06f5788640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f06f0147e00 0x7f06f00b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06f5788640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f00b5470 con 0x7f06f00b4f30 2024-03-30T13:14:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06f5788640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f00b5750 con 0x7f06f006c780 2024-03-30T13:14:44.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06f5788640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f06f00b5a30 con 0x7f06f0147e00 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06e7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f06f00b4f30 0x7f06f006c070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34016/0 (socket says 172.21.15.43:34016) 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06e7fff640 1 -- 172.21.15.43:0/2311591304 learned_addr learned my addr 172.21.15.43:0/2311591304 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06eeffd640 1 --2- 172.21.15.43:0/2311591304 >> v2:172.21.15.113:3300/0 conn(0x7f06f0147e00 0x7f06f00b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06ee7fc640 1 -- 172.21.15.43:0/2311591304 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 990545684 0 0) 0x7f06f00b5470 con 0x7f06f00b4f30 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06ee7fc640 1 -- 172.21.15.43:0/2311591304 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06dc003530 con 0x7f06f00b4f30 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06eeffd640 1 -- 172.21.15.43:0/2311591304 >> v1:172.21.15.113:6789/0 conn(0x7f06f006c780 legacy=0x7f06f00adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06eeffd640 1 -- 172.21.15.43:0/2311591304 >> v1:172.21.15.43:6789/0 conn(0x7f06f00b4f30 legacy=0x7f06f006c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.045+0000 7f06eeffd640 1 -- 172.21.15.43:0/2311591304 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f00b5d10 con 0x7f06f0147e00 2024-03-30T13:14:44.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06eeffd640 1 --2- 172.21.15.43:0/2311591304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f0147e00 0x7f06f00b49f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f06e00097c0 tx=0x7f06e001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17aa190ee225818 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06ee7fc640 1 -- 172.21.15.43:0/2311591304 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00b0070 con 0x7f06f0147e00 2024-03-30T13:14:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06ee7fc640 1 -- 172.21.15.43:0/2311591304 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e00ab600 con 0x7f06f0147e00 2024-03-30T13:14:44.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06ee7fc640 1 -- 172.21.15.43:0/2311591304 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00ab9b0 con 0x7f06f0147e00 2024-03-30T13:14:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 -- 172.21.15.43:0/2311591304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f0147e00 msgr2=0x7f06f00b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 --2- 172.21.15.43:0/2311591304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f0147e00 0x7f06f00b49f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f06e00097c0 tx=0x7f06e001c4a0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 -- 172.21.15.43:0/2311591304 shutdown_connections 2024-03-30T13:14:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 --2- 172.21.15.43:0/2311591304 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f0147e00 0x7f06f00b49f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 -- 172.21.15.43:0/2311591304 >> 172.21.15.43:0/2311591304 conn(0x7f06f00d1070 msgr2=0x7f06f00d1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 -- 172.21.15.43:0/2311591304 shutdown_connections 2024-03-30T13:14:44.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 -- 172.21.15.43:0/2311591304 wait complete. 2024-03-30T13:14:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 Processor -- start 2024-03-30T13:14:44.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.049+0000 7f06f5788640 1 -- start start 2024-03-30T13:14:44.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f006c780 0x7f06f0146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00b4f30 0x7f06f00d0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06f0147e00 0x7f06f01699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00aeef0 con 0x7f06f00b4f30 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f06f00aed70 con 0x7f06f006c780 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f06f016c130 con 0x7f06f0147e00 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00b4f30 0x7f06f00d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00b4f30 0x7f06f00d0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44148/0 (socket says 172.21.15.43:44148) 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 -- 172.21.15.43:0/1140198617 learned_addr learned my addr 172.21.15.43:0/1140198617 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06eeffd640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f006c780 0x7f06f0146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06ef7fe640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06f0147e00 0x7f06f01699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 -- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06f0147e00 msgr2=0x7f06f01699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06f0147e00 0x7f06f01699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 -- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f006c780 msgr2=0x7f06f0146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f006c780 0x7f06f0146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 -- 172.21.15.43:0/1140198617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f015eea0 con 0x7f06f00b4f30 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e7fff640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00b4f30 0x7f06f00d0710 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f06d800c500 tx=0x7f06d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e77fe640 1 -- 172.21.15.43:0/1140198617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8017070 con 0x7f06f00b4f30 2024-03-30T13:14:44.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e77fe640 1 -- 172.21.15.43:0/1140198617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d8005cf0 con 0x7f06f00b4f30 2024-03-30T13:14:44.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e77fe640 1 -- 172.21.15.43:0/1140198617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d80072f0 con 0x7f06f00b4f30 2024-03-30T13:14:44.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f015f140 con 0x7f06f00b4f30 2024-03-30T13:14:44.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f015d850 con 0x7f06f00b4f30 2024-03-30T13:14:44.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f0068900 con 0x7f06f00b4f30 2024-03-30T13:14:44.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.053+0000 7f06e77fe640 1 -- 172.21.15.43:0/1140198617 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f06d801a070 con 0x7f06f00b4f30 2024-03-30T13:14:44.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.057+0000 7f06e77fe640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06d0067fc0 0x7f06d006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.057+0000 7f06e77fe640 1 -- 172.21.15.43:0/1140198617 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f06d80859e0 con 0x7f06f00b4f30 2024-03-30T13:14:44.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.057+0000 7f06eeffd640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06d0067fc0 0x7f06d006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.057+0000 7f06eeffd640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06d0067fc0 0x7f06d006a480 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ad000 tx=0x7f06e0008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.061+0000 7f06e77fe640 1 -- 172.21.15.43:0/1140198617 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f06d80559f0 con 0x7f06f00b4f30 2024-03-30T13:14:44.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f06f0070a90 con 0x7f06f00b4f30 2024-03-30T13:14:44.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06e77fe640 1 -- 172.21.15.43:0/1140198617 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7f06d8059cf0 con 0x7f06f00b4f30 2024-03-30T13:14:44.230 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06d0067fc0 msgr2=0x7f06d006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06d0067fc0 0x7f06d006a480 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f06e00ad000 tx=0x7f06e0008040 comp rx=0 tx=0).stop 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00b4f30 msgr2=0x7f06f00d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00b4f30 0x7f06f00d0710 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f06d800c500 tx=0x7f06d800c9c0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 shutdown_connections 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f06d0067fc0 0x7f06d006a480 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f06f0147e00 0x7f06f01699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06f00b4f30 0x7f06f00d0710 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 --2- 172.21.15.43:0/1140198617 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f06f006c780 0x7f06f0146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 >> 172.21.15.43:0/1140198617 conn(0x7f06f00d1070 msgr2=0x7f06f00b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:44.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 shutdown_connections 2024-03-30T13:14:44.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.225+0000 7f06f5788640 1 -- 172.21.15.43:0/1140198617 wait complete. 2024-03-30T13:14:44.249 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:14:44.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.365+0000 7f8d3e069640 1 Processor -- start 2024-03-30T13:14:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.365+0000 7f8d3e069640 1 -- start start 2024-03-30T13:14:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d3e069640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f8d38144930 0x7f8d380b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d3e069640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d380b33c0 con 0x7f8d3806cd80 2024-03-30T13:14:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d3e069640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d380b3660 con 0x7f8d380b4870 2024-03-30T13:14:44.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d3e069640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f8d380cdb90 con 0x7f8d38144930 2024-03-30T13:14:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d37fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8d3806cd80 0x7f8d380ce020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34026/0 (socket says 172.21.15.43:34026) 2024-03-30T13:14:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d37fff640 1 -- 172.21.15.43:0/879573657 learned_addr learned my addr 172.21.15.43:0/879573657 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d377fe640 1 --2- 172.21.15.43:0/879573657 >> v2:172.21.15.113:3300/0 conn(0x7f8d38144930 0x7f8d380b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153826897 0 0) 0x7f8d380b33c0 con 0x7f8d3806cd80 2024-03-30T13:14:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d20003530 con 0x7f8d3806cd80 2024-03-30T13:14:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 872246987 0 0) 0x7f8d20003530 con 0x7f8d3806cd80 2024-03-30T13:14:44.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 >> v2:172.21.15.113:3300/0 conn(0x7f8d38144930 msgr2=0x7f8d380b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 --2- 172.21.15.43:0/879573657 >> v2:172.21.15.113:3300/0 conn(0x7f8d38144930 0x7f8d380b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 >> v1:172.21.15.113:6789/0 conn(0x7f8d380b4870 legacy=0x7f8d3806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d380b3940 con 0x7f8d3806cd80 2024-03-30T13:14:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f8d30001680 con 0x7f8d3806cd80 2024-03-30T13:14:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 >> v1:172.21.15.43:6789/0 conn(0x7f8d3806cd80 legacy=0x7f8d380ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d200039c0 0x7f8d20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d200051d0 0x7f8d2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d380cdb90 con 0x7f8d20004810 2024-03-30T13:14:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2000c170 con 0x7f8d200039c0 2024-03-30T13:14:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8d2000c370 con 0x7f8d200051d0 2024-03-30T13:14:44.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d377fe640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d200039c0 0x7f8d20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d37fff640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d200051d0 0x7f8d2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d36ffd640 1 -- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d200051d0 msgr2=0x7f8d2000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d200051d0 0x7f8d2000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d36ffd640 1 -- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d200039c0 msgr2=0x7f8d20003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d200039c0 0x7f8d20003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d36ffd640 1 -- 172.21.15.43:0/879573657 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d20003530 con 0x7f8d20004810 2024-03-30T13:14:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.369+0000 7f8d37fff640 1 -- 172.21.15.43:0/879573657 reap_dead start 2024-03-30T13:14:44.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d20004c90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c001780 tx=0x7f8d2c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d28b918847dcc9d server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c0013a0 con 0x7f8d20004810 2024-03-30T13:14:44.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2c0052f0 con 0x7f8d20004810 2024-03-30T13:14:44.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d367fc640 1 -- 172.21.15.43:0/879573657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c003840 con 0x7f8d20004810 2024-03-30T13:14:44.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- 172.21.15.43:0/879573657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 msgr2=0x7f8d20004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d20004c90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c001780 tx=0x7f8d2c01c3d0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- 172.21.15.43:0/879573657 shutdown_connections 2024-03-30T13:14:44.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 --2- 172.21.15.43:0/879573657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d20004c90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- 172.21.15.43:0/879573657 >> 172.21.15.43:0/879573657 conn(0x7f8d380d00a0 msgr2=0x7f8d380d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:44.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- 172.21.15.43:0/879573657 shutdown_connections 2024-03-30T13:14:44.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- 172.21.15.43:0/879573657 wait complete. 2024-03-30T13:14:44.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 Processor -- start 2024-03-30T13:14:44.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- start start 2024-03-30T13:14:44.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d381f8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d381e91b0 0x7f8d381e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d381e9630 0x7f8d381ea0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38071d80 con 0x7f8d20004810 2024-03-30T13:14:44.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8d38071c00 con 0x7f8d381e9630 2024-03-30T13:14:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d3e069640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8d38071f00 con 0x7f8d381e91b0 2024-03-30T13:14:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d381f8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d381f8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44174/0 (socket says 172.21.15.43:44174) 2024-03-30T13:14:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 -- 172.21.15.43:0/4205023628 learned_addr learned my addr 172.21.15.43:0/4205023628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d37fff640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d381e9630 0x7f8d381ea0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d381e91b0 0x7f8d381e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 -- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d381e91b0 msgr2=0x7f8d381e9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d381e91b0 0x7f8d381e9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 -- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d381e9630 msgr2=0x7f8d381ea0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d381e9630 0x7f8d381ea0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d377fe640 1 -- 172.21.15.43:0/4205023628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d38143b40 con 0x7f8d20004810 2024-03-30T13:14:44.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.373+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d381e91b0 0x7f8d381e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:44.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d377fe640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d381f8780 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800f430 tx=0x7f8d2800f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d1ffff640 1 -- 172.21.15.43:0/4205023628 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d280147e0 con 0x7f8d20004810 2024-03-30T13:14:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d1ffff640 1 -- 172.21.15.43:0/4205023628 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2801b070 con 0x7f8d20004810 2024-03-30T13:14:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3815d1e0 con 0x7f8d20004810 2024-03-30T13:14:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d1ffff640 1 -- 172.21.15.43:0/4205023628 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d28014be0 con 0x7f8d20004810 2024-03-30T13:14:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3815d7a0 con 0x7f8d20004810 2024-03-30T13:14:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d38068900 con 0x7f8d20004810 2024-03-30T13:14:44.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d1ffff640 1 -- 172.21.15.43:0/4205023628 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8d28016420 con 0x7f8d20004810 2024-03-30T13:14:44.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d1ffff640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8d24067fc0 0x7f8d2406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d1ffff640 1 -- 172.21.15.43:0/4205023628 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f8d280876a0 con 0x7f8d20004810 2024-03-30T13:14:44.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8d24067fc0 0x7f8d2406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.377+0000 7f8d36ffd640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8d24067fc0 0x7f8d2406a480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c001750 tx=0x7f8d2c0011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.381+0000 7f8d1ffff640 1 -- 172.21.15.43:0/4205023628 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8d28057d30 con 0x7f8d20004810 2024-03-30T13:14:44.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8d380b4fa0 con 0x7f8d20004810 2024-03-30T13:14:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d1ffff640 1 -- 172.21.15.43:0/4205023628 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7f8d28057630 con 0x7f8d20004810 2024-03-30T13:14:44.551 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:14:44.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8d24067fc0 msgr2=0x7f8d2406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8d24067fc0 0x7f8d2406a480 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c001750 tx=0x7f8d2c0011a0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 msgr2=0x7f8d381f8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d381f8780 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800f430 tx=0x7f8d2800f8f0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 shutdown_connections 2024-03-30T13:14:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8d24067fc0 0x7f8d2406a480 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8d381e9630 0x7f8d381ea0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8d381e91b0 0x7f8d381e9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 --2- 172.21.15.43:0/4205023628 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d20004810 0x7f8d381f8780 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 >> 172.21.15.43:0/4205023628 conn(0x7f8d380d00a0 msgr2=0x7f8d380b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:44.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.545+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 shutdown_connections 2024-03-30T13:14:44.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.549+0000 7f8d3e069640 1 -- 172.21.15.43:0/4205023628 wait complete. 2024-03-30T13:14:44.571 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-30T13:14:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c0858c640 1 Processor -- start 2024-03-30T13:14:44.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c0858c640 1 -- start start 2024-03-30T13:14:44.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c0858c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c0015e600 0x7f8c000b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c0858c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c000d3e60 con 0x7f8c000d1100 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c0858c640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c000b7680 con 0x7f8c000d1560 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c0858c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8c000b7450 con 0x7f8c0015e600 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c05b27640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c000d1100 0x7f8c000b8160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34030/0 (socket says 172.21.15.43:34030) 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c05b27640 1 -- 172.21.15.43:0/3826073214 learned_addr learned my addr 172.21.15.43:0/3826073214 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c06328640 1 --2- 172.21.15.43:0/3826073214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c0015e600 0x7f8c000b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c05326640 1 -- 172.21.15.43:0/3826073214 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1720592377 0 0) 0x7f8c000d3e60 con 0x7f8c000d1100 2024-03-30T13:14:44.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.681+0000 7f8c05326640 1 -- 172.21.15.43:0/3826073214 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8be8003530 con 0x7f8c000d1100 2024-03-30T13:14:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c06328640 1 -- 172.21.15.43:0/3826073214 >> v1:172.21.15.113:6789/0 conn(0x7f8c000d1560 legacy=0x7f8c000d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c05326640 1 -- 172.21.15.43:0/3826073214 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 634369906 0 0) 0x7f8be8003530 con 0x7f8c000d1100 2024-03-30T13:14:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c06328640 1 -- 172.21.15.43:0/3826073214 >> v1:172.21.15.43:6789/0 conn(0x7f8c000d1100 legacy=0x7f8c000b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c06328640 1 -- 172.21.15.43:0/3826073214 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c000b7960 con 0x7f8c0015e600 2024-03-30T13:14:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c06328640 1 --2- 172.21.15.43:0/3826073214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c0015e600 0x7f8c000b9f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00097f0 tx=0x7f8bf001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75e63bc9652725ed server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c05326640 1 -- 172.21.15.43:0/3826073214 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf00b0070 con 0x7f8c0015e600 2024-03-30T13:14:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c05326640 1 -- 172.21.15.43:0/3826073214 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf00ab400 con 0x7f8c0015e600 2024-03-30T13:14:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c05326640 1 -- 172.21.15.43:0/3826073214 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4001680 con 0x7f8c0015e600 2024-03-30T13:14:44.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c0858c640 1 -- 172.21.15.43:0/3826073214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c0015e600 msgr2=0x7f8c000b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c0858c640 1 --2- 172.21.15.43:0/3826073214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c0015e600 0x7f8c000b9f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00097f0 tx=0x7f8bf001c400 comp rx=0 tx=0).stop 2024-03-30T13:14:44.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c0858c640 1 -- 172.21.15.43:0/3826073214 shutdown_connections 2024-03-30T13:14:44.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c0858c640 1 --2- 172.21.15.43:0/3826073214 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c0015e600 0x7f8c000b9f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c0858c640 1 -- 172.21.15.43:0/3826073214 >> 172.21.15.43:0/3826073214 conn(0x7f8c000b6230 msgr2=0x7f8c000b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:44.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c0858c640 1 -- 172.21.15.43:0/3826073214 shutdown_connections 2024-03-30T13:14:44.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.685+0000 7f8c0858c640 1 -- 172.21.15.43:0/3826073214 wait complete. 2024-03-30T13:14:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 Processor -- start 2024-03-30T13:14:44.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 -- start start 2024-03-30T13:14:44.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c000d1100 0x7f8c0014d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c000d1560 0x7f8c00150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0015e600 0x7f8c00151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8c000b9410 con 0x7f8c000d1560 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8c000b9290 con 0x7f8c0015e600 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8c0016bfe0 con 0x7f8c000d1100 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c000d1560 0x7f8c00150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c000d1560 0x7f8c00150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44182/0 (socket says 172.21.15.43:44182) 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 -- 172.21.15.43:0/2731978204 learned_addr learned my addr 172.21.15.43:0/2731978204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c06328640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c000d1100 0x7f8c0014d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c06b29640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0015e600 0x7f8c00151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 -- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c000d1100 msgr2=0x7f8c0014d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c000d1100 0x7f8c0014d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 -- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0015e600 msgr2=0x7f8c00151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0015e600 0x7f8c00151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 -- 172.21.15.43:0/2731978204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0014e350 con 0x7f8c000d1560 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c06328640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c000d1100 0x7f8c0014d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c06b29640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0015e600 0x7f8c00151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c05b27640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c000d1560 0x7f8c00150d40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c550 tx=0x7f8bf400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8bfeffd640 1 -- 172.21.15.43:0/2731978204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4017070 con 0x7f8c000d1560 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8bfeffd640 1 -- 172.21.15.43:0/2731978204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf4005cf0 con 0x7f8c000d1560 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c00154ff0 con 0x7f8c000d1560 2024-03-30T13:14:44.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8bfeffd640 1 -- 172.21.15.43:0/2731978204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40072b0 con 0x7f8c000d1560 2024-03-30T13:14:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c001555b0 con 0x7f8c000d1560 2024-03-30T13:14:44.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c00068890 con 0x7f8c000d1560 2024-03-30T13:14:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.689+0000 7f8bfeffd640 1 -- 172.21.15.43:0/2731978204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8bf401a070 con 0x7f8c000d1560 2024-03-30T13:14:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.693+0000 7f8bfeffd640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8bec067fc0 0x7f8bec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.693+0000 7f8bfeffd640 1 -- 172.21.15.43:0/2731978204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f8bf4085af0 con 0x7f8c000d1560 2024-03-30T13:14:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.693+0000 7f8c06328640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8bec067fc0 0x7f8bec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:44.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.693+0000 7f8c06328640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8bec067fc0 0x7f8bec06a480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00ad000 tx=0x7f8bf00a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:44.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.697+0000 7f8bfeffd640 1 -- 172.21.15.43:0/2731978204 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8bf4056150 con 0x7f8c000d1560 2024-03-30T13:14:44.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.869+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8c00074ad0 con 0x7f8c000d1560 2024-03-30T13:14:44.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.869+0000 7f8bfeffd640 1 -- 172.21.15.43:0/2731978204 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f8bf4055a50 con 0x7f8c000d1560 2024-03-30T13:14:44.875 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-03-30T13:14:44.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8bec067fc0 msgr2=0x7f8bec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8bec067fc0 0x7f8bec06a480 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00ad000 tx=0x7f8bf00a6000 comp rx=0 tx=0).stop 2024-03-30T13:14:44.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c000d1560 msgr2=0x7f8c00150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:44.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c000d1560 0x7f8c00150d40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c550 tx=0x7f8bf400ca10 comp rx=0 tx=0).stop 2024-03-30T13:14:44.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 shutdown_connections 2024-03-30T13:14:44.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8bec067fc0 0x7f8bec06a480 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8c0015e600 0x7f8c00151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c000d1560 0x7f8c00150d40 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 --2- 172.21.15.43:0/2731978204 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8c000d1100 0x7f8c0014d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 >> 172.21.15.43:0/2731978204 conn(0x7f8c000b6230 msgr2=0x7f8c000d2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 shutdown_connections 2024-03-30T13:14:44.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:44.873+0000 7f8c0858c640 1 -- 172.21.15.43:0/2731978204 wait complete. 2024-03-30T13:14:44.893 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi113.front.sepia.ceph.com... 2024-03-30T13:14:44.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:44.894 DEBUG:teuthology.orchestra.run.smithi113:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-30T13:14:44.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi113.stderr:ceph-fuse[33309]: fuse finished with error 0 and tester_r 0 2024-03-30T13:14:44.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:14:44.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:44.908 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-30T13:14:45.063 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:14:45.064 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:14:45.064 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:45.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22574 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T13:14:45.071 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:14:45.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:45.071 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-30T13:14:45.117 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-30T13:14:45.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:14:45.118 INFO:tasks.cephfs.mount:Terminating background process 2024-03-30T13:14:45.118 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:14:45.119 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:45.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18812 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T13:14:45.127 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:14:45.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:45.127 DEBUG:teuthology.orchestra.run.smithi113:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-30T13:14:45.173 INFO:teuthology.orchestra.run.smithi113.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-30T13:14:45.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:14:45.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:14:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6435418640 1 Processor -- start 2024-03-30T13:14:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6435418640 1 -- start start 2024-03-30T13:14:45.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6435418640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6430162b90 0x7f643006e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6435418640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6430072070 con 0x7f643006ec50 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6435418640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f643006dc30 con 0x7f6430147e10 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6435418640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6430071e50 con 0x7f6430162b90 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f643006ec50 0x7f643006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34040/0 (socket says 172.21.15.43:34040) 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642f7fe640 1 -- 172.21.15.43:0/1698013648 learned_addr learned my addr 172.21.15.43:0/1698013648 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6427fff640 1 --2- 172.21.15.43:0/1698013648 >> v2:172.21.15.113:3300/0 conn(0x7f6430162b90 0x7f643006e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2571706345 0 0) 0x7f6430072070 con 0x7f643006ec50 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f641c003530 con 0x7f643006ec50 2024-03-30T13:14:45.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1047903534 0 0) 0x7f641c003530 con 0x7f643006ec50 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 >> v2:172.21.15.113:3300/0 conn(0x7f6430162b90 msgr2=0x7f643006e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 --2- 172.21.15.43:0/1698013648 >> v2:172.21.15.113:3300/0 conn(0x7f6430162b90 0x7f643006e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 >> v1:172.21.15.113:6789/0 conn(0x7f6430147e10 legacy=0x7f6430071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643006df10 con 0x7f643006ec50 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f6428001a00 con 0x7f643006ec50 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 >> v1:172.21.15.43:6789/0 conn(0x7f643006ec50 legacy=0x7f643006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f641c0039c0 0x7f641c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f641c004810 0x7f641c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0051d0 0x7f641c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6430071e50 con 0x7f641c0039c0 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f641c00c170 con 0x7f641c0051d0 2024-03-30T13:14:45.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f641c00c370 con 0x7f641c004810 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642effd640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f641c0039c0 0x7f641c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f6427fff640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f641c004810 0x7f641c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642f7fe640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0051d0 0x7f641c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642effd640 1 -- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f641c004810 msgr2=0x7f641c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642effd640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f641c004810 0x7f641c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642effd640 1 -- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0051d0 msgr2=0x7f641c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642effd640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0051d0 0x7f641c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642f7fe640 1 -- 172.21.15.43:0/1698013648 reap_dead start 2024-03-30T13:14:45.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.305+0000 7f642effd640 1 -- 172.21.15.43:0/1698013648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c003530 con 0x7f641c0039c0 2024-03-30T13:14:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f642effd640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f641c0039c0 0x7f641c003de0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6418001780 tx=0x7f641801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af839c3514fd078b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64180013a0 con 0x7f641c0039c0 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64180052f0 con 0x7f641c0039c0 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f642e7fc640 1 -- 172.21.15.43:0/1698013648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418003840 con 0x7f641c0039c0 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 -- 172.21.15.43:0/1698013648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f641c0039c0 msgr2=0x7f641c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f641c0039c0 0x7f641c003de0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6418001780 tx=0x7f641801c400 comp rx=0 tx=0).stop 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 -- 172.21.15.43:0/1698013648 shutdown_connections 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 --2- 172.21.15.43:0/1698013648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f641c0039c0 0x7f641c003de0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 -- 172.21.15.43:0/1698013648 >> 172.21.15.43:0/1698013648 conn(0x7f64300cfee0 msgr2=0x7f64300d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 -- 172.21.15.43:0/1698013648 shutdown_connections 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 -- 172.21.15.43:0/1698013648 wait complete. 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 Processor -- start 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 -- start start 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0039c0 0x7f6430154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.309+0000 7f6435418640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f643015df80 0x7f6430144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6435418640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64300ad8e0 0x7f64300add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6435418640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f64300b2ef0 con 0x7f64300ad8e0 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6435418640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f643006e280 con 0x7f641c0039c0 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6435418640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f64300b3070 con 0x7f643015df80 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64300ad8e0 0x7f64300add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64300ad8e0 0x7f64300add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44206/0 (socket says 172.21.15.43:44206) 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 -- 172.21.15.43:0/69901709 learned_addr learned my addr 172.21.15.43:0/69901709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642effd640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0039c0 0x7f6430154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6427fff640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f643015df80 0x7f6430144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 -- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f643015df80 msgr2=0x7f6430144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f643015df80 0x7f6430144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 -- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0039c0 msgr2=0x7f6430154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0039c0 0x7f6430154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 -- 172.21.15.43:0/69901709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6430144ba0 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642effd640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0039c0 0x7f6430154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642f7fe640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64300ad8e0 0x7f64300add60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f643006f030 tx=0x7f6428007af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f64277fe640 1 -- 172.21.15.43:0/69901709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642800e9e0 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f64277fe640 1 -- 172.21.15.43:0/69901709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642800eba0 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6430144e80 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f64277fe640 1 -- 172.21.15.43:0/69901709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6428014430 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6430145440 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64300ae2a0 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f64277fe640 1 -- 172.21.15.43:0/69901709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f64280145d0 con 0x7f64300ad8e0 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f64277fe640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6410067ff0 0x7f641006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f64277fe640 1 -- 172.21.15.43:0/69901709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f64280864a0 con 0x7f64300ad8e0 2024-03-30T13:14:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642effd640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6410067ff0 0x7f641006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.313+0000 7f642effd640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6410067ff0 0x7f641006a4b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6418001750 tx=0x7f64180011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.317+0000 7f64277fe640 1 -- 172.21.15.43:0/69901709 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f64280569d0 con 0x7f64300ad8e0 2024-03-30T13:14:45.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.509+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6430068890 con 0x7f64300ad8e0 2024-03-30T13:14:45.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f64277fe640 1 -- 172.21.15.43:0/69901709 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5187 (secure 0 0 0) 0x7f64280562d0 con 0x7f64300ad8e0 2024-03-30T13:14:45.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:45.517 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4271726137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4271726137},{"type":"v1","addr":"172.21.15.43:6834","nonce":4271726137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1748034672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1748034672},{"type":"v1","addr":"172.21.15.43:6835","nonce":1748034672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3254700273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3254700273},{"type":"v1","addr":"172.21.15.113:6835","nonce":3254700273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:12:27.356973+0000","modified":"2024-03-30T13:14:35.770003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4880":{"gid":4880,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1104336007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1104336007},{"type":"v1","addr":"172.21.15.113:6837","nonce":1104336007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4880,"qdb_cluster":[4880]},"id":2}]} 2024-03-30T13:14:45.517 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-03-30T13:14:45.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6410067ff0 msgr2=0x7f641006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6410067ff0 0x7f641006a4b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6418001750 tx=0x7f64180011a0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64300ad8e0 msgr2=0x7f64300add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64300ad8e0 0x7f64300add60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f643006f030 tx=0x7f6428007af0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 shutdown_connections 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6410067ff0 0x7f641006a4b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f64300ad8e0 0x7f64300add60 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f643015df80 0x7f6430144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 --2- 172.21.15.43:0/69901709 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f641c0039c0 0x7f6430154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 >> 172.21.15.43:0/69901709 conn(0x7f64300cfee0 msgr2=0x7f64301607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 shutdown_connections 2024-03-30T13:14:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.513+0000 7f6435418640 1 -- 172.21.15.43:0/69901709 wait complete. 2024-03-30T13:14:45.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:14:45.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.641+0000 7fef9ce5c640 1 Processor -- start 2024-03-30T13:14:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.641+0000 7fef9ce5c640 1 -- start start 2024-03-30T13:14:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9ce5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef9815e600 0x7fef980b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9ce5c640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fef980d1100 0x7fef980d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9ce5c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef980b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9ce5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fef980b7450 con 0x7fef9815e600 2024-03-30T13:14:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9ce5c640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fef980d1aa0 con 0x7fef980d1100 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9ce5c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fef980d4140 con 0x7fef980b8160 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef9815e600 0x7fef980b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef9815e600 0x7fef980b9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44210/0 (socket says 172.21.15.43:44210) 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 -- 172.21.15.43:0/298254923 learned_addr learned my addr 172.21.15.43:0/298254923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef95d9b640 1 --2- 172.21.15.43:0/298254923 >> v2:172.21.15.113:3300/0 conn(0x7fef980d1100 0x7fef980d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef96d9d640 1 --2- 172.21.15.43:0/298254923 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef980b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 -- 172.21.15.43:0/298254923 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 msgr2=0x7fef980b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 --2- 172.21.15.43:0/298254923 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef980b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 -- 172.21.15.43:0/298254923 >> v2:172.21.15.113:3300/0 conn(0x7fef980d1100 msgr2=0x7fef980d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 --2- 172.21.15.43:0/298254923 >> v2:172.21.15.113:3300/0 conn(0x7fef980d1100 0x7fef980d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 -- 172.21.15.43:0/298254923 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef980d3e60 con 0x7fef9815e600 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef96d9d640 1 --2- 172.21.15.43:0/298254923 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef980b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef95d9b640 1 --2- 172.21.15.43:0/298254923 >> v2:172.21.15.113:3300/0 conn(0x7fef980d1100 0x7fef980d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:45.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9659c640 1 --2- 172.21.15.43:0/298254923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef980b9f00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fef980d1df0 tx=0x7fef8801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18a7942ee17bdcb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9559a640 1 -- 172.21.15.43:0/298254923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef880aa610 con 0x7fef9815e600 2024-03-30T13:14:45.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9559a640 1 -- 172.21.15.43:0/298254923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef880af070 con 0x7fef9815e600 2024-03-30T13:14:45.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.645+0000 7fef9559a640 1 -- 172.21.15.43:0/298254923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef880aa9d0 con 0x7fef9815e600 2024-03-30T13:14:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- 172.21.15.43:0/298254923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 msgr2=0x7fef980b9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/298254923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef980b9f00 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fef980d1df0 tx=0x7fef8801c4f0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- 172.21.15.43:0/298254923 shutdown_connections 2024-03-30T13:14:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/298254923 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef980b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/298254923 >> v2:172.21.15.113:3300/0 conn(0x7fef980d1100 0x7fef980d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/298254923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef980b9f00 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- 172.21.15.43:0/298254923 >> 172.21.15.43:0/298254923 conn(0x7fef980b6230 msgr2=0x7fef980b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:45.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- 172.21.15.43:0/298254923 shutdown_connections 2024-03-30T13:14:45.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- 172.21.15.43:0/298254923 wait complete. 2024-03-30T13:14:45.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 Processor -- start 2024-03-30T13:14:45.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- start start 2024-03-30T13:14:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef9814c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef980d1100 0x7fef98151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef9814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fef980d2780 con 0x7fef9815e600 2024-03-30T13:14:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fef980d2600 con 0x7fef980d1100 2024-03-30T13:14:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9ce5c640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fef980d2900 con 0x7fef980b8160 2024-03-30T13:14:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef9814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9659c640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef9814c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef9814f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44224/0 (socket says 172.21.15.43:44224) 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 -- 172.21.15.43:0/3508164947 learned_addr learned my addr 172.21.15.43:0/3508164947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef95d9b640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef980d1100 0x7fef98151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 -- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 msgr2=0x7fef9814c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef9814c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 -- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef980d1100 msgr2=0x7fef98151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef980d1100 0x7fef98151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef96d9d640 1 -- 172.21.15.43:0/3508164947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef981610f0 con 0x7fef9815e600 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.649+0000 7fef9659c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef9814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef96d9d640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef9814f440 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fef9000cef0 tx=0x7fef9000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef86ffd640 1 -- 172.21.15.43:0/3508164947 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9001a070 con 0x7fef9815e600 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef86ffd640 1 -- 172.21.15.43:0/3508164947 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9000d840 con 0x7fef9815e600 2024-03-30T13:14:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9815e9e0 con 0x7fef9815e600 2024-03-30T13:14:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef86ffd640 1 -- 172.21.15.43:0/3508164947 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9000dc30 con 0x7fef9815e600 2024-03-30T13:14:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef98161390 con 0x7fef9815e600 2024-03-30T13:14:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef98068870 con 0x7fef9815e600 2024-03-30T13:14:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef86ffd640 1 -- 172.21.15.43:0/3508164947 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fef9001d070 con 0x7fef9815e600 2024-03-30T13:14:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef86ffd640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef78067ff0 0x7fef7806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef86ffd640 1 -- 172.21.15.43:0/3508164947 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fef90086e30 con 0x7fef9815e600 2024-03-30T13:14:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef9659c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef78067ff0 0x7fef7806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.653+0000 7fef9659c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef78067ff0 0x7fef7806a4b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fef880ae000 tx=0x7fef880a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.657+0000 7fef86ffd640 1 -- 172.21.15.43:0/3508164947 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fef90056da0 con 0x7fef9815e600 2024-03-30T13:14:45.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.817+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef98152a60 con 0x7fef9815e600 2024-03-30T13:14:45.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.821+0000 7fef86ffd640 1 -- 172.21.15.43:0/3508164947 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+17021 (secure 0 0 0) 0x7fef9005b0a0 con 0x7fef9815e600 2024-03-30T13:14:45.827 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:45.827 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:14:40.895915+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3c","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:14:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef78067ff0 msgr2=0x7fef7806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef78067ff0 0x7fef7806a4b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fef880ae000 tx=0x7fef880a7000 comp rx=0 tx=0).stop 2024-03-30T13:14:45.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 msgr2=0x7fef9814f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef9814f440 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fef9000cef0 tx=0x7fef9000a710 comp rx=0 tx=0).stop 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 shutdown_connections 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fef78067ff0 0x7fef7806a4b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9815e600 0x7fef9814f440 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fef980d1100 0x7fef98151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 --2- 172.21.15.43:0/3508164947 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fef980b8160 0x7fef9814c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 >> 172.21.15.43:0/3508164947 conn(0x7fef980b6230 msgr2=0x7fef98153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 shutdown_connections 2024-03-30T13:14:45.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.825+0000 7fef9ce5c640 1 -- 172.21.15.43:0/3508164947 wait complete. 2024-03-30T13:14:45.846 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-30T13:10:43.880356+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-30T13:12:25.924561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-03-30T13:12:26.302922+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 45, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-30T13:14:45.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-30T13:14:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8e57d640 1 Processor -- start 2024-03-30T13:14:45.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8e57d640 1 -- start start 2024-03-30T13:14:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8e57d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fea8806cd80 0x7fea88166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8e57d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8806d710 con 0x7fea8813f140 2024-03-30T13:14:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8e57d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea8806d9f0 con 0x7fea8815ff40 2024-03-30T13:14:45.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8e57d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fea8806e320 con 0x7fea8806cd80 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea87fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fea8813f140 0x7fea881606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34044/0 (socket says 172.21.15.43:34044) 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea87fff640 1 -- 172.21.15.43:0/2903697173 learned_addr learned my addr 172.21.15.43:0/2903697173 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8cb1a640 1 --2- 172.21.15.43:0/2903697173 >> v2:172.21.15.43:3300/0 conn(0x7fea8806cd80 0x7fea88166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea86ffd640 1 -- 172.21.15.43:0/2903697173 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031690244 0 0) 0x7fea8806d710 con 0x7fea8813f140 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea86ffd640 1 -- 172.21.15.43:0/2903697173 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea70003530 con 0x7fea8813f140 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8cb1a640 1 -- 172.21.15.43:0/2903697173 >> v1:172.21.15.113:6789/0 conn(0x7fea8815ff40 legacy=0x7fea8806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8cb1a640 1 -- 172.21.15.43:0/2903697173 >> v1:172.21.15.43:6789/0 conn(0x7fea8813f140 legacy=0x7fea881606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.957+0000 7fea8cb1a640 1 -- 172.21.15.43:0/2903697173 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8806dcd0 con 0x7fea8806cd80 2024-03-30T13:14:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8cb1a640 1 --2- 172.21.15.43:0/2903697173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8806cd80 0x7fea88166fe0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fea80009d90 tx=0x7fea8001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c94f9a4ded1f471 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea86ffd640 1 -- 172.21.15.43:0/2903697173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b2070 con 0x7fea8806cd80 2024-03-30T13:14:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea86ffd640 1 -- 172.21.15.43:0/2903697173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea800a5940 con 0x7fea8806cd80 2024-03-30T13:14:45.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea86ffd640 1 -- 172.21.15.43:0/2903697173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800a5d10 con 0x7fea8806cd80 2024-03-30T13:14:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- 172.21.15.43:0/2903697173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8806cd80 msgr2=0x7fea88166fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 --2- 172.21.15.43:0/2903697173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8806cd80 0x7fea88166fe0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fea80009d90 tx=0x7fea8001c970 comp rx=0 tx=0).stop 2024-03-30T13:14:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- 172.21.15.43:0/2903697173 shutdown_connections 2024-03-30T13:14:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 --2- 172.21.15.43:0/2903697173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8806cd80 0x7fea88166fe0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- 172.21.15.43:0/2903697173 >> 172.21.15.43:0/2903697173 conn(0x7fea880d00a0 msgr2=0x7fea880d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- 172.21.15.43:0/2903697173 shutdown_connections 2024-03-30T13:14:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- 172.21.15.43:0/2903697173 wait complete. 2024-03-30T13:14:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 Processor -- start 2024-03-30T13:14:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- start start 2024-03-30T13:14:45.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8806cd80 0x7fea88139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8813f140 0x7fea8815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8815ff40 0x7fea88137040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fea8816a0d0 con 0x7fea8813f140 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fea88169f50 con 0x7fea8806cd80 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.961+0000 7fea8e57d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fea8816a250 con 0x7fea8815ff40 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8813f140 0x7fea8815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8813f140 0x7fea8815b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44250/0 (socket says 172.21.15.43:44250) 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 -- 172.21.15.43:0/3591097068 learned_addr learned my addr 172.21.15.43:0/3591097068 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea8cb1a640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8815ff40 0x7fea88137040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea87fff640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8806cd80 0x7fea88139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 -- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8815ff40 msgr2=0x7fea88137040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8815ff40 0x7fea88137040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 -- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8806cd80 msgr2=0x7fea88139210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8806cd80 0x7fea88139210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 -- 172.21.15.43:0/3591097068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea880b47a0 con 0x7fea8813f140 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea877fe640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8813f140 0x7fea8815b5a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c390 tx=0x7fea7c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea84ff9640 1 -- 172.21.15.43:0/3591097068 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c017070 con 0x7fea8813f140 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea84ff9640 1 -- 172.21.15.43:0/3591097068 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea7c0052d0 con 0x7fea8813f140 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea880b4a80 con 0x7fea8813f140 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea84ff9640 1 -- 172.21.15.43:0/3591097068 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c003820 con 0x7fea8813f140 2024-03-30T13:14:45.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea88136840 con 0x7fea8813f140 2024-03-30T13:14:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea88068900 con 0x7fea8813f140 2024-03-30T13:14:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea84ff9640 1 -- 172.21.15.43:0/3591097068 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fea7c01a070 con 0x7fea8813f140 2024-03-30T13:14:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea84ff9640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea74067fc0 0x7fea7406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea84ff9640 1 -- 172.21.15.43:0/3591097068 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fea7c085930 con 0x7fea8813f140 2024-03-30T13:14:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.965+0000 7fea87fff640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea74067fc0 0x7fea7406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.969+0000 7fea87fff640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea74067fc0 0x7fea7406a480 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fea780011f0 tx=0x7fea78007d40 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:45.969+0000 7fea84ff9640 1 -- 172.21.15.43:0/3591097068 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fea7c005480 con 0x7fea8813f140 2024-03-30T13:14:46.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.125+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea880afac0 con 0x7fea8813f140 2024-03-30T13:14:46.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea84ff9640 1 -- 172.21.15.43:0/3591097068 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fea7c055940 con 0x7fea8813f140 2024-03-30T13:14:46.133 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea74067fc0 msgr2=0x7fea7406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea74067fc0 0x7fea7406a480 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fea780011f0 tx=0x7fea78007d40 comp rx=0 tx=0).stop 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8813f140 msgr2=0x7fea8815b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8813f140 0x7fea8815b5a0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c390 tx=0x7fea7c00c850 comp rx=0 tx=0).stop 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 shutdown_connections 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fea74067fc0 0x7fea7406a480 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fea8815ff40 0x7fea88137040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea8813f140 0x7fea8815b5a0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 --2- 172.21.15.43:0/3591097068 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fea8806cd80 0x7fea88139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 >> 172.21.15.43:0/3591097068 conn(0x7fea880d00a0 msgr2=0x7fea880afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 shutdown_connections 2024-03-30T13:14:46.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.129+0000 7fea8e57d640 1 -- 172.21.15.43:0/3591097068 wait complete. 2024-03-30T13:14:46.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-30T13:14:46.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb777ae6640 1 Processor -- start 2024-03-30T13:14:46.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb777ae6640 1 -- start start 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb777ae6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fb77015ff40 0x7fb77006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb777ae6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77006cbb0 0x7fb7700ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb777ae6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb77006d710 con 0x7fb77013f140 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb777ae6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fb77006e320 con 0x7fb77015ff40 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb777ae6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb770160de0 con 0x7fb77006cbb0 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb775882640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb77013f140 0x7fb7701606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34052/0 (socket says 172.21.15.43:34052) 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb775882640 1 -- 172.21.15.43:0/858871003 learned_addr learned my addr 172.21.15.43:0/858871003 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb776083640 1 --2- 172.21.15.43:0/858871003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77006cbb0 0x7fb7700ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb775081640 1 --2- 172.21.15.43:0/858871003 >> v2:172.21.15.113:3300/0 conn(0x7fb77015ff40 0x7fb77006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb774880640 1 -- 172.21.15.43:0/858871003 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223379500 0 0) 0x7fb77006d710 con 0x7fb77013f140 2024-03-30T13:14:46.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb774880640 1 -- 172.21.15.43:0/858871003 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb75c003530 con 0x7fb77013f140 2024-03-30T13:14:46.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb774880640 1 -- 172.21.15.43:0/858871003 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2086782980 0 0) 0x7fb75c003530 con 0x7fb77013f140 2024-03-30T13:14:46.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb776083640 1 -- 172.21.15.43:0/858871003 >> v2:172.21.15.113:3300/0 conn(0x7fb77015ff40 msgr2=0x7fb77006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:46.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb776083640 1 --2- 172.21.15.43:0/858871003 >> v2:172.21.15.113:3300/0 conn(0x7fb77015ff40 0x7fb77006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb776083640 1 -- 172.21.15.43:0/858871003 >> v1:172.21.15.43:6789/0 conn(0x7fb77013f140 legacy=0x7fb7701606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.265+0000 7fb776083640 1 -- 172.21.15.43:0/858871003 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77006d9f0 con 0x7fb77006cbb0 2024-03-30T13:14:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb776083640 1 --2- 172.21.15.43:0/858871003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77006cbb0 0x7fb7700ce940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d90 tx=0x7fb76801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31e78216647dd47e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb774880640 1 -- 172.21.15.43:0/858871003 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680b2070 con 0x7fb77006cbb0 2024-03-30T13:14:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb774880640 1 -- 172.21.15.43:0/858871003 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7680a5580 con 0x7fb77006cbb0 2024-03-30T13:14:46.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb774880640 1 -- 172.21.15.43:0/858871003 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c001680 con 0x7fb77006cbb0 2024-03-30T13:14:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- 172.21.15.43:0/858871003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77006cbb0 msgr2=0x7fb7700ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 --2- 172.21.15.43:0/858871003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77006cbb0 0x7fb7700ce940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb768009d90 tx=0x7fb76801c890 comp rx=0 tx=0).stop 2024-03-30T13:14:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- 172.21.15.43:0/858871003 shutdown_connections 2024-03-30T13:14:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 --2- 172.21.15.43:0/858871003 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77006cbb0 0x7fb7700ce940 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 --2- 172.21.15.43:0/858871003 >> v2:172.21.15.113:3300/0 conn(0x7fb77015ff40 0x7fb77006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- 172.21.15.43:0/858871003 >> 172.21.15.43:0/858871003 conn(0x7fb7700d00a0 msgr2=0x7fb7700d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:46.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- 172.21.15.43:0/858871003 shutdown_connections 2024-03-30T13:14:46.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- 172.21.15.43:0/858871003 wait complete. 2024-03-30T13:14:46.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 Processor -- start 2024-03-30T13:14:46.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- start start 2024-03-30T13:14:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77006cbb0 0x7fb77013d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb77013f140 0x7fb770073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77015ff40 0x7fb77015b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700cf810 con 0x7fb77013f140 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb7700cf690 con 0x7fb77006cbb0 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb777ae6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb7700cf990 con 0x7fb77015ff40 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb77013f140 0x7fb770073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb77013f140 0x7fb770073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44264/0 (socket says 172.21.15.43:44264) 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 -- 172.21.15.43:0/2459595605 learned_addr learned my addr 172.21.15.43:0/2459595605 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775882640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77006cbb0 0x7fb77013d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb776083640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77015ff40 0x7fb77015b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 -- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77015ff40 msgr2=0x7fb77015b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77015ff40 0x7fb77015b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 -- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77006cbb0 msgr2=0x7fb77013d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77006cbb0 0x7fb77013d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 -- 172.21.15.43:0/2459595605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7700b40a0 con 0x7fb77013f140 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.269+0000 7fb775081640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb77013f140 0x7fb770073030 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e870 tx=0x7fb76000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb7667fc640 1 -- 172.21.15.43:0/2459595605 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760017070 con 0x7fb77013f140 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb7667fc640 1 -- 172.21.15.43:0/2459595605 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76000c940 con 0x7fb77013f140 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7700b4380 con 0x7fb77013f140 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb7667fc640 1 -- 172.21.15.43:0/2459595605 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76000cd30 con 0x7fb77013f140 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7700b4940 con 0x7fb77013f140 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb770068900 con 0x7fb77013f140 2024-03-30T13:14:46.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb7667fc640 1 -- 172.21.15.43:0/2459595605 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb76001a070 con 0x7fb77013f140 2024-03-30T13:14:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb7667fc640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb754067fc0 0x7fb75406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb7667fc640 1 -- 172.21.15.43:0/2459595605 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fb760086ac0 con 0x7fb77013f140 2024-03-30T13:14:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb775882640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb754067fc0 0x7fb75406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.273+0000 7fb775882640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb754067fc0 0x7fb75406a480 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007a10 tx=0x7fb76c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.277+0000 7fb7667fc640 1 -- 172.21.15.43:0/2459595605 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb760056f60 con 0x7fb77013f140 2024-03-30T13:14:46.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.433+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb7700b4d40 con 0x7fb77013f140 2024-03-30T13:14:46.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.433+0000 7fb7667fc640 1 -- 172.21.15.43:0/2459595605 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) ==== 104+0+11 (secure 0 0 0) 0x7fb760056860 con 0x7fb77013f140 2024-03-30T13:14:46.440 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-03-30T13:14:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb754067fc0 msgr2=0x7fb75406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb754067fc0 0x7fb75406a480 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb76c007a10 tx=0x7fb76c009040 comp rx=0 tx=0).stop 2024-03-30T13:14:46.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb77013f140 msgr2=0x7fb770073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb77013f140 0x7fb770073030 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb76000e870 tx=0x7fb76000ec70 comp rx=0 tx=0).stop 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 shutdown_connections 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb754067fc0 0x7fb75406a480 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb77015ff40 0x7fb77015b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb77013f140 0x7fb770073030 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 --2- 172.21.15.43:0/2459595605 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb77006cbb0 0x7fb77013d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 >> 172.21.15.43:0/2459595605 conn(0x7fb7700d00a0 msgr2=0x7fb7700b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 shutdown_connections 2024-03-30T13:14:46.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.437+0000 7fb777ae6640 1 -- 172.21.15.43:0/2459595605 wait complete. 2024-03-30T13:14:46.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-30T13:14:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f2951767640 1 Processor -- start 2024-03-30T13:14:46.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f2951767640 1 -- start start 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f2951767640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f294c144930 0x7f294c0b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f2951767640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f294c06cd80 0x7f294c0ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f2951767640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f294c0b33c0 con 0x7f294c0b4870 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f2951767640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f294c0cdb90 con 0x7f294c06cd80 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f2951767640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f294c0b3660 con 0x7f294c144930 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f294c06cd80 0x7f294c0ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f294c06cd80 0x7f294c0ce020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44276/0 (socket says 172.21.15.43:44276) 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294b7fe640 1 -- 172.21.15.43:0/1401123801 learned_addr learned my addr 172.21.15.43:0/1401123801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:46.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f294c0b4870 0x7f294c06c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34054/0 (socket says 172.21.15.43:34054) 2024-03-30T13:14:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294affd640 1 --2- 172.21.15.43:0/1401123801 >> v2:172.21.15.113:3300/0 conn(0x7f294c144930 0x7f294c0b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294b7fe640 1 -- 172.21.15.43:0/1401123801 >> v2:172.21.15.113:3300/0 conn(0x7f294c144930 msgr2=0x7f294c0b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294b7fe640 1 --2- 172.21.15.43:0/1401123801 >> v2:172.21.15.113:3300/0 conn(0x7f294c144930 0x7f294c0b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294b7fe640 1 -- 172.21.15.43:0/1401123801 >> v1:172.21.15.43:6789/0 conn(0x7f294c0b4870 legacy=0x7f294c06c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.569+0000 7f294b7fe640 1 -- 172.21.15.43:0/1401123801 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c0b3940 con 0x7f294c06cd80 2024-03-30T13:14:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294b7fe640 1 --2- 172.21.15.43:0/1401123801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c06cd80 0x7f294c0ce020 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f294c0cdd10 tx=0x7f293c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd21b4068ae0c048 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2949ffb640 1 -- 172.21.15.43:0/1401123801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0a57b0 con 0x7f294c06cd80 2024-03-30T13:14:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2949ffb640 1 -- 172.21.15.43:0/1401123801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f293c0a5950 con 0x7f294c06cd80 2024-03-30T13:14:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2949ffb640 1 -- 172.21.15.43:0/1401123801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293c0a5d20 con 0x7f294c06cd80 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- 172.21.15.43:0/1401123801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c06cd80 msgr2=0x7f294c0ce020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 --2- 172.21.15.43:0/1401123801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c06cd80 0x7f294c0ce020 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f294c0cdd10 tx=0x7f293c01c930 comp rx=0 tx=0).stop 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- 172.21.15.43:0/1401123801 shutdown_connections 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 --2- 172.21.15.43:0/1401123801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c06cd80 0x7f294c0ce020 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 --2- 172.21.15.43:0/1401123801 >> v2:172.21.15.113:3300/0 conn(0x7f294c144930 0x7f294c0b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- 172.21.15.43:0/1401123801 >> 172.21.15.43:0/1401123801 conn(0x7f294c0d00a0 msgr2=0x7f294c0d04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- 172.21.15.43:0/1401123801 shutdown_connections 2024-03-30T13:14:46.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- 172.21.15.43:0/1401123801 wait complete. 2024-03-30T13:14:46.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 Processor -- start 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- start start 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c06cd80 0x7f294c1f8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c0b4870 0x7f294c06c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c144930 0x7f294c1f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f294c0ceef0 con 0x7f294c0b4870 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f294c0ced70 con 0x7f294c144930 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f294c0cf070 con 0x7f294c06cd80 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c0b4870 0x7f294c06c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c0b4870 0x7f294c06c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44286/0 (socket says 172.21.15.43:44286) 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 -- 172.21.15.43:0/1146072082 learned_addr learned my addr 172.21.15.43:0/1146072082 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294affd640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c06cd80 0x7f294c1f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294b7fe640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c144930 0x7f294c1f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 -- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c06cd80 msgr2=0x7f294c1f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c06cd80 0x7f294c1f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 -- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c144930 msgr2=0x7f294c1f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c144930 0x7f294c1f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 -- 172.21.15.43:0/1146072082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c156f80 con 0x7f294c0b4870 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f294a7fc640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c0b4870 0x7f294c06c5c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f294000c420 tx=0x7f294000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f29337fe640 1 -- 172.21.15.43:0/1146072082 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400074e0 con 0x7f294c0b4870 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f29337fe640 1 -- 172.21.15.43:0/1146072082 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2940007680 con 0x7f294c0b4870 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c1591f0 con 0x7f294c0b4870 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f29337fe640 1 -- 172.21.15.43:0/1146072082 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940004a30 con 0x7f294c0b4870 2024-03-30T13:14:46.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.573+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c1594d0 con 0x7f294c0b4870 2024-03-30T13:14:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.577+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294c068900 con 0x7f294c0b4870 2024-03-30T13:14:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.577+0000 7f29337fe640 1 -- 172.21.15.43:0/1146072082 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f294001a020 con 0x7f294c0b4870 2024-03-30T13:14:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.577+0000 7f29337fe640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2938067fc0 0x7f293806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.577+0000 7f29337fe640 1 -- 172.21.15.43:0/1146072082 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f2940085f30 con 0x7f294c0b4870 2024-03-30T13:14:46.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.577+0000 7f294affd640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2938067fc0 0x7f293806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.577+0000 7f294affd640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2938067fc0 0x7f293806a480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f294400b930 tx=0x7f2944008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.581+0000 7f29337fe640 1 -- 172.21.15.43:0/1146072082 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2940056500 con 0x7f294c0b4870 2024-03-30T13:14:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.737+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f294c1572d0 con 0x7f294c0b4870 2024-03-30T13:14:46.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.737+0000 7f29337fe640 1 -- 172.21.15.43:0/1146072082 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) ==== 100+0+11 (secure 0 0 0) 0x7f2940055e00 con 0x7f294c0b4870 2024-03-30T13:14:46.743 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-03-30T13:14:46.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2938067fc0 msgr2=0x7f293806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2938067fc0 0x7f293806a480 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f294400b930 tx=0x7f2944008040 comp rx=0 tx=0).stop 2024-03-30T13:14:46.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c0b4870 msgr2=0x7f294c06c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c0b4870 0x7f294c06c5c0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f294000c420 tx=0x7f294000c8e0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 shutdown_connections 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2938067fc0 0x7f293806a480 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f294c144930 0x7f294c1f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f294c0b4870 0x7f294c06c5c0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 --2- 172.21.15.43:0/1146072082 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f294c06cd80 0x7f294c1f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 >> 172.21.15.43:0/1146072082 conn(0x7f294c0d00a0 msgr2=0x7f294c15df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 shutdown_connections 2024-03-30T13:14:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.741+0000 7f2951767640 1 -- 172.21.15.43:0/1146072082 wait complete. 2024-03-30T13:14:46.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:14:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f4824270640 1 Processor -- start 2024-03-30T13:14:46.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f4824270640 1 -- start start 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f4824270640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cef80 0x7f481c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f4824270640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f481c0b3f50 0x7f481c0cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f4824270640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481c0b4500 con 0x7f481c0cf360 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f4824270640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f481c0b47e0 con 0x7f481c0b3f50 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f4824270640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f481c0b4ac0 con 0x7f481c0cef80 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482280d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f481c0b3f50 0x7f481c0cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482180b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f481c0cf360 0x7f481c0b3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34064/0 (socket says 172.21.15.43:34064) 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482180b640 1 -- 172.21.15.43:0/1485981905 learned_addr learned my addr 172.21.15.43:0/1485981905 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482280d640 1 --2- 172.21.15.43:0/1485981905 >> v2:172.21.15.43:3300/0 conn(0x7f481c0b3f50 0x7f481c0cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44302/0 (socket says 172.21.15.43:44302) 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482200c640 1 --2- 172.21.15.43:0/1485981905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cef80 0x7f481c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482280d640 1 -- 172.21.15.43:0/1485981905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cef80 msgr2=0x7f481c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482280d640 1 --2- 172.21.15.43:0/1485981905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cef80 0x7f481c0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482280d640 1 -- 172.21.15.43:0/1485981905 >> v1:172.21.15.43:6789/0 conn(0x7f481c0cf360 legacy=0x7f481c0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482280d640 1 -- 172.21.15.43:0/1485981905 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c0b4da0 con 0x7f481c0b3f50 2024-03-30T13:14:46.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.869+0000 7f482100a640 1 -- 172.21.15.43:0/1485981905 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140340989 0 0) 0x7f481c0b4500 con 0x7f481c0cf360 2024-03-30T13:14:46.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482280d640 1 --2- 172.21.15.43:0/1485981905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c0cf7c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4814009d90 tx=0x7f481401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7ef109fe0b1de7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482100a640 1 -- 172.21.15.43:0/1485981905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140b2070 con 0x7f481c0b3f50 2024-03-30T13:14:46.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482100a640 1 -- 172.21.15.43:0/1485981905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48140a5830 con 0x7f481c0b3f50 2024-03-30T13:14:46.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482100a640 1 -- 172.21.15.43:0/1485981905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48140a5c00 con 0x7f481c0b3f50 2024-03-30T13:14:46.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- 172.21.15.43:0/1485981905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 msgr2=0x7f481c0cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 --2- 172.21.15.43:0/1485981905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c0cf7c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f4814009d90 tx=0x7f481401c970 comp rx=0 tx=0).stop 2024-03-30T13:14:46.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- 172.21.15.43:0/1485981905 shutdown_connections 2024-03-30T13:14:46.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 --2- 172.21.15.43:0/1485981905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c0cf7c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 --2- 172.21.15.43:0/1485981905 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cef80 0x7f481c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- 172.21.15.43:0/1485981905 >> 172.21.15.43:0/1485981905 conn(0x7f481c0d4310 msgr2=0x7f481c0d4720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- 172.21.15.43:0/1485981905 shutdown_connections 2024-03-30T13:14:46.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- 172.21.15.43:0/1485981905 wait complete. 2024-03-30T13:14:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 Processor -- start 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- start start 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f481c0cef80 0x7f481c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cf360 0x7f481c156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f481c16aeb0 con 0x7f481c0b3f50 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f481c16ad30 con 0x7f481c0cef80 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f4824270640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f481c16b030 con 0x7f481c0cf360 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c15dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44304/0 (socket says 172.21.15.43:44304) 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 -- 172.21.15.43:0/2894612401 learned_addr learned my addr 172.21.15.43:0/2894612401 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:46.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482280d640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cf360 0x7f481c156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 -- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cf360 msgr2=0x7f481c156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482180b640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f481c0cef80 0x7f481c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cf360 0x7f481c156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 -- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f481c0cef80 msgr2=0x7f481c1566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f481c0cef80 0x7f481c1566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482200c640 1 -- 172.21.15.43:0/2894612401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c0c76e0 con 0x7f481c0b3f50 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.873+0000 7f482280d640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cf360 0x7f481c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f482200c640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c15dc50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f481800e930 tx=0x7f481800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4812ffd640 1 -- 172.21.15.43:0/2894612401 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4818017070 con 0x7f481c0b3f50 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4812ffd640 1 -- 172.21.15.43:0/2894612401 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481800ca00 con 0x7f481c0b3f50 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c158970 con 0x7f481c0b3f50 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4812ffd640 1 -- 172.21.15.43:0/2894612401 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f481800cdb0 con 0x7f481c0b3f50 2024-03-30T13:14:46.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c15e190 con 0x7f481c0b3f50 2024-03-30T13:14:46.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481c0c69f0 con 0x7f481c0b3f50 2024-03-30T13:14:46.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4812ffd640 1 -- 172.21.15.43:0/2894612401 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f481801a070 con 0x7f481c0b3f50 2024-03-30T13:14:46.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4812ffd640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4800067fc0 0x7f480006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:46.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f4812ffd640 1 -- 172.21.15.43:0/2894612401 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f4818086710 con 0x7f481c0b3f50 2024-03-30T13:14:46.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f482180b640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4800067fc0 0x7f480006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:46.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.877+0000 7f482180b640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4800067fc0 0x7f480006a480 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f481c15c3d0 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:46.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:46.881+0000 7f4812ffd640 1 -- 172.21.15.43:0/2894612401 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4818056e30 con 0x7f481c0b3f50 2024-03-30T13:14:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.077+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f481c068890 con 0x7f481c0b3f50 2024-03-30T13:14:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.077+0000 7f4812ffd640 1 -- 172.21.15.43:0/2894612401 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5187 (secure 0 0 0) 0x7f4818056730 con 0x7f481c0b3f50 2024-03-30T13:14:47.084 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:47.084 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4271726137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4271726137},{"type":"v1","addr":"172.21.15.43:6834","nonce":4271726137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1748034672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1748034672},{"type":"v1","addr":"172.21.15.43:6835","nonce":1748034672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3254700273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3254700273},{"type":"v1","addr":"172.21.15.113:6835","nonce":3254700273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:12:27.356973+0000","modified":"2024-03-30T13:14:35.770003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4880":{"gid":4880,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1104336007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1104336007},{"type":"v1","addr":"172.21.15.113:6837","nonce":1104336007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4880,"qdb_cluster":[4880]},"id":2}]} 2024-03-30T13:14:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-03-30T13:14:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4800067fc0 msgr2=0x7f480006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4800067fc0 0x7f480006a480 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f481c15c3d0 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 msgr2=0x7f481c15dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c15dc50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f481800e930 tx=0x7f481800edf0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 shutdown_connections 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4800067fc0 0x7f480006a480 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f481c0cf360 0x7f481c156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f481c0cef80 0x7f481c1566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 --2- 172.21.15.43:0/2894612401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0b3f50 0x7f481c15dc50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 >> 172.21.15.43:0/2894612401 conn(0x7f481c0d4310 msgr2=0x7f481c0d32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 shutdown_connections 2024-03-30T13:14:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.081+0000 7f4824270640 1 -- 172.21.15.43:0/2894612401 wait complete. 2024-03-30T13:14:47.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:14:47.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4adb2d640 1 Processor -- start 2024-03-30T13:14:47.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4adb2d640 1 -- start start 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4adb2d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a80cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4adb2d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4a80b41f0 0x7fc4a806e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4adb2d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a806e5c0 con 0x7fc4a806ca50 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4adb2d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a80b34c0 con 0x7fc4a80b41f0 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4adb2d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a80ccef0 con 0x7fc4a806c670 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4a80b41f0 0x7fc4a806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4a80b41f0 0x7fc4a806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44320/0 (socket says 172.21.15.43:44320) 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/315575207 learned_addr learned my addr 172.21.15.43:0/315575207 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a77fe640 1 --2- 172.21.15.43:0/315575207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a80cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/315575207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 msgr2=0x7fc4a80cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a6ffd640 1 --2- 172.21.15.43:0/315575207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a80cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/315575207 >> v1:172.21.15.113:6789/0 conn(0x7fc4a806ca50 legacy=0x7fc4a815fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.221+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/315575207 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a80b3710 con 0x7fc4a80b41f0 2024-03-30T13:14:47.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a6ffd640 1 --2- 172.21.15.43:0/315575207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a80b41f0 0x7fc4a806e080 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc490009870 tx=0x7fc49001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0d737029f7b576 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a67fc640 1 -- 172.21.15.43:0/315575207 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4900b0070 con 0x7fc4a80b41f0 2024-03-30T13:14:47.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a67fc640 1 -- 172.21.15.43:0/315575207 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4900ab600 con 0x7fc4a80b41f0 2024-03-30T13:14:47.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a67fc640 1 -- 172.21.15.43:0/315575207 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4900ab9d0 con 0x7fc4a80b41f0 2024-03-30T13:14:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- 172.21.15.43:0/315575207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a80b41f0 msgr2=0x7fc4a806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/315575207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a80b41f0 0x7fc4a806e080 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc490009870 tx=0x7fc49001c560 comp rx=0 tx=0).stop 2024-03-30T13:14:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- 172.21.15.43:0/315575207 shutdown_connections 2024-03-30T13:14:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/315575207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a80b41f0 0x7fc4a806e080 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/315575207 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a80cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- 172.21.15.43:0/315575207 >> 172.21.15.43:0/315575207 conn(0x7fc4a80cfee0 msgr2=0x7fc4a80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:47.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- 172.21.15.43:0/315575207 shutdown_connections 2024-03-30T13:14:47.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- 172.21.15.43:0/315575207 wait complete. 2024-03-30T13:14:47.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 Processor -- start 2024-03-30T13:14:47.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- start start 2024-03-30T13:14:47.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a806ca50 0x7fc4a8158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4a80b41f0 0x7fc4a815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a806cfe0 con 0x7fc4a806ca50 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a806ce60 con 0x7fc4a80b41f0 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4adb2d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a8168ef0 con 0x7fc4a806c670 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a806ca50 0x7fc4a8158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a806ca50 0x7fc4a8158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44332/0 (socket says 172.21.15.43:44332) 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/69922124 learned_addr learned my addr 172.21.15.43:0/69922124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a7fff640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4a80b41f0 0x7fc4a815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.225+0000 7fc4a77fe640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 msgr2=0x7fc4a8155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a6ffd640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a8155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4a80b41f0 msgr2=0x7fc4a815a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a6ffd640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4a80b41f0 0x7fc4a815a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a6ffd640 1 -- 172.21.15.43:0/69922124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a813d750 con 0x7fc4a806ca50 2024-03-30T13:14:47.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a77fe640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:47.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a6ffd640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a806ca50 0x7fc4a8158d10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc4900ae040 tx=0x7fc490019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc49ffff640 1 -- 172.21.15.43:0/69922124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4900b9070 con 0x7fc4a806ca50 2024-03-30T13:14:47.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc49ffff640 1 -- 172.21.15.43:0/69922124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4900b4650 con 0x7fc4a806ca50 2024-03-30T13:14:47.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a813da30 con 0x7fc4a806ca50 2024-03-30T13:14:47.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc49ffff640 1 -- 172.21.15.43:0/69922124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4900b0040 con 0x7fc4a806ca50 2024-03-30T13:14:47.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a813e2d0 con 0x7fc4a806ca50 2024-03-30T13:14:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a813de20 con 0x7fc4a806ca50 2024-03-30T13:14:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc49ffff640 1 -- 172.21.15.43:0/69922124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc4900bc070 con 0x7fc4a806ca50 2024-03-30T13:14:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc49ffff640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc488067fc0 0x7fc48806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc49ffff640 1 -- 172.21.15.43:0/69922124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fc49011fa20 con 0x7fc4a806ca50 2024-03-30T13:14:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a77fe640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc488067fc0 0x7fc48806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.229+0000 7fc4a77fe640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc488067fc0 0x7fc48806a480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc49800b9f0 tx=0x7fc498008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.233+0000 7fc49ffff640 1 -- 172.21.15.43:0/69922124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc490016150 con 0x7fc4a806ca50 2024-03-30T13:14:47.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.393+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4a8068890 con 0x7fc4a806ca50 2024-03-30T13:14:47.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.393+0000 7fc49ffff640 1 -- 172.21.15.43:0/69922124 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+17021 (secure 0 0 0) 0x7fc4900ef910 con 0x7fc4a806ca50 2024-03-30T13:14:47.399 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:47.399 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:14:40.895915+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3c","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:14:47.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc488067fc0 msgr2=0x7fc48806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc488067fc0 0x7fc48806a480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc49800b9f0 tx=0x7fc498008040 comp rx=0 tx=0).stop 2024-03-30T13:14:47.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a806ca50 msgr2=0x7fc4a8158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a806ca50 0x7fc4a8158d10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fc4900ae040 tx=0x7fc490019300 comp rx=0 tx=0).stop 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 shutdown_connections 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc488067fc0 0x7fc48806a480 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc4a80b41f0 0x7fc4a815a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4a806ca50 0x7fc4a8158d10 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 --2- 172.21.15.43:0/69922124 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc4a806c670 0x7fc4a8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 >> 172.21.15.43:0/69922124 conn(0x7fc4a80cfee0 msgr2=0x7fc4a815dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 shutdown_connections 2024-03-30T13:14:47.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.397+0000 7fc4adb2d640 1 -- 172.21.15.43:0/69922124 wait complete. 2024-03-30T13:14:47.417 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T13:14:47.418 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:14:47.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.529+0000 7fc8cda21640 1 Processor -- start 2024-03-30T13:14:47.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.529+0000 7fc8cda21640 1 -- start start 2024-03-30T13:14:47.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8cda21640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fc8c80b3b80 0x7fc8c806c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8cda21640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c80ad8e0 con 0x7fc8c80b3720 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8cda21640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8c815e750 con 0x7fc8c815e370 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8cda21640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fc8c80b4550 con 0x7fc8c80b3b80 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c67fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc8c80b3720 0x7fc8c806bf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34070/0 (socket says 172.21.15.43:34070) 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c67fc640 1 -- 172.21.15.43:0/3702545934 learned_addr learned my addr 172.21.15.43:0/3702545934 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/3702545934 >> v2:172.21.15.113:3300/0 conn(0x7fc8c80b3b80 0x7fc8c806c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830337785 0 0) 0x7fc8c80ad8e0 con 0x7fc8c80b3720 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8b0003530 con 0x7fc8c80b3720 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3215420329 0 0) 0x7fc8b0003530 con 0x7fc8c80b3720 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 >> v2:172.21.15.113:3300/0 conn(0x7fc8c80b3b80 msgr2=0x7fc8c806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 --2- 172.21.15.43:0/3702545934 >> v2:172.21.15.113:3300/0 conn(0x7fc8c80b3b80 0x7fc8c806c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 >> v1:172.21.15.113:6789/0 conn(0x7fc8c815e370 legacy=0x7fc8c80afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c806e8a0 con 0x7fc8c80b3720 2024-03-30T13:14:47.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7fc8bc001680 con 0x7fc8c80b3720 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 >> v1:172.21.15.43:6789/0 conn(0x7fc8c80b3720 legacy=0x7fc8c806bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00039c0 0x7fc8b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0004810 0x7fc8b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8b00051d0 0x7fc8b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b000c040 con 0x7fc8b00051d0 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b000c1f0 con 0x7fc8b00039c0 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc8b000c3f0 con 0x7fc8b0004810 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8b00051d0 0x7fc8b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c6ffd640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00039c0 0x7fc8b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c67fc640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0004810 0x7fc8b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 -- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0004810 msgr2=0x7fc8b0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b0004810 0x7fc8b0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 -- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00039c0 msgr2=0x7fc8b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8b00039c0 0x7fc8b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 -- 172.21.15.43:0/3702545934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b0003530 con 0x7fc8b00051d0 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 -- 172.21.15.43:0/3702545934 reap_dead start 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.533+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8b00051d0 0x7fc8b000b9d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009fc0 tx=0x7fc8c001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18402c11d94f630 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0009c10 con 0x7fc8b00051d0 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c00a5d10 con 0x7fc8b00051d0 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c5ffb640 1 -- 172.21.15.43:0/3702545934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0003be0 con 0x7fc8b00051d0 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- 172.21.15.43:0/3702545934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8b00051d0 msgr2=0x7fc8b000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8b00051d0 0x7fc8b000b9d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009fc0 tx=0x7fc8c001ccc0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- 172.21.15.43:0/3702545934 shutdown_connections 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 --2- 172.21.15.43:0/3702545934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8b00051d0 0x7fc8b000b9d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- 172.21.15.43:0/3702545934 >> 172.21.15.43:0/3702545934 conn(0x7fc8c80b2b30 msgr2=0x7fc8c80d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- 172.21.15.43:0/3702545934 shutdown_connections 2024-03-30T13:14:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- 172.21.15.43:0/3702545934 wait complete. 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 Processor -- start 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- start start 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00051d0 0x7fc8c815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8c8160d10 0x7fc8c8147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c8161190 0x7fc8c81482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c806ee60 con 0x7fc8c8161190 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c806ba70 con 0x7fc8c8160d10 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fc8c806b700 con 0x7fc8b00051d0 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c8161190 0x7fc8c81482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c8161190 0x7fc8c81482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44350/0 (socket says 172.21.15.43:44350) 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 -- 172.21.15.43:0/2962580282 learned_addr learned my addr 172.21.15.43:0/2962580282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c67fc640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8c8160d10 0x7fc8c8147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c6ffd640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00051d0 0x7fc8c815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 -- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00051d0 msgr2=0x7fc8c815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00051d0 0x7fc8c815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 -- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8c8160d10 msgr2=0x7fc8c8147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8c8160d10 0x7fc8c8147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 -- 172.21.15.43:0/2962580282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c815ae40 con 0x7fc8c8161190 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c67fc640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8c8160d10 0x7fc8c8147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8c77fe640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c8161190 0x7fc8c81482c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00987f0 tx=0x7fc8c0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8affff640 1 -- 172.21.15.43:0/2962580282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c00aed50 con 0x7fc8c8161190 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8affff640 1 -- 172.21.15.43:0/2962580282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c00bd070 con 0x7fc8c8161190 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c80cdf00 con 0x7fc8c8161190 2024-03-30T13:14:47.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8affff640 1 -- 172.21.15.43:0/2962580282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0003ba0 con 0x7fc8c8161190 2024-03-30T13:14:47.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.537+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c815ec60 con 0x7fc8c8161190 2024-03-30T13:14:47.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.541+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8c8068890 con 0x7fc8c8161190 2024-03-30T13:14:47.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.541+0000 7fc8affff640 1 -- 172.21.15.43:0/2962580282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fc8c00a5d10 con 0x7fc8c8161190 2024-03-30T13:14:47.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.541+0000 7fc8affff640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc8b4067fc0 0x7fc8b406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.541+0000 7fc8affff640 1 -- 172.21.15.43:0/2962580282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fc8c0121fa0 con 0x7fc8c8161190 2024-03-30T13:14:47.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.541+0000 7fc8c6ffd640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc8b4067fc0 0x7fc8b406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.541+0000 7fc8c6ffd640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc8b4067fc0 0x7fc8b406a480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8000c10 tx=0x7fc8b8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.545+0000 7fc8affff640 1 -- 172.21.15.43:0/2962580282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fc8c00c7020 con 0x7fc8c8161190 2024-03-30T13:14:47.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.733+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8c80b1340 con 0x7fc8c8161190 2024-03-30T13:14:47.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.733+0000 7fc8affff640 1 -- 172.21.15.43:0/2962580282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5187 (secure 0 0 0) 0x7fc8c00169d0 con 0x7fc8c8161190 2024-03-30T13:14:47.739 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:47.739 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4271726137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4271726137},{"type":"v1","addr":"172.21.15.43:6834","nonce":4271726137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1748034672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1748034672},{"type":"v1","addr":"172.21.15.43:6835","nonce":1748034672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3254700273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3254700273},{"type":"v1","addr":"172.21.15.113:6835","nonce":3254700273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:12:27.356973+0000","modified":"2024-03-30T13:14:35.770003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4880":{"gid":4880,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1104336007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1104336007},{"type":"v1","addr":"172.21.15.113:6837","nonce":1104336007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4880,"qdb_cluster":[4880]},"id":2}]} 2024-03-30T13:14:47.739 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-03-30T13:14:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc8b4067fc0 msgr2=0x7fc8b406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc8b4067fc0 0x7fc8b406a480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8000c10 tx=0x7fc8b8009040 comp rx=0 tx=0).stop 2024-03-30T13:14:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c8161190 msgr2=0x7fc8c81482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c8161190 0x7fc8c81482c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc8c00987f0 tx=0x7fc8c0009750 comp rx=0 tx=0).stop 2024-03-30T13:14:47.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 shutdown_connections 2024-03-30T13:14:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fc8b4067fc0 0x7fc8b406a480 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8c8161190 0x7fc8c81482c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fc8c8160d10 0x7fc8c8147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 --2- 172.21.15.43:0/2962580282 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fc8b00051d0 0x7fc8c815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 >> 172.21.15.43:0/2962580282 conn(0x7fc8c80b2b30 msgr2=0x7fc8c806dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 shutdown_connections 2024-03-30T13:14:47.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.737+0000 7fc8cda21640 1 -- 172.21.15.43:0/2962580282 wait complete. 2024-03-30T13:14:47.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:14:47.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.873+0000 7f9ee74d0640 1 Processor -- start 2024-03-30T13:14:47.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.873+0000 7f9ee74d0640 1 -- start start 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ee0144270 0x7f9ee0144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee00b33f0 con 0x7f9ee00b4490 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9ee0144fa0 con 0x7f9ee0144270 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee015fce0 con 0x7f9ee00b2ef0 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee526c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9ee0144270 0x7f9ee0144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee5a6d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9ee00b4490 0x7f9ee006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34072/0 (socket says 172.21.15.43:34072) 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee5a6d640 1 -- 172.21.15.43:0/1756811670 learned_addr learned my addr 172.21.15.43:0/1756811670 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee4a6b640 1 --2- 172.21.15.43:0/1756811670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee526c640 1 -- 172.21.15.43:0/1756811670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 msgr2=0x7f9ee015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee526c640 1 --2- 172.21.15.43:0/1756811670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee015f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee526c640 1 -- 172.21.15.43:0/1756811670 >> v1:172.21.15.43:6789/0 conn(0x7f9ee00b4490 legacy=0x7f9ee006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ed7fff640 1 -- 172.21.15.43:0/1756811670 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568323799 0 0) 0x7f9ee00b33f0 con 0x7f9ee00b4490 2024-03-30T13:14:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee526c640 1 -- 172.21.15.43:0/1756811670 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00b36d0 con 0x7f9ee0144270 2024-03-30T13:14:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee4a6b640 1 --2- 172.21.15.43:0/1756811670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:47.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee526c640 1 --2- 172.21.15.43:0/1756811670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0144270 0x7f9ee0144650 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0097c0 tx=0x7f9edc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147f49830fa97b server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ed7fff640 1 -- 172.21.15.43:0/1756811670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0a7070 con 0x7f9ee0144270 2024-03-30T13:14:47.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ed7fff640 1 -- 172.21.15.43:0/1756811670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9edc0b0070 con 0x7f9ee0144270 2024-03-30T13:14:47.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ed7fff640 1 -- 172.21.15.43:0/1756811670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc0ab890 con 0x7f9ee0144270 2024-03-30T13:14:47.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1756811670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0144270 msgr2=0x7f9ee0144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1756811670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0144270 0x7f9ee0144650 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0097c0 tx=0x7f9edc01c470 comp rx=0 tx=0).stop 2024-03-30T13:14:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1756811670 shutdown_connections 2024-03-30T13:14:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1756811670 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1756811670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee0144270 0x7f9ee0144650 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1756811670 >> 172.21.15.43:0/1756811670 conn(0x7f9ee00cfee0 msgr2=0x7f9ee00d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1756811670 shutdown_connections 2024-03-30T13:14:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.877+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1756811670 wait complete. 2024-03-30T13:14:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 Processor -- start 2024-03-30T13:14:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 -- start start 2024-03-30T13:14:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee0140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee00b4490 0x7f9ee01f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee0144270 0x7f9ee013f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee00cdd10 con 0x7f9ee00b4490 2024-03-30T13:14:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee006cf90 con 0x7f9ee0144270 2024-03-30T13:14:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee00cde90 con 0x7f9ee00b2ef0 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee00b4490 0x7f9ee01f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee00b4490 0x7f9ee01f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44368/0 (socket says 172.21.15.43:44368) 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 -- 172.21.15.43:0/1459276185 learned_addr learned my addr 172.21.15.43:0/1459276185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee526c640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee0140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 -- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 msgr2=0x7f9ee0140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee5a6d640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee0144270 0x7f9ee013f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee0140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 -- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee0144270 msgr2=0x7f9ee013f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee0144270 0x7f9ee013f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 -- 172.21.15.43:0/1459276185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee01409e0 con 0x7f9ee00b4490 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee4a6b640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee00b4490 0x7f9ee01f4ec0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000c7f0 tx=0x7f9ed000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ed5ffb640 1 -- 172.21.15.43:0/1459276185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0015070 con 0x7f9ee00b4490 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ed5ffb640 1 -- 172.21.15.43:0/1459276185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed0004220 con 0x7f9ee00b4490 2024-03-30T13:14:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee013a5b0 con 0x7f9ee00b4490 2024-03-30T13:14:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ed5ffb640 1 -- 172.21.15.43:0/1459276185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0003620 con 0x7f9ee00b4490 2024-03-30T13:14:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee013f230 con 0x7f9ee00b4490 2024-03-30T13:14:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.881+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ee013aad0 con 0x7f9ee00b4490 2024-03-30T13:14:47.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.885+0000 7f9ed5ffb640 1 -- 172.21.15.43:0/1459276185 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9ed00037c0 con 0x7f9ee00b4490 2024-03-30T13:14:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.885+0000 7f9ed5ffb640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ec4067fc0 0x7f9ec406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.885+0000 7f9ed5ffb640 1 -- 172.21.15.43:0/1459276185 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f9ed0086560 con 0x7f9ee00b4490 2024-03-30T13:14:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.885+0000 7f9ee526c640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ec4067fc0 0x7f9ec406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.885+0000 7f9ee526c640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ec4067fc0 0x7f9ec406a480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0af000 tx=0x7f9edc0a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:47.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:47.889+0000 7f9ed5ffb640 1 -- 172.21.15.43:0/1459276185 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9ee013aad0 con 0x7f9ee00b4490 2024-03-30T13:14:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.077+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ee0068890 con 0x7f9ee00b4490 2024-03-30T13:14:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.077+0000 7f9ed5ffb640 1 -- 172.21.15.43:0/1459276185 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5187 (secure 0 0 0) 0x7f9ed0056b30 con 0x7f9ee00b4490 2024-03-30T13:14:48.084 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:48.085 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4271726137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4271726137},{"type":"v1","addr":"172.21.15.43:6834","nonce":4271726137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1748034672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1748034672},{"type":"v1","addr":"172.21.15.43:6835","nonce":1748034672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3254700273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3254700273},{"type":"v1","addr":"172.21.15.113:6835","nonce":3254700273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-30T13:12:27.356973+0000","modified":"2024-03-30T13:14:35.770003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4880":{"gid":4880,"name":"d","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.113:6837/1104336007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":1104336007},{"type":"v1","addr":"172.21.15.113:6837","nonce":1104336007}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4880,"qdb_cluster":[4880]},"id":2}]} 2024-03-30T13:14:48.085 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-03-30T13:14:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ec4067fc0 msgr2=0x7f9ec406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ec4067fc0 0x7f9ec406a480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0af000 tx=0x7f9edc0a7000 comp rx=0 tx=0).stop 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee00b4490 msgr2=0x7f9ee01f4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee00b4490 0x7f9ee01f4ec0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000c7f0 tx=0x7f9ed000ccb0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 shutdown_connections 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9ec4067fc0 0x7f9ec406a480 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9ee0144270 0x7f9ee013f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ee00b4490 0x7f9ee01f4ec0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 --2- 172.21.15.43:0/1459276185 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9ee00b2ef0 0x7f9ee0140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 >> 172.21.15.43:0/1459276185 conn(0x7f9ee00cfee0 msgr2=0x7f9ee006b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 shutdown_connections 2024-03-30T13:14:48.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.081+0000 7f9ee74d0640 1 -- 172.21.15.43:0/1459276185 wait complete. 2024-03-30T13:14:48.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:14:48.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c8ef40640 1 Processor -- start 2024-03-30T13:14:48.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c8ef40640 1 -- start start 2024-03-30T13:14:48.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c8ef40640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c88166e70 0x7f9c880b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c8ef40640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c880b3890 con 0x7f9c88163370 2024-03-30T13:14:48.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c8ef40640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c88166390 con 0x7f9c8806c670 2024-03-30T13:14:48.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c8ef40640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9c880b3b70 con 0x7f9c88166e70 2024-03-30T13:14:48.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c87fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c88166e70 0x7f9c880b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c87fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c88166e70 0x7f9c880b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44378/0 (socket says 172.21.15.43:44378) 2024-03-30T13:14:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c87fff640 1 -- 172.21.15.43:0/1487254448 learned_addr learned my addr 172.21.15.43:0/1487254448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c87fff640 1 -- 172.21.15.43:0/1487254448 >> v1:172.21.15.113:6789/0 conn(0x7f9c8806c670 legacy=0x7f9c880714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c87fff640 1 -- 172.21.15.43:0/1487254448 >> v1:172.21.15.43:6789/0 conn(0x7f9c88163370 legacy=0x7f9c88163750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c87fff640 1 -- 172.21.15.43:0/1487254448 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c88166670 con 0x7f9c88166e70 2024-03-30T13:14:48.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.209+0000 7f9c877fe640 1 -- 172.21.15.43:0/1487254448 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569430083 0 0) 0x7f9c880b3890 con 0x7f9c88163370 2024-03-30T13:14:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c87fff640 1 --2- 172.21.15.43:0/1487254448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c88166e70 0x7f9c880b2ef0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9c700098a0 tx=0x7f9c7001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2c390f3a00194 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c877fe640 1 -- 172.21.15.43:0/1487254448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700ab650 con 0x7f9c88166e70 2024-03-30T13:14:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c877fe640 1 -- 172.21.15.43:0/1487254448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c700b0070 con 0x7f9c88166e70 2024-03-30T13:14:48.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c877fe640 1 -- 172.21.15.43:0/1487254448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c700aba10 con 0x7f9c88166e70 2024-03-30T13:14:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- 172.21.15.43:0/1487254448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c88166e70 msgr2=0x7f9c880b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/1487254448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c88166e70 0x7f9c880b2ef0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9c700098a0 tx=0x7f9c7001c540 comp rx=0 tx=0).stop 2024-03-30T13:14:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- 172.21.15.43:0/1487254448 shutdown_connections 2024-03-30T13:14:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/1487254448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c88166e70 0x7f9c880b2ef0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- 172.21.15.43:0/1487254448 >> 172.21.15.43:0/1487254448 conn(0x7f9c880cfee0 msgr2=0x7f9c880d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- 172.21.15.43:0/1487254448 shutdown_connections 2024-03-30T13:14:48.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- 172.21.15.43:0/1487254448 wait complete. 2024-03-30T13:14:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 Processor -- start 2024-03-30T13:14:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- start start 2024-03-30T13:14:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8806c670 0x7f9c8815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c88163370 0x7f9c8815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88166e70 0x7f9c8815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9c88071fc0 con 0x7f9c8806c670 2024-03-30T13:14:48.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9c880b3640 con 0x7f9c88166e70 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9c88072140 con 0x7f9c88163370 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8806c670 0x7f9c8815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8806c670 0x7f9c8815c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44390/0 (socket says 172.21.15.43:44390) 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 -- 172.21.15.43:0/2218257104 learned_addr learned my addr 172.21.15.43:0/2218257104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c87fff640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c88163370 0x7f9c8815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 -- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c88163370 msgr2=0x7f9c8815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c88163370 0x7f9c8815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 -- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88166e70 msgr2=0x7f9c8815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8d4dd640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88166e70 0x7f9c8815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88166e70 0x7f9c8815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 -- 172.21.15.43:0/2218257104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8815b370 con 0x7f9c8806c670 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8d4dd640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88166e70 0x7f9c8815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ccdc640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8806c670 0x7f9c8815c410 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c790 tx=0x7f9c8000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c857fa640 1 -- 172.21.15.43:0/2218257104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80003db0 con 0x7f9c8806c670 2024-03-30T13:14:48.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c857fa640 1 -- 172.21.15.43:0/2218257104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c80005070 con 0x7f9c8806c670 2024-03-30T13:14:48.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.213+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8815b650 con 0x7f9c8806c670 2024-03-30T13:14:48.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c857fa640 1 -- 172.21.15.43:0/2218257104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c80005460 con 0x7f9c8806c670 2024-03-30T13:14:48.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8815bc10 con 0x7f9c8806c670 2024-03-30T13:14:48.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c88068890 con 0x7f9c8806c670 2024-03-30T13:14:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c857fa640 1 -- 172.21.15.43:0/2218257104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9c80005600 con 0x7f9c8806c670 2024-03-30T13:14:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c857fa640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067ff0 0x7f9c6806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c857fa640 1 -- 172.21.15.43:0/2218257104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7f9c80086660 con 0x7f9c8806c670 2024-03-30T13:14:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c87fff640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067ff0 0x7f9c6806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.217+0000 7f9c87fff640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067ff0 0x7f9c6806a4b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9c700af000 tx=0x7f9c700a7000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.221+0000 7f9c857fa640 1 -- 172.21.15.43:0/2218257104 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9c80056c20 con 0x7f9c8806c670 2024-03-30T13:14:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.381+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c8806f680 con 0x7f9c8806c670 2024-03-30T13:14:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c857fa640 1 -- 172.21.15.43:0/2218257104 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+17021 (secure 0 0 0) 0x7f9c80056520 con 0x7f9c8806c670 2024-03-30T13:14:48.390 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:48.390 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:14:40.895915+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-03-30T13:12:25.924561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-03-30T13:12:26.302922+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.20","mappings":[{"from":4,"to":2}]},{"pgid":"6.3c","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":20}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067ff0 msgr2=0x7f9c6806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067ff0 0x7f9c6806a4b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9c700af000 tx=0x7f9c700a7000 comp rx=0 tx=0).stop 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8806c670 msgr2=0x7f9c8815c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8806c670 0x7f9c8815c410 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000c790 tx=0x7f9c8000cc50 comp rx=0 tx=0).stop 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 shutdown_connections 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9c68067ff0 0x7f9c6806a4b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9c88166e70 0x7f9c8815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9c88163370 0x7f9c8815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 --2- 172.21.15.43:0/2218257104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8806c670 0x7f9c8815c410 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 >> 172.21.15.43:0/2218257104 conn(0x7f9c880cfee0 msgr2=0x7f9c88151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:48.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 shutdown_connections 2024-03-30T13:14:48.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.385+0000 7f9c8ef40640 1 -- 172.21.15.43:0/2218257104 wait complete. 2024-03-30T13:14:48.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-30T13:14:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb125c0b640 1 Processor -- start 2024-03-30T13:14:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb125c0b640 1 -- start start 2024-03-30T13:14:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb125c0b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb12006c670 0x7fb1200cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb125c0b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200b41f0 0x7fb12006e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb125c0b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb12006e5c0 con 0x7fb12006ca50 2024-03-30T13:14:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb125c0b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb1200b34c0 con 0x7fb12006c670 2024-03-30T13:14:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb125c0b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb1200ccef0 con 0x7fb1200b41f0 2024-03-30T13:14:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11effd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200b41f0 0x7fb12006e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb12006c670 0x7fb1200cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb12006ca50 0x7fb12015fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34094/0 (socket says 172.21.15.43:34094) 2024-03-30T13:14:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11ffff640 1 -- 172.21.15.43:0/2169811105 learned_addr learned my addr 172.21.15.43:0/2169811105 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11f7fe640 1 -- 172.21.15.43:0/2169811105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200b41f0 msgr2=0x7fb12006e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11f7fe640 1 --2- 172.21.15.43:0/2169811105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200b41f0 0x7fb12006e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11f7fe640 1 -- 172.21.15.43:0/2169811105 >> v1:172.21.15.43:6789/0 conn(0x7fb12006ca50 legacy=0x7fb12015fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11e7fc640 1 -- 172.21.15.43:0/2169811105 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 767490042 0 0) 0x7fb12006e5c0 con 0x7fb12006ca50 2024-03-30T13:14:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.525+0000 7fb11f7fe640 1 -- 172.21.15.43:0/2169811105 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1200b3710 con 0x7fb12006c670 2024-03-30T13:14:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb11f7fe640 1 --2- 172.21.15.43:0/2169811105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12006c670 0x7fb1200cc910 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb1100097c0 tx=0x7fb11001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8579e3f3f4e414fd server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb11e7fc640 1 -- 172.21.15.43:0/2169811105 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100b0070 con 0x7fb12006c670 2024-03-30T13:14:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb11e7fc640 1 -- 172.21.15.43:0/2169811105 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1100ab600 con 0x7fb12006c670 2024-03-30T13:14:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb11e7fc640 1 -- 172.21.15.43:0/2169811105 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1100ab9d0 con 0x7fb12006c670 2024-03-30T13:14:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 -- 172.21.15.43:0/2169811105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12006c670 msgr2=0x7fb1200cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 --2- 172.21.15.43:0/2169811105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12006c670 0x7fb1200cc910 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb1100097c0 tx=0x7fb11001c4a0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 -- 172.21.15.43:0/2169811105 shutdown_connections 2024-03-30T13:14:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 --2- 172.21.15.43:0/2169811105 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb1200b41f0 0x7fb12006e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 --2- 172.21.15.43:0/2169811105 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12006c670 0x7fb1200cc910 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 -- 172.21.15.43:0/2169811105 >> 172.21.15.43:0/2169811105 conn(0x7fb1200cfee0 msgr2=0x7fb1200d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:48.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 -- 172.21.15.43:0/2169811105 shutdown_connections 2024-03-30T13:14:48.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 -- 172.21.15.43:0/2169811105 wait complete. 2024-03-30T13:14:48.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 Processor -- start 2024-03-30T13:14:48.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.529+0000 7fb125c0b640 1 -- start start 2024-03-30T13:14:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12006c670 0x7fb120134590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb12006ca50 0x7fb12015b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1200b41f0 0x7fb1201323e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb12006cfe0 con 0x7fb1200b41f0 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb12006ce60 con 0x7fb12006c670 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb120168ef0 con 0x7fb12006ca50 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1200b41f0 0x7fb1201323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1200b41f0 0x7fb1201323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44406/0 (socket says 172.21.15.43:44406) 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 -- 172.21.15.43:0/3561290060 learned_addr learned my addr 172.21.15.43:0/3561290060 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11f7fe640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12006c670 0x7fb120134590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11effd640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb12006ca50 0x7fb12015b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 -- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb12006ca50 msgr2=0x7fb12015b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb12006ca50 0x7fb12015b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 -- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12006c670 msgr2=0x7fb120134590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12006c670 0x7fb120134590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 -- 172.21.15.43:0/3561290060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb12015bc30 con 0x7fb1200b41f0 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11effd640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb12006ca50 0x7fb12015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11ffff640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1200b41f0 0x7fb1201323e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c7b0 tx=0x7fb11800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb107fff640 1 -- 172.21.15.43:0/3561290060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb118018070 con 0x7fb1200b41f0 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb107fff640 1 -- 172.21.15.43:0/3561290060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb11800a640 con 0x7fb1200b41f0 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1201f71c0 con 0x7fb1200b41f0 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb107fff640 1 -- 172.21.15.43:0/3561290060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800aa30 con 0x7fb1200b41f0 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb120134ee0 con 0x7fb1200b41f0 2024-03-30T13:14:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb120068890 con 0x7fb1200b41f0 2024-03-30T13:14:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb107fff640 1 -- 172.21.15.43:0/3561290060 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb11801b070 con 0x7fb1200b41f0 2024-03-30T13:14:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb107fff640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb10c067fc0 0x7fb10c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb107fff640 1 -- 172.21.15.43:0/3561290060 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 6187+0+0 (secure 0 0 0) 0x7fb118085be0 con 0x7fb1200b41f0 2024-03-30T13:14:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.533+0000 7fb11f7fe640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb10c067fc0 0x7fb10c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.537+0000 7fb11f7fe640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb10c067fc0 0x7fb10c06a480 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb1100ad000 tx=0x7fb1100a6000 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.537+0000 7fb107fff640 1 -- 172.21.15.43:0/3561290060 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb118056240 con 0x7fb1200b41f0 2024-03-30T13:14:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.729+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb1200b32a0 con 0x7fb1200b41f0 2024-03-30T13:14:48.737 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-03-30T13:14:48.733+0000 7f98346a4640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-30T13:14:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.733+0000 7fb107fff640 1 -- 172.21.15.43:0/3561290060 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7fb118055b40 con 0x7fb1200b41f0 2024-03-30T13:14:48.740 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-30T13:14:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb10c067fc0 msgr2=0x7fb10c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb10c067fc0 0x7fb10c06a480 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb1100ad000 tx=0x7fb1100a6000 comp rx=0 tx=0).stop 2024-03-30T13:14:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1200b41f0 msgr2=0x7fb1201323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1200b41f0 0x7fb1201323e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb11800c7b0 tx=0x7fb11800cc70 comp rx=0 tx=0).stop 2024-03-30T13:14:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 shutdown_connections 2024-03-30T13:14:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb10c067fc0 0x7fb10c06a480 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1200b41f0 0x7fb1201323e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb12006ca50 0x7fb12015b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 --2- 172.21.15.43:0/3561290060 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb12006c670 0x7fb120134590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 >> 172.21.15.43:0/3561290060 conn(0x7fb1200cfee0 msgr2=0x7fb12015e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 shutdown_connections 2024-03-30T13:14:48.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.737+0000 7fb125c0b640 1 -- 172.21.15.43:0/3561290060 wait complete. 2024-03-30T13:14:48.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-30T13:14:48.825 INFO:tasks.ceph.mds.d.smithi113.stdout:starting mds.d at 2024-03-30T13:14:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.873+0000 7ff666f17640 1 Processor -- start 2024-03-30T13:14:48.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.873+0000 7ff666f17640 1 -- start start 2024-03-30T13:14:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff666f17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660159c90 0x7ff660160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff666f17640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff660070800 0x7ff6600b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff666f17640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff66015c970 con 0x7ff66015d6d0 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff666f17640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7ff66015d240 con 0x7ff660070800 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff666f17640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600ce6b0 con 0x7ff660159c90 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff664cb3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660159c90 0x7ff660160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff6654b4640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7ff660070800 0x7ff6600b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff664cb3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660159c90 0x7ff660160970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:42074/0 (socket says 172.21.15.43:42074) 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff664cb3640 1 -- 172.21.15.43:0/555332248 learned_addr learned my addr 172.21.15.43:0/555332248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff6654b4640 1 -- 172.21.15.43:0/555332248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660159c90 msgr2=0x7ff660160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff6654b4640 1 --2- 172.21.15.43:0/555332248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660159c90 0x7ff660160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff6654b4640 1 -- 172.21.15.43:0/555332248 >> v1:172.21.15.113:6789/0 conn(0x7ff66015d6d0 legacy=0x7ff6600700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff6654b4640 1 -- 172.21.15.43:0/555332248 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6600ccbe0 con 0x7ff660070800 2024-03-30T13:14:48.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff6654b4640 1 --2- 172.21.15.43:0/555332248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660070800 0x7ff6600b6f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff658009d90 tx=0x7ff65801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23331217cf05e8b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff65f7fe640 1 -- 172.21.15.43:0/555332248 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6580b2070 con 0x7ff660070800 2024-03-30T13:14:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff65f7fe640 1 -- 172.21.15.43:0/555332248 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6580a5830 con 0x7ff660070800 2024-03-30T13:14:48.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.877+0000 7ff65f7fe640 1 -- 172.21.15.43:0/555332248 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6580a5c00 con 0x7ff660070800 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- 172.21.15.43:0/555332248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660070800 msgr2=0x7ff6600b6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 --2- 172.21.15.43:0/555332248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660070800 0x7ff6600b6f30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff658009d90 tx=0x7ff65801c970 comp rx=0 tx=0).stop 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- 172.21.15.43:0/555332248 shutdown_connections 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 --2- 172.21.15.43:0/555332248 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff660070800 0x7ff6600b6f30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 --2- 172.21.15.43:0/555332248 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660159c90 0x7ff660160970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- 172.21.15.43:0/555332248 >> 172.21.15.43:0/555332248 conn(0x7ff6600ceae0 msgr2=0x7ff6600ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- 172.21.15.43:0/555332248 shutdown_connections 2024-03-30T13:14:48.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- 172.21.15.43:0/555332248 wait complete. 2024-03-30T13:14:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 Processor -- start 2024-03-30T13:14:48.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- start start 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660070800 0x7ff6600ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660159c90 0x7ff6600cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff66015d6d0 0x7ff660156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600d02f0 con 0x7ff660159c90 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7ff6600d0170 con 0x7ff66015d6d0 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7ff6600d0470 con 0x7ff660070800 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660159c90 0x7ff6600cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660159c90 0x7ff6600cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44410/0 (socket says 172.21.15.43:44410) 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 -- 172.21.15.43:0/1281741911 learned_addr learned my addr 172.21.15.43:0/1281741911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff664cb3640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660070800 0x7ff6600ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff6654b4640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff66015d6d0 0x7ff660156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 -- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660070800 msgr2=0x7ff6600ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660070800 0x7ff6600ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 -- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff66015d6d0 msgr2=0x7ff660156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff66015d6d0 0x7ff660156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 -- 172.21.15.43:0/1281741911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6600b17e0 con 0x7ff660159c90 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff6654b4640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff66015d6d0 0x7ff660156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65ffff640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660159c90 0x7ff6600cb690 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c420 tx=0x7ff65400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65d7fa640 1 -- 172.21.15.43:0/1281741911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6540052d0 con 0x7ff660159c90 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65d7fa640 1 -- 172.21.15.43:0/1281741911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff654005470 con 0x7ff660159c90 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff65d7fa640 1 -- 172.21.15.43:0/1281741911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654007260 con 0x7ff660159c90 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.881+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6600b5c90 con 0x7ff660159c90 2024-03-30T13:14:48.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.885+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6600b6530 con 0x7ff660159c90 2024-03-30T13:14:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.885+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff660068900 con 0x7ff660159c90 2024-03-30T13:14:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.885+0000 7ff65d7fa640 1 -- 172.21.15.43:0/1281741911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7ff654007400 con 0x7ff660159c90 2024-03-30T13:14:48.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.885+0000 7ff65d7fa640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff64c067fc0 0x7ff64c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.885+0000 7ff65d7fa640 1 -- 172.21.15.43:0/1281741911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6245+0+0 (secure 0 0 0) 0x7ff654085bf0 con 0x7ff660159c90 2024-03-30T13:14:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.885+0000 7ff664cb3640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff64c067fc0 0x7ff64c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:48.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.885+0000 7ff664cb3640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff64c067fc0 0x7ff64c06a480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff650009ae0 tx=0x7ff650008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:48.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:48.889+0000 7ff65d7fa640 1 -- 172.21.15.43:0/1281741911 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7ff66015c970 con 0x7ff660159c90 2024-03-30T13:14:49.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.093+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff6600d1c50 con 0x7ff660159c90 2024-03-30T13:14:49.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.737+0000 7ff65d7fa640 1 -- 172.21.15.43:0/1281741911 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7ff654056210 con 0x7ff660159c90 2024-03-30T13:14:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff64c067fc0 msgr2=0x7ff64c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff64c067fc0 0x7ff64c06a480 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff650009ae0 tx=0x7ff650008040 comp rx=0 tx=0).stop 2024-03-30T13:14:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660159c90 msgr2=0x7ff6600cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660159c90 0x7ff6600cb690 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7ff65400c420 tx=0x7ff65400c8e0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 shutdown_connections 2024-03-30T13:14:49.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7ff64c067fc0 0x7ff64c06a480 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7ff66015d6d0 0x7ff660156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff660159c90 0x7ff6600cb690 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 --2- 172.21.15.43:0/1281741911 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7ff660070800 0x7ff6600ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 >> 172.21.15.43:0/1281741911 conn(0x7ff6600ceae0 msgr2=0x7ff6600b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 shutdown_connections 2024-03-30T13:14:49.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.741+0000 7ff666f17640 1 -- 172.21.15.43:0/1281741911 wait complete. 2024-03-30T13:14:49.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-30T13:14:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9ff21d3640 1 Processor -- start 2024-03-30T13:14:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9ff21d3640 1 -- start start 2024-03-30T13:14:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9ff21d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15bd50 0x7f9fec06e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9ff21d3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fec0cefa0 0x7f9fec0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9ff21d3640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fec0b3680 con 0x7f9fec15c130 2024-03-30T13:14:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9ff21d3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9fec0cfd70 con 0x7f9fec0cefa0 2024-03-30T13:14:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9ff21d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec0d0050 con 0x7f9fec15bd50 2024-03-30T13:14:49.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9febfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fec0cefa0 0x7f9fec0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9febfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fec0cefa0 0x7f9fec0b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44416/0 (socket says 172.21.15.43:44416) 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9febfff640 1 -- 172.21.15.43:0/1633889693 learned_addr learned my addr 172.21.15.43:0/1633889693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9feb7fe640 1 --2- 172.21.15.43:0/1633889693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15bd50 0x7f9fec06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9febfff640 1 -- 172.21.15.43:0/1633889693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15bd50 msgr2=0x7f9fec06e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9febfff640 1 --2- 172.21.15.43:0/1633889693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15bd50 0x7f9fec06e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9febfff640 1 -- 172.21.15.43:0/1633889693 >> v1:172.21.15.113:6789/0 conn(0x7f9fec15c130 legacy=0x7f9fec0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9febfff640 1 -- 172.21.15.43:0/1633889693 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec0b3960 con 0x7f9fec0cefa0 2024-03-30T13:14:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.901+0000 7f9feb7fe640 1 --2- 172.21.15.43:0/1633889693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15bd50 0x7f9fec06e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9febfff640 1 --2- 172.21.15.43:0/1633889693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec0cefa0 0x7f9fec0b1b60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009870 tx=0x7f9fe401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9814f9c6b1e6424 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9fea7fc640 1 -- 172.21.15.43:0/1633889693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40aa650 con 0x7f9fec0cefa0 2024-03-30T13:14:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9fea7fc640 1 -- 172.21.15.43:0/1633889693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe40af070 con 0x7f9fec0cefa0 2024-03-30T13:14:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9fea7fc640 1 -- 172.21.15.43:0/1633889693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40aaa10 con 0x7f9fec0cefa0 2024-03-30T13:14:49.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1633889693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec0cefa0 msgr2=0x7f9fec0b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1633889693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec0cefa0 0x7f9fec0b1b60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009870 tx=0x7f9fe401c4d0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1633889693 shutdown_connections 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1633889693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec0cefa0 0x7f9fec0b1b60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1633889693 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15bd50 0x7f9fec06e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1633889693 >> 172.21.15.43:0/1633889693 conn(0x7f9fec071b80 msgr2=0x7f9fec0b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1633889693 shutdown_connections 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1633889693 wait complete. 2024-03-30T13:14:49.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 Processor -- start 2024-03-30T13:14:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.905+0000 7f9ff21d3640 1 -- start start 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fec0cefa0 0x7f9fec15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec15bd50 0x7f9fec152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15c130 0x7f9fec159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0ce340 con 0x7f9fec15bd50 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec0ce1c0 con 0x7f9fec0cefa0 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec0ce4c0 con 0x7f9fec15c130 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec15bd50 0x7f9fec152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec15bd50 0x7f9fec152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44420/0 (socket says 172.21.15.43:44420) 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 -- 172.21.15.43:0/1285806664 learned_addr learned my addr 172.21.15.43:0/1285806664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feb7fe640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fec0cefa0 0x7f9fec15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9febfff640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15c130 0x7f9fec159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 -- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15c130 msgr2=0x7f9fec159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15c130 0x7f9fec159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 -- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fec0cefa0 msgr2=0x7f9fec15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fec0cefa0 0x7f9fec15d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 -- 172.21.15.43:0/1285806664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec0c77a0 con 0x7f9fec15bd50 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9febfff640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15c130 0x7f9fec159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feaffd640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec15bd50 0x7f9fec152ef0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000c500 tx=0x7f9fe000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9fd3fff640 1 -- 172.21.15.43:0/1285806664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0016020 con 0x7f9fec15bd50 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9fd3fff640 1 -- 172.21.15.43:0/1285806664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe0005220 con 0x7f9fec15bd50 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fec06f970 con 0x7f9fec15bd50 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9fd3fff640 1 -- 172.21.15.43:0/1285806664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0007030 con 0x7f9fec15bd50 2024-03-30T13:14:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fec0c5cf0 con 0x7f9fec15bd50 2024-03-30T13:14:49.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fec0687f0 con 0x7f9fec15bd50 2024-03-30T13:14:49.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9fd3fff640 1 -- 172.21.15.43:0/1285806664 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f9fe00071f0 con 0x7f9fec15bd50 2024-03-30T13:14:49.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9fd3fff640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9fd8067fc0 0x7f9fd806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:49.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9fd3fff640 1 -- 172.21.15.43:0/1285806664 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 6245+0+0 (secure 0 0 0) 0x7f9fe0086230 con 0x7f9fec15bd50 2024-03-30T13:14:49.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.909+0000 7f9feb7fe640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9fd8067fc0 0x7f9fd806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:49.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.913+0000 7f9feb7fe640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9fd8067fc0 0x7f9fd806a480 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009ed0 tx=0x7f9fdc00b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:49.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:49.913+0000 7f9fd3fff640 1 -- 172.21.15.43:0/1285806664 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f9fe0020020 con 0x7f9fec15bd50 2024-03-30T13:14:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:50.177+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9fec0b32c0 con 0x7f9fec15bd50 2024-03-30T13:14:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.157+0000 7f9fd3fff640 1 -- 172.21.15.43:0/1285806664 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7f9fe0056730 con 0x7f9fec15bd50 2024-03-30T13:14:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9fd8067fc0 msgr2=0x7f9fd806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9fd8067fc0 0x7f9fd806a480 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009ed0 tx=0x7f9fdc00b040 comp rx=0 tx=0).stop 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec15bd50 msgr2=0x7f9fec152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec15bd50 0x7f9fec152ef0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000c500 tx=0x7f9fe000c9c0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 shutdown_connections 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9fd8067fc0 0x7f9fd806a480 secure :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009ed0 tx=0x7f9fdc00b040 comp rx=0 tx=0).stop 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9fec15c130 0x7f9fec159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fec15bd50 0x7f9fec152ef0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 --2- 172.21.15.43:0/1285806664 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f9fec0cefa0 0x7f9fec15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 >> 172.21.15.43:0/1285806664 conn(0x7f9fec071b80 msgr2=0x7f9fec06cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:51.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.161+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 shutdown_connections 2024-03-30T13:14:51.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.165+0000 7f9ff21d3640 1 -- 172.21.15.43:0/1285806664 wait complete. 2024-03-30T13:14:51.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-30T13:14:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d9643640 1 Processor -- start 2024-03-30T13:14:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d9643640 1 -- start start 2024-03-30T13:14:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d9643640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60d4145cf0 0x7f60d4070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d9643640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f60d40700f0 0x7f60d4165d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d9643640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d40cfac0 con 0x7f60d4070c60 2024-03-30T13:14:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d9643640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f60d406c670 con 0x7f60d4145cf0 2024-03-30T13:14:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d9643640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f60d40cfd60 con 0x7f60d40700f0 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d27fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f60d4070c60 0x7f60d406f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36820/0 (socket says 172.21.15.43:36820) 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d27fc640 1 -- 172.21.15.43:0/3349093025 learned_addr learned my addr 172.21.15.43:0/3349093025 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/3349093025 >> v2:172.21.15.43:3300/0 conn(0x7f60d4145cf0 0x7f60d4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d37fe640 1 --2- 172.21.15.43:0/3349093025 >> v2:172.21.15.113:3300/0 conn(0x7f60d40700f0 0x7f60d4165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d1ffb640 1 -- 172.21.15.43:0/3349093025 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359561193 0 0) 0x7f60d40cfac0 con 0x7f60d4070c60 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d1ffb640 1 -- 172.21.15.43:0/3349093025 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60c0003530 con 0x7f60d4070c60 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d2ffd640 1 -- 172.21.15.43:0/3349093025 >> v2:172.21.15.113:3300/0 conn(0x7f60d40700f0 msgr2=0x7f60d4165d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/3349093025 >> v2:172.21.15.113:3300/0 conn(0x7f60d40700f0 0x7f60d4165d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d2ffd640 1 -- 172.21.15.43:0/3349093025 >> v1:172.21.15.43:6789/0 conn(0x7f60d4070c60 legacy=0x7f60d406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.325+0000 7f60d2ffd640 1 -- 172.21.15.43:0/3349093025 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d40d0040 con 0x7f60d4145cf0 2024-03-30T13:14:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/3349093025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4145cf0 0x7f60d4070720 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f60d415c700 tx=0x7f60c401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce8ac67eb7b6d5e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d1ffb640 1 -- 172.21.15.43:0/3349093025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40b2070 con 0x7f60d4145cf0 2024-03-30T13:14:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d1ffb640 1 -- 172.21.15.43:0/3349093025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60c40a5930 con 0x7f60d4145cf0 2024-03-30T13:14:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d1ffb640 1 -- 172.21.15.43:0/3349093025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60c40a5ce0 con 0x7f60d4145cf0 2024-03-30T13:14:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- 172.21.15.43:0/3349093025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4145cf0 msgr2=0x7f60d4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 --2- 172.21.15.43:0/3349093025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4145cf0 0x7f60d4070720 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f60d415c700 tx=0x7f60c401c870 comp rx=0 tx=0).stop 2024-03-30T13:14:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- 172.21.15.43:0/3349093025 shutdown_connections 2024-03-30T13:14:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 --2- 172.21.15.43:0/3349093025 >> v2:172.21.15.113:3300/0 conn(0x7f60d40700f0 0x7f60d4165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 --2- 172.21.15.43:0/3349093025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4145cf0 0x7f60d4070720 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- 172.21.15.43:0/3349093025 >> 172.21.15.43:0/3349093025 conn(0x7f60d40b2ce0 msgr2=0x7f60d40b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:51.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- 172.21.15.43:0/3349093025 shutdown_connections 2024-03-30T13:14:51.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- 172.21.15.43:0/3349093025 wait complete. 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 Processor -- start 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- start start 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d40700f0 0x7f60d40b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4070c60 0x7f60d41575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d4145cf0 0x7f60d4157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f60d415d670 con 0x7f60d4070c60 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f60d415d4f0 con 0x7f60d40700f0 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d9643640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f60d4168a80 con 0x7f60d4145cf0 2024-03-30T13:14:51.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4070c60 0x7f60d41575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4070c60 0x7f60d41575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36638/0 (socket says 172.21.15.43:36638) 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 -- 172.21.15.43:0/1417790728 learned_addr learned my addr 172.21.15.43:0/1417790728 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d40700f0 0x7f60d40b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d37fe640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d4145cf0 0x7f60d4157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 -- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d4145cf0 msgr2=0x7f60d4157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d4145cf0 0x7f60d4157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 -- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d40700f0 msgr2=0x7f60d40b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d40700f0 0x7f60d40b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d27fc640 1 -- 172.21.15.43:0/1417790728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d40b12f0 con 0x7f60d4070c60 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.329+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d40700f0 0x7f60d40b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60d27fc640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4070c60 0x7f60d41575e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c460 tx=0x7f60bc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60cb7fe640 1 -- 172.21.15.43:0/1417790728 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc017070 con 0x7f60d4070c60 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60cb7fe640 1 -- 172.21.15.43:0/1417790728 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc005d10 con 0x7f60d4070c60 2024-03-30T13:14:51.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d40b15d0 con 0x7f60d4070c60 2024-03-30T13:14:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60cb7fe640 1 -- 172.21.15.43:0/1417790728 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc0072b0 con 0x7f60d4070c60 2024-03-30T13:14:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d40b1b90 con 0x7f60d4070c60 2024-03-30T13:14:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60d40b28d0 con 0x7f60d4070c60 2024-03-30T13:14:51.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60cb7fe640 1 -- 172.21.15.43:0/1417790728 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f60bc01a070 con 0x7f60d4070c60 2024-03-30T13:14:51.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60cb7fe640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f60b4067fc0 0x7f60b406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:51.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60cb7fe640 1 -- 172.21.15.43:0/1417790728 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5807+0+0 (secure 0 0 0) 0x7f60bc085850 con 0x7f60d4070c60 2024-03-30T13:14:51.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f60b4067fc0 0x7f60b406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:51.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.333+0000 7f60d2ffd640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f60b4067fc0 0x7f60b406a480 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f60c40af000 tx=0x7f60c40a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:51.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.337+0000 7f60cb7fe640 1 -- 172.21.15.43:0/1417790728 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f60bc056030 con 0x7f60d4070c60 2024-03-30T13:14:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:51.497+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f60d415e5a0 con 0x7f60d4070c60 2024-03-30T13:14:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.165+0000 7f60cb7fe640 1 -- 172.21.15.43:0/1417790728 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7f60bc055930 con 0x7f60d4070c60 2024-03-30T13:14:52.170 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-03-30T13:14:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.169+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f60b4067fc0 msgr2=0x7f60b406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.169+0000 7f60d9643640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f60b4067fc0 0x7f60b406a480 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f60c40af000 tx=0x7f60c40a7070 comp rx=0 tx=0).stop 2024-03-30T13:14:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.169+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4070c60 msgr2=0x7f60d41575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.169+0000 7f60d9643640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4070c60 0x7f60d41575e0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f60bc00c460 tx=0x7f60bc00c920 comp rx=0 tx=0).stop 2024-03-30T13:14:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 shutdown_connections 2024-03-30T13:14:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f60b4067fc0 0x7f60b406a480 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f60d4145cf0 0x7f60d4157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60d4070c60 0x7f60d41575e0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 --2- 172.21.15.43:0/1417790728 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f60d40700f0 0x7f60d40b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 >> 172.21.15.43:0/1417790728 conn(0x7f60d40b2ce0 msgr2=0x7f60d415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 shutdown_connections 2024-03-30T13:14:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.173+0000 7f60d9643640 1 -- 172.21.15.43:0/1417790728 wait complete. 2024-03-30T13:14:52.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-30T13:14:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.321+0000 7f2b7442e640 1 Processor -- start 2024-03-30T13:14:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b7442e640 1 -- start start 2024-03-30T13:14:52.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b7442e640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2b6c070800 0x7f2b6c160970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b7442e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6c15c970 con 0x7f2b6c159c90 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b7442e640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6c0ccbe0 con 0x7f2b6c15d6d0 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b7442e640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2b6c15d240 con 0x7f2b6c070800 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2b6c159c90 0x7f2b6c0700f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36828/0 (socket says 172.21.15.43:36828) 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 -- 172.21.15.43:0/1025374622 learned_addr learned my addr 172.21.15.43:0/1025374622 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2800460711 0 0) 0x7f2b68001120 con 0x7f2b6c159c90 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b58003530 con 0x7f2b6c159c90 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b719c9640 1 --2- 172.21.15.43:0/1025374622 >> v2:172.21.15.113:3300/0 conn(0x7f2b6c070800 0x7f2b6c160970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2633326445 0 0) 0x7f2b58003530 con 0x7f2b6c159c90 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 >> v2:172.21.15.113:3300/0 conn(0x7f2b6c070800 msgr2=0x7f2b6c160970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 --2- 172.21.15.43:0/1025374622 >> v2:172.21.15.113:3300/0 conn(0x7f2b6c070800 0x7f2b6c160970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 >> v1:172.21.15.113:6789/0 conn(0x7f2b6c15d6d0 legacy=0x7f2b6c0b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c15cbf0 con 0x7f2b6c159c90 2024-03-30T13:14:52.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f2b68001850 con 0x7f2b6c159c90 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 >> v1:172.21.15.43:6789/0 conn(0x7f2b6c159c90 legacy=0x7f2b6c0700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b580039c0 0x7f2b58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b58004810 0x7f2b58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580051d0 0x7f2b5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c15d240 con 0x7f2b580039c0 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5800c0a0 con 0x7f2b58004810 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5800c2a0 con 0x7f2b580051d0 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b580039c0 0x7f2b58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b719c9640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b58004810 0x7f2b58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 -- 172.21.15.43:0/1025374622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580051d0 msgr2=0x7f2b5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580051d0 0x7f2b5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 -- 172.21.15.43:0/1025374622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b58004810 msgr2=0x7f2b58004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b58004810 0x7f2b58004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b721ca640 1 -- 172.21.15.43:0/1025374622 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b58003530 con 0x7f2b580039c0 2024-03-30T13:14:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.325+0000 7f2b729cb640 1 -- 172.21.15.43:0/1025374622 reap_dead start 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b721ca640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b580039c0 0x7f2b58003e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c4c0 tx=0x7f2b6801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed006802b75b1d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6800c0e0 con 0x7f2b580039c0 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b680a9020 con 0x7f2b580039c0 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b711c8640 1 -- 172.21.15.43:0/1025374622 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68005b00 con 0x7f2b580039c0 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- 172.21.15.43:0/1025374622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b580039c0 msgr2=0x7f2b58003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b580039c0 0x7f2b58003e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c4c0 tx=0x7f2b6801cc60 comp rx=0 tx=0).stop 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- 172.21.15.43:0/1025374622 shutdown_connections 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 --2- 172.21.15.43:0/1025374622 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b580039c0 0x7f2b58003e00 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- 172.21.15.43:0/1025374622 >> 172.21.15.43:0/1025374622 conn(0x7f2b6c0ceae0 msgr2=0x7f2b6c0ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- 172.21.15.43:0/1025374622 shutdown_connections 2024-03-30T13:14:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- 172.21.15.43:0/1025374622 wait complete. 2024-03-30T13:14:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 Processor -- start 2024-03-30T13:14:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- start start 2024-03-30T13:14:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580039c0 0x7f2b6c0c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6c158970 0x7f2b6c148450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6540 0x7f2b6c0c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c160d00 con 0x7f2b6c158970 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6c160b80 con 0x7f2b6c0c6540 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b7442e640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6c160e80 con 0x7f2b580039c0 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b719c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6c158970 0x7f2b6c148450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b719c9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6c158970 0x7f2b6c148450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36662/0 (socket says 172.21.15.43:36662) 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b719c9640 1 -- 172.21.15.43:0/152186127 learned_addr learned my addr 172.21.15.43:0/152186127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b721ca640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580039c0 0x7f2b6c0c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b729cb640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6540 0x7f2b6c0c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.329+0000 7f2b719c9640 1 -- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580039c0 msgr2=0x7f2b6c0c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b719c9640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580039c0 0x7f2b6c0c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b719c9640 1 -- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6540 msgr2=0x7f2b6c0c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b719c9640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6540 0x7f2b6c0c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b719c9640 1 -- 172.21.15.43:0/152186127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6c158df0 con 0x7f2b6c158970 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b719c9640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6c158970 0x7f2b6c148450 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c011560 tx=0x7f2b5c011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b62ffd640 1 -- 172.21.15.43:0/152186127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c015870 con 0x7f2b6c158970 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b62ffd640 1 -- 172.21.15.43:0/152186127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b5c015a10 con 0x7f2b6c158970 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b6c148990 con 0x7f2b6c158970 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b62ffd640 1 -- 172.21.15.43:0/152186127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c015dc0 con 0x7f2b6c158970 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b6c159350 con 0x7f2b6c158970 2024-03-30T13:14:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6c068900 con 0x7f2b6c158970 2024-03-30T13:14:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b62ffd640 1 -- 172.21.15.43:0/152186127 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2b5c00f460 con 0x7f2b6c158970 2024-03-30T13:14:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b62ffd640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b50067fc0 0x7f2b5006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b62ffd640 1 -- 172.21.15.43:0/152186127 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f2b5c086b60 con 0x7f2b6c158970 2024-03-30T13:14:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b721ca640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b50067fc0 0x7f2b5006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.333+0000 7f2b721ca640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b50067fc0 0x7f2b5006a480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c4c0 tx=0x7f2b6c0ccdb0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.337+0000 7f2b62ffd640 1 -- 172.21.15.43:0/152186127 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f2b6c158df0 con 0x7f2b6c158970 2024-03-30T13:14:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.493+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f2b6c0b6ce0 con 0x7f2b6c158970 2024-03-30T13:14:52.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b62ffd640 1 -- 172.21.15.43:0/152186127 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f2b5c019030 con 0x7f2b6c158970 2024-03-30T13:14:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b50067fc0 msgr2=0x7f2b5006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b50067fc0 0x7f2b5006a480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2b6800c4c0 tx=0x7f2b6c0ccdb0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6c158970 msgr2=0x7f2b6c148450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6c158970 0x7f2b6c148450 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c011560 tx=0x7f2b5c011a20 comp rx=0 tx=0).stop 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 shutdown_connections 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2b50067fc0 0x7f2b5006a480 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2b6c0c6540 0x7f2b6c0c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b6c158970 0x7f2b6c148450 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 --2- 172.21.15.43:0/152186127 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2b580039c0 0x7f2b6c0c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 >> 172.21.15.43:0/152186127 conn(0x7f2b6c0ceae0 msgr2=0x7f2b6c06e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 shutdown_connections 2024-03-30T13:14:52.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.501+0000 7f2b7442e640 1 -- 172.21.15.43:0/152186127 wait complete. 2024-03-30T13:14:52.523 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining' 2024-03-30T13:14:52.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962f0e6640 1 Processor -- start 2024-03-30T13:14:52.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962f0e6640 1 -- start start 2024-03-30T13:14:52.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962f0e6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9628162b90 0x7f962806e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962f0e6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f962806ec50 0x7f962806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962f0e6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9628072070 con 0x7f9628147e10 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962f0e6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f9628071e50 con 0x7f9628162b90 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962f0e6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f962806e280 con 0x7f962806ec50 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f9627fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9628162b90 0x7f962806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f9627fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9628162b90 0x7f962806e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36668/0 (socket says 172.21.15.43:36668) 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f9627fff640 1 -- 172.21.15.43:0/2630569254 learned_addr learned my addr 172.21.15.43:0/2630569254 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962d683640 1 --2- 172.21.15.43:0/2630569254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f962806ec50 0x7f962806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f9627fff640 1 -- 172.21.15.43:0/2630569254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f962806ec50 msgr2=0x7f962806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f9627fff640 1 --2- 172.21.15.43:0/2630569254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f962806ec50 0x7f962806c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f9627fff640 1 -- 172.21.15.43:0/2630569254 >> v1:172.21.15.113:6789/0 conn(0x7f9628147e10 legacy=0x7f9628071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f9627fff640 1 -- 172.21.15.43:0/2630569254 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962806dc30 con 0x7f9628162b90 2024-03-30T13:14:52.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.633+0000 7f962d683640 1 --2- 172.21.15.43:0/2630569254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f962806ec50 0x7f962806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:52.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 --2- 172.21.15.43:0/2630569254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628162b90 0x7f962806e710 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f961c009870 tx=0x7f961c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f444645b9b2e66e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:52.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f96277fe640 1 -- 172.21.15.43:0/2630569254 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0b0070 con 0x7f9628162b90 2024-03-30T13:14:52.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f96277fe640 1 -- 172.21.15.43:0/2630569254 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961c0ab600 con 0x7f9628162b90 2024-03-30T13:14:52.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f96277fe640 1 -- 172.21.15.43:0/2630569254 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0ab9d0 con 0x7f9628162b90 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- 172.21.15.43:0/2630569254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628162b90 msgr2=0x7f962806e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 --2- 172.21.15.43:0/2630569254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628162b90 0x7f962806e710 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f961c009870 tx=0x7f961c01c490 comp rx=0 tx=0).stop 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- 172.21.15.43:0/2630569254 shutdown_connections 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 --2- 172.21.15.43:0/2630569254 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f962806ec50 0x7f962806c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 --2- 172.21.15.43:0/2630569254 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628162b90 0x7f962806e710 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- 172.21.15.43:0/2630569254 >> 172.21.15.43:0/2630569254 conn(0x7f96280cfee0 msgr2=0x7f96280d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- 172.21.15.43:0/2630569254 shutdown_connections 2024-03-30T13:14:52.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- 172.21.15.43:0/2630569254 wait complete. 2024-03-30T13:14:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 Processor -- start 2024-03-30T13:14:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- start start 2024-03-30T13:14:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806ec50 0x7f96281412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628147e10 0x7f96281674c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9628162b90 0x7f962815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f962816b460 con 0x7f9628147e10 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f962816b2e0 con 0x7f962806ec50 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f962816b5e0 con 0x7f9628162b90 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628147e10 0x7f96281674c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628147e10 0x7f96281674c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36676/0 (socket says 172.21.15.43:36676) 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 -- 172.21.15.43:0/839533750 learned_addr learned my addr 172.21.15.43:0/839533750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962d683640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9628162b90 0x7f962815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962ce82640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806ec50 0x7f96281412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 -- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9628162b90 msgr2=0x7f962815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9628162b90 0x7f962815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 -- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806ec50 msgr2=0x7f96281412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806ec50 0x7f96281412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:52.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 -- 172.21.15.43:0/839533750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96281634e0 con 0x7f9628147e10 2024-03-30T13:14:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f9627fff640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628147e10 0x7f96281674c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f961c0ae040 tx=0x7f961c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f96257fa640 1 -- 172.21.15.43:0/839533750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0ba020 con 0x7f9628147e10 2024-03-30T13:14:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f96257fa640 1 -- 172.21.15.43:0/839533750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f961c0b6660 con 0x7f9628147e10 2024-03-30T13:14:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96281637c0 con 0x7f9628147e10 2024-03-30T13:14:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f96257fa640 1 -- 172.21.15.43:0/839533750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961c0b0040 con 0x7f9628147e10 2024-03-30T13:14:52.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.637+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9628163d80 con 0x7f9628147e10 2024-03-30T13:14:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.641+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9628068890 con 0x7f9628147e10 2024-03-30T13:14:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.641+0000 7f96257fa640 1 -- 172.21.15.43:0/839533750 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f961c0bd020 con 0x7f9628147e10 2024-03-30T13:14:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.641+0000 7f96257fa640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9614067fc0 0x7f961406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.641+0000 7f96257fa640 1 -- 172.21.15.43:0/839533750 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f961c120270 con 0x7f9628147e10 2024-03-30T13:14:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.641+0000 7f962ce82640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9614067fc0 0x7f961406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:52.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.641+0000 7f962ce82640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9614067fc0 0x7f961406a480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9618004e30 tx=0x7f9618009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.645+0000 7f96257fa640 1 -- 172.21.15.43:0/839533750 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f961c0c4070 con 0x7f9628147e10 2024-03-30T13:14:52.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:52.869+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]} v 0) -- 0x7f96280b3a50 con 0x7f9628147e10 2024-03-30T13:14:53.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.165+0000 7f96257fa640 1 -- 172.21.15.43:0/839533750 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_uninlining.TestDataUninlining.test_data_uninlining"]}]=0 v200) ==== 147+0+0 (secure 0 0 0) 0x7f961c016110 con 0x7f9628147e10 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9614067fc0 msgr2=0x7f961406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9614067fc0 0x7f961406a480 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9618004e30 tx=0x7f9618009040 comp rx=0 tx=0).stop 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628147e10 msgr2=0x7f96281674c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628147e10 0x7f96281674c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f961c0ae040 tx=0x7f961c019230 comp rx=0 tx=0).stop 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 shutdown_connections 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f9614067fc0 0x7f961406a480 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f9628162b90 0x7f962815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9628147e10 0x7f96281674c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:53.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 --2- 172.21.15.43:0/839533750 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f962806ec50 0x7f96281412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 >> 172.21.15.43:0/839533750 conn(0x7f96280cfee0 msgr2=0x7f96280b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 shutdown_connections 2024-03-30T13:14:53.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:53.169+0000 7f962f0e6640 1 -- 172.21.15.43:0/839533750 wait complete. 2024-03-30T13:14:53.199 INFO:tasks.cephfs_test_runner:test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) ... ERROR 2024-03-30T13:14:53.199 INFO:tasks.cephfs_test_runner: 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 219, in run_test_worker 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: self.fs.mds_tell(['cache', 'drop'], i) 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1313, in mds_tell 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: return self.rank_tell(*args, **kwargs) 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-30T13:14:53.200 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-30T13:14:53.201 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner:Ran 1 test in 180.994s 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner: 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-30T13:14:53.202 INFO:tasks.cephfs_test_runner:ERROR: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 322, in test_data_uninlining 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: self.run_test_worker(opt_clear_cache, 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/test_uninlining.py", line 219, in run_test_worker 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: self.fs.mds_tell(['cache', 'drop'], i) 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1313, in mds_tell 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: return self.rank_tell(*args, **kwargs) 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1332, in rank_tell 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: return self.mds_tell(*args, mds_id=f"{self.id}:{rank}", **kwargs) 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs/filesystem.py", line 1319, in mds_tell 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: out = self.get_ceph_cmd_stdout("tell", f"mds.{mds_id}", *args, **kwargs) 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 55, in get_ceph_cmd_stdout 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: return self.run_ceph_cmd(**kwargs).stdout.getvalue() 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_test_case.py", line 30, in run_ceph_cmd 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: return self.mon_manager.run_cluster_cmd(**kwargs) 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/ceph_manager.py", line 1647, in run_cluster_cmd 2024-03-30T13:14:53.203 INFO:tasks.cephfs_test_runner: return self.controller.run(**kwargs) 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 447, in run 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: r = RemoteProcess(client, args, check_status=check_status, hostname=name, 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 65, in __init__ 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: self.command = quote(args) 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 257, in quote 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: return ' '.join(_quote(args)) 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 255, in _quote 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: yield pipes.quote(a) 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: File "/usr/lib/python3.8/shlex.py", line 325, in quote 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: if _find_unsafe(s) is None: 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner:TypeError: expected string or bytes-like object 2024-03-30T13:14:53.204 INFO:tasks.cephfs_test_runner: 2024-03-30T13:14:53.205 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:14:53.442 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=33bf6c56bd6b4b00811a359c831529f9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ac1710396f9f7c389fa4b407711bc5157a8c0bf7/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining) 2024-03-30T13:14:53.443 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-30T13:14:53.452 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-30T13:14:53.461 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-30T13:14:53.461 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:14:53.461 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:53.476 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22574 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T13:14:53.476 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-30T13:14:53.476 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:53.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=22574 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-30T13:14:53.522 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:14:53.522 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:53.532 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18812 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T13:14:53.533 DEBUG:teuthology.orchestra.run.smithi113:> set -ex 2024-03-30T13:14:53.533 DEBUG:teuthology.orchestra.run.smithi113:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-30T13:14:53.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18812 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274684k,nr_inodes=818671,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274688k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-30T13:14:53.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:53.582 DEBUG:teuthology.orchestra.run.smithi113:> ip netns list 2024-03-30T13:14:53.633 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-30T13:14:53.634 INFO:teuthology.orchestra.run.smithi113.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-30T13:14:53.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:53.634 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-30T13:14:53.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:53.691 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-30T13:14:53.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:53.757 DEBUG:teuthology.orchestra.run.smithi113:> sudo ip link delete ceph-brx 2024-03-30T13:14:53.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:53.922 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-03-30T13:14:53.929 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-30T13:14:53.929 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-30T13:14:53.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:53.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-30T13:14:53.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:53.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-30T13:14:54.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-30T13:14:54.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-03-30T13:14:54.226 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-30T13:14:54.236 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-30T13:14:54.236 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:14:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.357+0000 7f62a172b640 1 Processor -- start 2024-03-30T13:14:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.357+0000 7f62a172b640 1 -- start start 2024-03-30T13:14:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f62a172b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f629c15e600 0x7f629c0b9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f62a172b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f629c0d3e60 con 0x7f629c0d1100 2024-03-30T13:14:54.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f62a172b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f629c0b7680 con 0x7f629c0d1560 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f62a172b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f629c0b7450 con 0x7f629c15e600 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f629c0d1100 0x7f629c0b8160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36832/0 (socket says 172.21.15.43:36832) 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629a7fc640 1 -- 172.21.15.43:0/1805360864 learned_addr learned my addr 172.21.15.43:0/1805360864 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 500855152 0 0) 0x7f629c0d3e60 con 0x7f629c0d1100 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629affd640 1 --2- 172.21.15.43:0/1805360864 >> v2:172.21.15.113:3300/0 conn(0x7f629c15e600 0x7f629c0b9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6284003530 con 0x7f629c0d1100 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3336200829 0 0) 0x7f6284003530 con 0x7f629c0d1100 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 >> v2:172.21.15.113:3300/0 conn(0x7f629c15e600 msgr2=0x7f629c0b9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:54.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 --2- 172.21.15.43:0/1805360864 >> v2:172.21.15.113:3300/0 conn(0x7f629c15e600 0x7f629c0b9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 >> v1:172.21.15.113:6789/0 conn(0x7f629c0d1560 legacy=0x7f629c0d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c0b7960 con 0x7f629c0d1100 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f6290001680 con 0x7f629c0d1100 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 >> v1:172.21.15.43:6789/0 conn(0x7f629c0d1100 legacy=0x7f629c0b8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f6284003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6284004810 0x7f6284004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62840051d0 0x7f628400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f628400c040 con 0x7f62840039c0 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f628400c1f0 con 0x7f6284004810 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f628400c3f0 con 0x7f62840051d0 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629affd640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f6284003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629a7fc640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6284004810 0x7f6284004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629affd640 1 -- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62840051d0 msgr2=0x7f628400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629affd640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f62840051d0 0x7f628400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629affd640 1 -- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6284004810 msgr2=0x7f6284004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629affd640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6284004810 0x7f6284004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629affd640 1 -- 172.21.15.43:0/1805360864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6284003530 con 0x7f62840039c0 2024-03-30T13:14:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629b7fe640 1 -- 172.21.15.43:0/1805360864 reap_dead start 2024-03-30T13:14:54.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.361+0000 7f629a7fc640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6284004810 0x7f6284004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:54.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f6284003de0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f628c009fa0 tx=0x7f628c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb62bf04924e4862 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0b2070 con 0x7f62840039c0 2024-03-30T13:14:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628c0a5b30 con 0x7f62840039c0 2024-03-30T13:14:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f6299ffb640 1 -- 172.21.15.43:0/1805360864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c009bf0 con 0x7f62840039c0 2024-03-30T13:14:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- 172.21.15.43:0/1805360864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 msgr2=0x7f6284003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f6284003de0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f628c009fa0 tx=0x7f628c01cc80 comp rx=0 tx=0).stop 2024-03-30T13:14:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- 172.21.15.43:0/1805360864 shutdown_connections 2024-03-30T13:14:54.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 --2- 172.21.15.43:0/1805360864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f6284003de0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- 172.21.15.43:0/1805360864 >> 172.21.15.43:0/1805360864 conn(0x7f629c0b6230 msgr2=0x7f629c0b6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- 172.21.15.43:0/1805360864 shutdown_connections 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- 172.21.15.43:0/1805360864 wait complete. 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 Processor -- start 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- start start 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f629c160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f629c159fa0 0x7f629c15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f629c0cdc60 0x7f629c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f629c0b8d00 con 0x7f62840039c0 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f629c0b8b80 con 0x7f629c0cdc60 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62a172b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f629c0b8e80 con 0x7f629c159fa0 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f629c160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f629c160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36690/0 (socket says 172.21.15.43:36690) 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 -- 172.21.15.43:0/2215744238 learned_addr learned my addr 172.21.15.43:0/2215744238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629a7fc640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f629c159fa0 0x7f629c15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629b7fe640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f629c0cdc60 0x7f629c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 -- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f629c159fa0 msgr2=0x7f629c15a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f629c159fa0 0x7f629c15a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 -- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f629c0cdc60 msgr2=0x7f629c0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f629c0cdc60 0x7f629c0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 -- 172.21.15.43:0/2215744238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c15a960 con 0x7f62840039c0 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f629affd640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f629c160590 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b0040 tx=0x7f6284005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62837fe640 1 -- 172.21.15.43:0/2215744238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0be070 con 0x7f62840039c0 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62837fe640 1 -- 172.21.15.43:0/2215744238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628c0b3040 con 0x7f62840039c0 2024-03-30T13:14:54.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.365+0000 7f62837fe640 1 -- 172.21.15.43:0/2215744238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0b2050 con 0x7f62840039c0 2024-03-30T13:14:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c0cb590 con 0x7f62840039c0 2024-03-30T13:14:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c0cbb50 con 0x7f62840039c0 2024-03-30T13:14:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629c068890 con 0x7f62840039c0 2024-03-30T13:14:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f62837fe640 1 -- 172.21.15.43:0/2215744238 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f628c0c1070 con 0x7f62840039c0 2024-03-30T13:14:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f62837fe640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6288067fc0 0x7f628806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f62837fe640 1 -- 172.21.15.43:0/2215744238 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f628c120aa0 con 0x7f62840039c0 2024-03-30T13:14:54.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f629a7fc640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6288067fc0 0x7f628806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.369+0000 7f629a7fc640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6288067fc0 0x7f628806a480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f629c0cc860 tx=0x7f62900014e0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:54.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.373+0000 7f62837fe640 1 -- 172.21.15.43:0/2215744238 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f628c016990 con 0x7f62840039c0 2024-03-30T13:14:54.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.533+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f629c161ca0 con 0x7f6288067fc0 2024-03-30T13:14:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.533+0000 7f62837fe640 1 -- 172.21.15.43:0/2215744238 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13548 (secure 0 0 0) 0x7f629c161ca0 con 0x7f6288067fc0 2024-03-30T13:14:54.540 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:14:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6288067fc0 msgr2=0x7f628806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6288067fc0 0x7f628806a480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f629c0cc860 tx=0x7f62900014e0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 msgr2=0x7f629c160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f629c160590 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f628c0b0040 tx=0x7f6284005840 comp rx=0 tx=0).stop 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 shutdown_connections 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6288067fc0 0x7f628806a480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f629c0cdc60 0x7f629c0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f629c159fa0 0x7f629c15a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 --2- 172.21.15.43:0/2215744238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62840039c0 0x7f629c160590 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 >> 172.21.15.43:0/2215744238 conn(0x7f629c0b6230 msgr2=0x7f629c0d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 shutdown_connections 2024-03-30T13:14:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.537+0000 7f62a172b640 1 -- 172.21.15.43:0/2215744238 wait complete. 2024-03-30T13:14:54.559 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-03-30T13:14:53.919146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":515856,"kb_used_data":249748,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754458864,"statfs":{"total":773094113280,"available":772565876736,"internally_reserved":0,"allocated":255741952,"data_stored":253067709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":272354866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-176821680,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-109,"num_read_kb":-22,"num_write":-908,"num_write_kb":-175457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001845"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-30T13:14:52.172907+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:14:52.172907+0000","last_peered":"2024-03-30T13:14:52.172907+0000","last_clean":"2024-03-30T13:14:52.172907+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:14:52.172907+0000","last_undegraded":"2024-03-30T13:14:52.172907+0000","last_fullsized":"2024-03-30T13:14:52.172907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48464,"kb_used_data":17284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323376,"statfs":{"total":96636764160,"available":96587137024,"internally_reserved":0,"allocated":17698816,"data_stored":17363785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69536,"kb_used_data":38352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302304,"statfs":{"total":96636764160,"available":96565559296,"internally_reserved":0,"allocated":39272448,"data_stored":38937353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97616,"kb_used_data":66432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94274224,"statfs":{"total":96636764160,"available":96536805376,"internally_reserved":0,"allocated":68026368,"data_stored":67675863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35908,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335932,"statfs":{"total":96636764160,"available":96599994368,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":37528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":38428672,"data_stored":38099173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1167360,"data_stored":841743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89772,"kb_used_data":54428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282068,"statfs":{"total":96636764160,"available":96544837632,"internally_reserved":0,"allocated":55734272,"data_stored":55386506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65204,"kb_used_data":34024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306636,"statfs":{"total":96636764160,"available":96569995264,"internally_reserved":0,"allocated":34840576,"data_stored":34511911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:14:54.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:14:54.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9ee183640 1 Processor -- start 2024-03-30T13:14:54.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9ee183640 1 -- start start 2024-03-30T13:14:54.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9ee183640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9e8160a90 0x7fe9e815fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9ee183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e81602a0 0x7fe9e80b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9ee183640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9e80d03f0 con 0x7fe9e8145e10 2024-03-30T13:14:54.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9ee183640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe9e81606d0 con 0x7fe9e8160a90 2024-03-30T13:14:54.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9ee183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e80d06c0 con 0x7fe9e81602a0 2024-03-30T13:14:54.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9e6ffd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe9e8160a90 0x7fe9e815fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e81602a0 0x7fe9e80b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9e7fff640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e81602a0 0x7fe9e80b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:41968/0 (socket says 172.21.15.43:41968) 2024-03-30T13:14:54.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.677+0000 7fe9e7fff640 1 -- 172.21.15.43:0/3279480070 learned_addr learned my addr 172.21.15.43:0/3279480070 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e7fff640 1 -- 172.21.15.43:0/3279480070 >> v2:172.21.15.113:3300/0 conn(0x7fe9e8160a90 msgr2=0x7fe9e815fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e7fff640 1 --2- 172.21.15.43:0/3279480070 >> v2:172.21.15.113:3300/0 conn(0x7fe9e8160a90 0x7fe9e815fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e7fff640 1 -- 172.21.15.43:0/3279480070 >> v1:172.21.15.113:6789/0 conn(0x7fe9e8145e10 legacy=0x7fe9e80cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e67fc640 1 -- 172.21.15.43:0/3279480070 <== mon.1 v1:172.21.15.113:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1574695868 0 0) 0x7fe9e80d03f0 con 0x7fe9e8145e10 2024-03-30T13:14:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e7fff640 1 -- 172.21.15.43:0/3279480070 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e80d09a0 con 0x7fe9e81602a0 2024-03-30T13:14:54.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e7fff640 1 --2- 172.21.15.43:0/3279480070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e81602a0 0x7fe9e80b1810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009870 tx=0x7fe9e001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ad44edd05f81dc server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:54.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e67fc640 1 -- 172.21.15.43:0/3279480070 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e00ab650 con 0x7fe9e81602a0 2024-03-30T13:14:54.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e67fc640 1 -- 172.21.15.43:0/3279480070 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9e00b0070 con 0x7fe9e81602a0 2024-03-30T13:14:54.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9e67fc640 1 -- 172.21.15.43:0/3279480070 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e00aba10 con 0x7fe9e81602a0 2024-03-30T13:14:54.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 -- 172.21.15.43:0/3279480070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e81602a0 msgr2=0x7fe9e80b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 --2- 172.21.15.43:0/3279480070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e81602a0 0x7fe9e80b1810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009870 tx=0x7fe9e001c560 comp rx=0 tx=0).stop 2024-03-30T13:14:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 -- 172.21.15.43:0/3279480070 shutdown_connections 2024-03-30T13:14:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 --2- 172.21.15.43:0/3279480070 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e81602a0 0x7fe9e80b1810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 --2- 172.21.15.43:0/3279480070 >> v2:172.21.15.113:3300/0 conn(0x7fe9e8160a90 0x7fe9e815fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 -- 172.21.15.43:0/3279480070 >> 172.21.15.43:0/3279480070 conn(0x7fe9e80ceb90 msgr2=0x7fe9e80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 -- 172.21.15.43:0/3279480070 shutdown_connections 2024-03-30T13:14:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.681+0000 7fe9ee183640 1 -- 172.21.15.43:0/3279480070 wait complete. 2024-03-30T13:14:54.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 Processor -- start 2024-03-30T13:14:54.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 -- start start 2024-03-30T13:14:54.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e8145e10 0x7fe9e81618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9e81602a0 0x7fe9e815e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e8160a90 0x7fe9e80b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e80b26e0 con 0x7fe9e81602a0 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e80b2560 con 0x7fe9e8160a90 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe9e80b2860 con 0x7fe9e8145e10 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9e81602a0 0x7fe9e815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9e81602a0 0x7fe9e815e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36704/0 (socket says 172.21.15.43:36704) 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 -- 172.21.15.43:0/2890653898 learned_addr learned my addr 172.21.15.43:0/2890653898 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e77fe640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e8145e10 0x7fe9e81618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e7fff640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e8160a90 0x7fe9e80b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 -- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e8145e10 msgr2=0x7fe9e81618a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e8145e10 0x7fe9e81618a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 -- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e8160a90 msgr2=0x7fe9e80b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e8160a90 0x7fe9e80b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 -- 172.21.15.43:0/2890653898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e8161fb0 con 0x7fe9e81602a0 2024-03-30T13:14:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e7fff640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e8160a90 0x7fe9e80b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9e6ffd640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9e81602a0 0x7fe9e815e210 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000e8a0 tx=0x7fe9d000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9dffff640 1 -- 172.21.15.43:0/2890653898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0017070 con 0x7fe9e81602a0 2024-03-30T13:14:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9dffff640 1 -- 172.21.15.43:0/2890653898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d000c940 con 0x7fe9e81602a0 2024-03-30T13:14:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e8162290 con 0x7fe9e81602a0 2024-03-30T13:14:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9dffff640 1 -- 172.21.15.43:0/2890653898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d000ccf0 con 0x7fe9e81602a0 2024-03-30T13:14:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e8162850 con 0x7fe9e81602a0 2024-03-30T13:14:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.685+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e8068890 con 0x7fe9e81602a0 2024-03-30T13:14:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.689+0000 7fe9dffff640 1 -- 172.21.15.43:0/2890653898 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe9d001a070 con 0x7fe9e81602a0 2024-03-30T13:14:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.689+0000 7fe9dffff640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe9c8067fc0 0x7fe9c806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.689+0000 7fe9dffff640 1 -- 172.21.15.43:0/2890653898 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7fe9d0086650 con 0x7fe9e81602a0 2024-03-30T13:14:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.689+0000 7fe9e77fe640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe9c8067fc0 0x7fe9c806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.689+0000 7fe9e77fe640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe9c8067fc0 0x7fe9c806a480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8007fd0 tx=0x7fe9d8009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.693+0000 7fe9dffff640 1 -- 172.21.15.43:0/2890653898 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe9d0056f70 con 0x7fe9e81602a0 2024-03-30T13:14:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.849+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9e8162d50 con 0x7fe9c8067fc0 2024-03-30T13:14:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.849+0000 7fe9dffff640 1 -- 172.21.15.43:0/2890653898 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13548 (secure 0 0 0) 0x7fe9d8003980 con 0x7fe9c8067fc0 2024-03-30T13:14:54.856 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:14:54.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe9c8067fc0 msgr2=0x7fe9c806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe9c8067fc0 0x7fe9c806a480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8007fd0 tx=0x7fe9d8009040 comp rx=0 tx=0).stop 2024-03-30T13:14:54.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9e81602a0 msgr2=0x7fe9e815e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:54.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9e81602a0 0x7fe9e815e210 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000e8a0 tx=0x7fe9d000eca0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 shutdown_connections 2024-03-30T13:14:54.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe9c8067fc0 0x7fe9c806a480 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe9e8160a90 0x7fe9e80b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe9e81602a0 0x7fe9e815e210 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 --2- 172.21.15.43:0/2890653898 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe9e8145e10 0x7fe9e81618a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:54.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 >> 172.21.15.43:0/2890653898 conn(0x7fe9e80ceb90 msgr2=0x7fe9e80b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:54.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 shutdown_connections 2024-03-30T13:14:54.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.853+0000 7fe9ee183640 1 -- 172.21.15.43:0/2890653898 wait complete. 2024-03-30T13:14:54.873 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-03-30T13:14:53.919146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":515856,"kb_used_data":249748,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754458864,"statfs":{"total":773094113280,"available":772565876736,"internally_reserved":0,"allocated":255741952,"data_stored":253067709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":272354866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-176821680,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-109,"num_read_kb":-22,"num_write":-908,"num_write_kb":-175457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001845"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-30T13:14:52.172907+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:14:52.172907+0000","last_peered":"2024-03-30T13:14:52.172907+0000","last_clean":"2024-03-30T13:14:52.172907+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:14:52.172907+0000","last_undegraded":"2024-03-30T13:14:52.172907+0000","last_fullsized":"2024-03-30T13:14:52.172907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48464,"kb_used_data":17284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323376,"statfs":{"total":96636764160,"available":96587137024,"internally_reserved":0,"allocated":17698816,"data_stored":17363785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69536,"kb_used_data":38352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302304,"statfs":{"total":96636764160,"available":96565559296,"internally_reserved":0,"allocated":39272448,"data_stored":38937353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97616,"kb_used_data":66432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94274224,"statfs":{"total":96636764160,"available":96536805376,"internally_reserved":0,"allocated":68026368,"data_stored":67675863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35908,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335932,"statfs":{"total":96636764160,"available":96599994368,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":37528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":38428672,"data_stored":38099173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1167360,"data_stored":841743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89772,"kb_used_data":54428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282068,"statfs":{"total":96636764160,"available":96544837632,"internally_reserved":0,"allocated":55734272,"data_stored":55386506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65204,"kb_used_data":34024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306636,"statfs":{"total":96636764160,"available":96569995264,"internally_reserved":0,"allocated":34840576,"data_stored":34511911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:14:54.874 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-30T13:14:54.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:14:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.993+0000 7fbc60086640 1 Processor -- start 2024-03-30T13:14:55.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.993+0000 7fbc60086640 1 -- start start 2024-03-30T13:14:55.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fbc5806c670 0x7fbc580cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5806ca50 0x7fbc580ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc580ccef0 con 0x7fbc580b41f0 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fbc580b34c0 con 0x7fbc5806ca50 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fbc580b3710 con 0x7fbc5806c670 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5e623640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5806ca50 0x7fbc580ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5d621640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbc580b41f0 0x7fbc5806e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36834/0 (socket says 172.21.15.43:36834) 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5d621640 1 -- 172.21.15.43:0/3288477641 learned_addr learned my addr 172.21.15.43:0/3288477641 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5de22640 1 --2- 172.21.15.43:0/3288477641 >> v2:172.21.15.113:3300/0 conn(0x7fbc5806c670 0x7fbc580cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5e623640 1 -- 172.21.15.43:0/3288477641 >> v2:172.21.15.113:3300/0 conn(0x7fbc5806c670 msgr2=0x7fbc580cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5e623640 1 --2- 172.21.15.43:0/3288477641 >> v2:172.21.15.113:3300/0 conn(0x7fbc5806c670 0x7fbc580cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5e623640 1 -- 172.21.15.43:0/3288477641 >> v1:172.21.15.43:6789/0 conn(0x7fbc580b41f0 legacy=0x7fbc5806e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5ce20640 1 -- 172.21.15.43:0/3288477641 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1148127659 0 0) 0x7fbc580ccef0 con 0x7fbc580b41f0 2024-03-30T13:14:55.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5e623640 1 -- 172.21.15.43:0/3288477641 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc580b39f0 con 0x7fbc5806ca50 2024-03-30T13:14:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5e623640 1 --2- 172.21.15.43:0/3288477641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806ca50 0x7fbc580ce5b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbc5806ea10 tx=0x7fbc5001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de530f88c4f90d24 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5ce20640 1 -- 172.21.15.43:0/3288477641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500b2070 con 0x7fbc5806ca50 2024-03-30T13:14:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5ce20640 1 -- 172.21.15.43:0/3288477641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc500a5900 con 0x7fbc5806ca50 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc5ce20640 1 -- 172.21.15.43:0/3288477641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc500a5cd0 con 0x7fbc5806ca50 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 -- 172.21.15.43:0/3288477641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806ca50 msgr2=0x7fbc580ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 --2- 172.21.15.43:0/3288477641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806ca50 0x7fbc580ce5b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fbc5806ea10 tx=0x7fbc5001c930 comp rx=0 tx=0).stop 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 -- 172.21.15.43:0/3288477641 shutdown_connections 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 --2- 172.21.15.43:0/3288477641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806ca50 0x7fbc580ce5b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 --2- 172.21.15.43:0/3288477641 >> v2:172.21.15.113:3300/0 conn(0x7fbc5806c670 0x7fbc580cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:54.997+0000 7fbc60086640 1 -- 172.21.15.43:0/3288477641 >> 172.21.15.43:0/3288477641 conn(0x7fbc580cfee0 msgr2=0x7fbc580d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- 172.21.15.43:0/3288477641 shutdown_connections 2024-03-30T13:14:55.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- 172.21.15.43:0/3288477641 wait complete. 2024-03-30T13:14:55.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 Processor -- start 2024-03-30T13:14:55.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- start start 2024-03-30T13:14:55.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806c670 0x7fbc5813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc5806ca50 0x7fbc5813b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc580b41f0 0x7fbc5813b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58169070 con 0x7fbc5806c670 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fbc58168ef0 con 0x7fbc580b41f0 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fbc581691f0 con 0x7fbc5806ca50 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806c670 0x7fbc5813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806c670 0x7fbc5813aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36718/0 (socket says 172.21.15.43:36718) 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 -- 172.21.15.43:0/3637945401 learned_addr learned my addr 172.21.15.43:0/3637945401 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5e623640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc580b41f0 0x7fbc5813b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5d621640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc5806ca50 0x7fbc5813b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 -- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc5806ca50 msgr2=0x7fbc5813b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc5806ca50 0x7fbc5813b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 -- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc580b41f0 msgr2=0x7fbc5813b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc580b41f0 0x7fbc5813b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 -- 172.21.15.43:0/3637945401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc581415f0 con 0x7fbc5806c670 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5e623640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc580b41f0 0x7fbc5813b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc5de22640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806c670 0x7fbc5813aa70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00d5f0 tx=0x7fbc4c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc4a7fc640 1 -- 172.21.15.43:0/3637945401 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0125f0 con 0x7fbc5806c670 2024-03-30T13:14:55.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc4a7fc640 1 -- 172.21.15.43:0/3637945401 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc4c019070 con 0x7fbc5806c670 2024-03-30T13:14:55.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5813cfc0 con 0x7fbc5806c670 2024-03-30T13:14:55.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc4a7fc640 1 -- 172.21.15.43:0/3637945401 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c0129b0 con 0x7fbc5806c670 2024-03-30T13:14:55.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.001+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc58156fe0 con 0x7fbc5806c670 2024-03-30T13:14:55.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.005+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc58068890 con 0x7fbc5806c670 2024-03-30T13:14:55.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.005+0000 7fbc4a7fc640 1 -- 172.21.15.43:0/3637945401 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fbc4c012b50 con 0x7fbc5806c670 2024-03-30T13:14:55.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.005+0000 7fbc4a7fc640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc3c067fc0 0x7fbc3c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.005+0000 7fbc4a7fc640 1 -- 172.21.15.43:0/3637945401 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7fbc4c0867d0 con 0x7fbc5806c670 2024-03-30T13:14:55.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.005+0000 7fbc5d621640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc3c067fc0 0x7fbc3c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.005+0000 7fbc5d621640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc3c067fc0 0x7fbc3c06a480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc54007fd0 tx=0x7fbc54007ba0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:55.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.009+0000 7fbc4a7fc640 1 -- 172.21.15.43:0/3637945401 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fbc4c0570f0 con 0x7fbc5806c670 2024-03-30T13:14:55.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.169+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbc58156ad0 con 0x7fbc3c067fc0 2024-03-30T13:14:55.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.169+0000 7fbc4a7fc640 1 -- 172.21.15.43:0/3637945401 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13548 (secure 0 0 0) 0x7fbc58156ad0 con 0x7fbc3c067fc0 2024-03-30T13:14:55.178 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:55.179 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:14:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc3c067fc0 msgr2=0x7fbc3c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc3c067fc0 0x7fbc3c06a480 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbc54007fd0 tx=0x7fbc54007ba0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806c670 msgr2=0x7fbc5813aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806c670 0x7fbc5813aa70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00d5f0 tx=0x7fbc4c00dab0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 shutdown_connections 2024-03-30T13:14:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fbc3c067fc0 0x7fbc3c06a480 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fbc580b41f0 0x7fbc5813b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fbc5806ca50 0x7fbc5813b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 --2- 172.21.15.43:0/3637945401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5806c670 0x7fbc5813aa70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 >> 172.21.15.43:0/3637945401 conn(0x7fbc580cfee0 msgr2=0x7fbc58142c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 shutdown_connections 2024-03-30T13:14:55.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.173+0000 7fbc60086640 1 -- 172.21.15.43:0/3637945401 wait complete. 2024-03-30T13:14:55.196 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-03-30T13:14:53.919146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":515856,"kb_used_data":249748,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754458864,"statfs":{"total":773094113280,"available":772565876736,"internally_reserved":0,"allocated":255741952,"data_stored":253067709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":272354866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-176821680,"num_objects":-116,"num_object_clones":0,"num_object_copies":-232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-116,"num_whiteouts":0,"num_read":-109,"num_read_kb":-22,"num_write":-908,"num_write_kb":-175457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001845"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":222,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-30T13:14:52.172907+0000","last_change":"2024-03-30T13:10:45.554026+0000","last_active":"2024-03-30T13:14:52.172907+0000","last_peered":"2024-03-30T13:14:52.172907+0000","last_clean":"2024-03-30T13:14:52.172907+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:14:52.172907+0000","last_undegraded":"2024-03-30T13:14:52.172907+0000","last_fullsized":"2024-03-30T13:14:52.172907+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-30T13:10:44.533797+0000","last_clean_scrub_stamp":"2024-03-30T13:10:44.533797+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T23:16:26.966262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509493,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48464,"kb_used_data":17284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323376,"statfs":{"total":96636764160,"available":96587137024,"internally_reserved":0,"allocated":17698816,"data_stored":17363785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69536,"kb_used_data":38352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302304,"statfs":{"total":96636764160,"available":96565559296,"internally_reserved":0,"allocated":39272448,"data_stored":38937353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97616,"kb_used_data":66432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94274224,"statfs":{"total":96636764160,"available":96536805376,"internally_reserved":0,"allocated":68026368,"data_stored":67675863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509494,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35908,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335932,"statfs":{"total":96636764160,"available":96599994368,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72872,"kb_used_data":37528,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94298968,"statfs":{"total":96636764160,"available":96562143232,"internally_reserved":0,"allocated":38428672,"data_stored":38099173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509493,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1167360,"data_stored":841743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":89772,"kb_used_data":54428,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94282068,"statfs":{"total":96636764160,"available":96544837632,"internally_reserved":0,"allocated":55734272,"data_stored":55386506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65204,"kb_used_data":34024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306636,"statfs":{"total":96636764160,"available":96569995264,"internally_reserved":0,"allocated":34840576,"data_stored":34511911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:14:55.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-30T13:14:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0422d31640 1 Processor -- start 2024-03-30T13:14:55.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0422d31640 1 -- start start 2024-03-30T13:14:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0422d31640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f041c144270 0x7f041c144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0422d31640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c15f7a0 0x7f041c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0422d31640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f041c0b4ba0 con 0x7f041c0b2ef0 2024-03-30T13:14:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0422d31640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f041c144fa0 con 0x7f041c144270 2024-03-30T13:14:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0422d31640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f041c0b33f0 con 0x7f041c15f7a0 2024-03-30T13:14:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f04212ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c15f7a0 0x7f041c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f0420acd640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f041c144270 0x7f041c144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f04212ce640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c15f7a0 0x7f041c06c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:41978/0 (socket says 172.21.15.43:41978) 2024-03-30T13:14:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f04212ce640 1 -- 172.21.15.43:0/2445074619 learned_addr learned my addr 172.21.15.43:0/2445074619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f04212ce640 1 -- 172.21.15.43:0/2445074619 >> v2:172.21.15.113:3300/0 conn(0x7f041c144270 msgr2=0x7f041c144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f04212ce640 1 --2- 172.21.15.43:0/2445074619 >> v2:172.21.15.113:3300/0 conn(0x7f041c144270 0x7f041c144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f04212ce640 1 -- 172.21.15.43:0/2445074619 >> v1:172.21.15.113:6789/0 conn(0x7f041c0b2ef0 legacy=0x7f041c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.313+0000 7f04212ce640 1 -- 172.21.15.43:0/2445074619 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c0b36d0 con 0x7f041c15f7a0 2024-03-30T13:14:55.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f04212ce640 1 --2- 172.21.15.43:0/2445074619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c15f7a0 0x7f041c06c670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0414009e50 tx=0x7f041401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=616d76282d1c4bbe server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f041b7fe640 1 -- 172.21.15.43:0/2445074619 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140a57e0 con 0x7f041c15f7a0 2024-03-30T13:14:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f041b7fe640 1 -- 172.21.15.43:0/2445074619 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04140a5980 con 0x7f041c15f7a0 2024-03-30T13:14:55.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f041b7fe640 1 -- 172.21.15.43:0/2445074619 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04140a5d50 con 0x7f041c15f7a0 2024-03-30T13:14:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 -- 172.21.15.43:0/2445074619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c15f7a0 msgr2=0x7f041c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 --2- 172.21.15.43:0/2445074619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c15f7a0 0x7f041c06c670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0414009e50 tx=0x7f041401ca00 comp rx=0 tx=0).stop 2024-03-30T13:14:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 -- 172.21.15.43:0/2445074619 shutdown_connections 2024-03-30T13:14:55.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 --2- 172.21.15.43:0/2445074619 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c15f7a0 0x7f041c06c670 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 --2- 172.21.15.43:0/2445074619 >> v2:172.21.15.113:3300/0 conn(0x7f041c144270 0x7f041c144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 -- 172.21.15.43:0/2445074619 >> 172.21.15.43:0/2445074619 conn(0x7f041c0cfee0 msgr2=0x7f041c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 -- 172.21.15.43:0/2445074619 shutdown_connections 2024-03-30T13:14:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 -- 172.21.15.43:0/2445074619 wait complete. 2024-03-30T13:14:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 Processor -- start 2024-03-30T13:14:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.317+0000 7f0422d31640 1 -- start start 2024-03-30T13:14:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b2ef0 0x7f041c155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c144270 0x7f041c13d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15f7a0 0x7f041c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f041c0cdd10 con 0x7f041c144270 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f041c15fc30 con 0x7f041c15f7a0 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f041c0cde90 con 0x7f041c0b2ef0 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c144270 0x7f041c13d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c144270 0x7f041c13d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36732/0 (socket says 172.21.15.43:36732) 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 -- 172.21.15.43:0/457691571 learned_addr learned my addr 172.21.15.43:0/457691571 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0420acd640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b2ef0 0x7f041c155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f04212ce640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15f7a0 0x7f041c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 -- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b2ef0 msgr2=0x7f041c155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b2ef0 0x7f041c155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 -- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15f7a0 msgr2=0x7f041c152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15f7a0 0x7f041c152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 -- 172.21.15.43:0/457691571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c13e220 con 0x7f041c144270 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f041bfff640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c144270 0x7f041c13d100 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f041000c370 tx=0x7f041000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f04197fa640 1 -- 172.21.15.43:0/457691571 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100052d0 con 0x7f041c144270 2024-03-30T13:14:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f04197fa640 1 -- 172.21.15.43:0/457691571 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0410005470 con 0x7f041c144270 2024-03-30T13:14:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c156150 con 0x7f041c144270 2024-03-30T13:14:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f04197fa640 1 -- 172.21.15.43:0/457691571 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410007260 con 0x7f041c144270 2024-03-30T13:14:55.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c151680 con 0x7f041c144270 2024-03-30T13:14:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041c068890 con 0x7f041c144270 2024-03-30T13:14:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f04197fa640 1 -- 172.21.15.43:0/457691571 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0410007400 con 0x7f041c144270 2024-03-30T13:14:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f04197fa640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0408067fc0 0x7f040806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:55.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f04197fa640 1 -- 172.21.15.43:0/457691571 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f04100857f0 con 0x7f041c144270 2024-03-30T13:14:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.321+0000 7f0420acd640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0408067fc0 0x7f040806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.325+0000 7f0420acd640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0408067fc0 0x7f040806a480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f040c009af0 tx=0x7f040c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.325+0000 7f04197fa640 1 -- 172.21.15.43:0/457691571 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f0410056110 con 0x7f041c144270 2024-03-30T13:14:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.489+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f041c142890 con 0x7f041c144270 2024-03-30T13:14:55.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.489+0000 7f04197fa640 1 -- 172.21.15.43:0/457691571 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+13271 (secure 0 0 0) 0x7f0410055a10 con 0x7f041c144270 2024-03-30T13:14:55.495 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:14:55.496 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":49,"fsid":"ace86ea7-ebbf-4c39-b9cf-d22b6fea8142","created":"2024-03-30T13:10:31.141958+0000","modified":"2024-03-30T13:14:52.159870+0000","last_up_change":"2024-03-30T13:10:42.168260+0000","last_in_change":"2024-03-30T13:10:33.478764+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-30T13:10:43.880356+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1e4c298-2105-4de8-a9a5-4b8e41a07a47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6825","nonce":3443768212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6827","nonce":3443768212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6831","nonce":3443768212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":3443768212},{"type":"v1","addr":"172.21.15.43:6829","nonce":3443768212}]},"public_addr":"172.21.15.43:6825/3443768212","cluster_addr":"172.21.15.43:6827/3443768212","heartbeat_back_addr":"172.21.15.43:6831/3443768212","heartbeat_front_addr":"172.21.15.43:6829/3443768212","state":["exists","up"]},{"osd":1,"uuid":"46a54234-af69-46ba-8c50-436769f3e1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6817","nonce":2193216203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6819","nonce":2193216203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6823","nonce":2193216203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2193216203},{"type":"v1","addr":"172.21.15.43:6821","nonce":2193216203}]},"public_addr":"172.21.15.43:6817/2193216203","cluster_addr":"172.21.15.43:6819/2193216203","heartbeat_back_addr":"172.21.15.43:6823/2193216203","heartbeat_front_addr":"172.21.15.43:6821/2193216203","state":["exists","up"]},{"osd":2,"uuid":"aa1ce7b0-6694-4d32-aba7-8cdb92c40906","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6802","nonce":3862342839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6805","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6807","nonce":3862342839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6815","nonce":3862342839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":3862342839},{"type":"v1","addr":"172.21.15.43:6812","nonce":3862342839}]},"public_addr":"172.21.15.43:6802/3862342839","cluster_addr":"172.21.15.43:6807/3862342839","heartbeat_back_addr":"172.21.15.43:6815/3862342839","heartbeat_front_addr":"172.21.15.43:6812/3862342839","state":["exists","up"]},{"osd":3,"uuid":"517f3d3e-31e4-45dc-a6a7-d2f7d779f73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6801","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6803","nonce":367000764}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6806","nonce":367000764}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6811","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6813","nonce":367000764}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":367000764},{"type":"v1","addr":"172.21.15.43:6809","nonce":367000764}]},"public_addr":"172.21.15.43:6803/367000764","cluster_addr":"172.21.15.43:6806/367000764","heartbeat_back_addr":"172.21.15.43:6813/367000764","heartbeat_front_addr":"172.21.15.43:6809/367000764","state":["exists","up"]},{"osd":4,"uuid":"985ea672-67df-4c19-b4f6-fba3aced4f30","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6824","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6825","nonce":840052469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6826","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6827","nonce":840052469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6830","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6831","nonce":840052469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6828","nonce":840052469},{"type":"v1","addr":"172.21.15.113:6829","nonce":840052469}]},"public_addr":"172.21.15.113:6825/840052469","cluster_addr":"172.21.15.113:6827/840052469","heartbeat_back_addr":"172.21.15.113:6831/840052469","heartbeat_front_addr":"172.21.15.113:6829/840052469","state":["exists","up"]},{"osd":5,"uuid":"56e8bbbc-9c57-402b-8b46-0eec1ec86b2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6816","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6817","nonce":2302616840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6818","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6819","nonce":2302616840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6822","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6823","nonce":2302616840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6820","nonce":2302616840},{"type":"v1","addr":"172.21.15.113:6821","nonce":2302616840}]},"public_addr":"172.21.15.113:6817/2302616840","cluster_addr":"172.21.15.113:6819/2302616840","heartbeat_back_addr":"172.21.15.113:6823/2302616840","heartbeat_front_addr":"172.21.15.113:6821/2302616840","state":["exists","up"]},{"osd":6,"uuid":"a4fe9bf0-75fb-421e-8bee-1ab4bd63a4ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6800","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6801","nonce":594900508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6802","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6803","nonce":594900508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6806","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6807","nonce":594900508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6804","nonce":594900508},{"type":"v1","addr":"172.21.15.113:6805","nonce":594900508}]},"public_addr":"172.21.15.113:6801/594900508","cluster_addr":"172.21.15.113:6803/594900508","heartbeat_back_addr":"172.21.15.113:6807/594900508","heartbeat_front_addr":"172.21.15.113:6805/594900508","state":["exists","up"]},{"osd":7,"uuid":"33b78f88-19b5-4778-9760-3137db597d09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6808","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6809","nonce":3435220755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6810","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6811","nonce":3435220755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6814","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6815","nonce":3435220755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6812","nonce":3435220755},{"type":"v1","addr":"172.21.15.113:6813","nonce":3435220755}]},"public_addr":"172.21.15.113:6809/3435220755","cluster_addr":"172.21.15.113:6811/3435220755","heartbeat_back_addr":"172.21.15.113:6815/3435220755","heartbeat_front_addr":"172.21.15.113:6813/3435220755","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.040371+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.974279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.053906+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.995515+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:39.092870+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.738938+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.946610+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-30T13:10:38.912225+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.113:6837/1104336007":"2024-03-31T13:14:48.735197+0000","172.21.15.113:6836/1104336007":"2024-03-31T13:14:48.735197+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-30T13:14:55.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0408067fc0 msgr2=0x7f040806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0408067fc0 0x7f040806a480 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f040c009af0 tx=0x7f040c008040 comp rx=0 tx=0).stop 2024-03-30T13:14:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c144270 msgr2=0x7f041c13d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c144270 0x7f041c13d100 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f041000c370 tx=0x7f041000c830 comp rx=0 tx=0).stop 2024-03-30T13:14:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 shutdown_connections 2024-03-30T13:14:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0408067fc0 0x7f040806a480 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f041c15f7a0 0x7f041c152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f041c144270 0x7f041c13d100 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 --2- 172.21.15.43:0/457691571 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f041c0b2ef0 0x7f041c155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:55.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 >> 172.21.15.43:0/457691571 conn(0x7f041c0cfee0 msgr2=0x7f041c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:55.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 shutdown_connections 2024-03-30T13:14:55.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:55.493+0000 7f0422d31640 1 -- 172.21.15.43:0/457691571 wait complete. 2024-03-30T13:14:56.514 INFO:tasks.ceph:Scrubbing osd.0 2024-03-30T13:14:56.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:14:56.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca73f2b640 1 Processor -- start 2024-03-30T13:14:56.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca73f2b640 1 -- start start 2024-03-30T13:14:56.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca73f2b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fca6c0ce320 0x7fca6c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca73f2b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca6c072200 0x7fca6c0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca73f2b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca6c144d80 con 0x7fca6c1449a0 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca73f2b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fca6c15fc80 con 0x7fca6c072200 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca73f2b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fca6c072b20 con 0x7fca6c0ce320 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca724c8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca6c072200 0x7fca6c0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca724c8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fca6c072200 0x7fca6c0725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36738/0 (socket says 172.21.15.43:36738) 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca724c8640 1 -- 172.21.15.43:0/1886539315 learned_addr learned my addr 172.21.15.43:0/1886539315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:56.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca714c6640 1 --2- 172.21.15.43:0/1886539315 >> v2:172.21.15.113:3300/0 conn(0x7fca6c0ce320 0x7fca6c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca724c8640 1 -- 172.21.15.43:0/1886539315 >> v2:172.21.15.113:3300/0 conn(0x7fca6c0ce320 msgr2=0x7fca6c0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca724c8640 1 --2- 172.21.15.43:0/1886539315 >> v2:172.21.15.113:3300/0 conn(0x7fca6c0ce320 0x7fca6c0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca724c8640 1 -- 172.21.15.43:0/1886539315 >> v1:172.21.15.113:6789/0 conn(0x7fca6c1449a0 legacy=0x7fca6c15ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:56.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.653+0000 7fca724c8640 1 -- 172.21.15.43:0/1886539315 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c15f3f0 con 0x7fca6c072200 2024-03-30T13:14:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca724c8640 1 --2- 172.21.15.43:0/1886539315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c072200 0x7fca6c0725e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fca6c06f840 tx=0x7fca6401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f02db9ced374108 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca70cc5640 1 -- 172.21.15.43:0/1886539315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640a5720 con 0x7fca6c072200 2024-03-30T13:14:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca70cc5640 1 -- 172.21.15.43:0/1886539315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca640a58c0 con 0x7fca6c072200 2024-03-30T13:14:56.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca70cc5640 1 -- 172.21.15.43:0/1886539315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640a5c90 con 0x7fca6c072200 2024-03-30T13:14:56.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 -- 172.21.15.43:0/1886539315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c072200 msgr2=0x7fca6c0725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 --2- 172.21.15.43:0/1886539315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c072200 0x7fca6c0725e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fca6c06f840 tx=0x7fca6401c860 comp rx=0 tx=0).stop 2024-03-30T13:14:56.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 -- 172.21.15.43:0/1886539315 shutdown_connections 2024-03-30T13:14:56.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 --2- 172.21.15.43:0/1886539315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c072200 0x7fca6c0725e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 --2- 172.21.15.43:0/1886539315 >> v2:172.21.15.113:3300/0 conn(0x7fca6c0ce320 0x7fca6c0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 -- 172.21.15.43:0/1886539315 >> 172.21.15.43:0/1886539315 conn(0x7fca6c0b1b80 msgr2=0x7fca6c0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:56.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 -- 172.21.15.43:0/1886539315 shutdown_connections 2024-03-30T13:14:56.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 -- 172.21.15.43:0/1886539315 wait complete. 2024-03-30T13:14:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 Processor -- start 2024-03-30T13:14:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.657+0000 7fca73f2b640 1 -- start start 2024-03-30T13:14:56.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca6c072200 0x7fca6c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca6c0ce320 0x7fca6c140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c1449a0 0x7fca6c140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0cf010 con 0x7fca6c1449a0 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0cde00 con 0x7fca6c0ce320 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fca6c0cf190 con 0x7fca6c072200 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c1449a0 0x7fca6c140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c1449a0 0x7fca6c140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36750/0 (socket says 172.21.15.43:36750) 2024-03-30T13:14:56.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 -- 172.21.15.43:0/833479337 learned_addr learned my addr 172.21.15.43:0/833479337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca714c6640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca6c0ce320 0x7fca6c140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca71cc7640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca6c072200 0x7fca6c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 -- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca6c072200 msgr2=0x7fca6c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca6c072200 0x7fca6c15a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 -- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca6c0ce320 msgr2=0x7fca6c140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca6c0ce320 0x7fca6c140570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 -- 172.21.15.43:0/833479337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c15af10 con 0x7fca6c1449a0 2024-03-30T13:14:56.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca71cc7640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fca6c072200 0x7fca6c15a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca714c6640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fca6c0ce320 0x7fca6c140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca724c8640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c1449a0 0x7fca6c140ab0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fca640af040 tx=0x7fca64019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640bc020 con 0x7fca6c1449a0 2024-03-30T13:14:56.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca640b8670 con 0x7fca6c1449a0 2024-03-30T13:14:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64009510 con 0x7fca6c1449a0 2024-03-30T13:14:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c167c60 con 0x7fca6c1449a0 2024-03-30T13:14:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c168220 con 0x7fca6c1449a0 2024-03-30T13:14:56.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fca6c0687f0 con 0x7fca6c1449a0 2024-03-30T13:14:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fca640bf020 con 0x7fca6c1449a0 2024-03-30T13:14:56.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fca50067fc0 0x7fca5006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7fca641219b0 con 0x7fca6c1449a0 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fca5006e460 0x7fca500708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 --> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca640c0e70 con 0x7fca5006e460 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fca64121e40 con 0x7fca6c1449a0 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca71cc7640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fca50067fc0 0x7fca5006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca714c6640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fca5006e460 0x7fca500708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.661+0000 7fca714c6640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fca5006e460 0x7fca500708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.665+0000 7fca71cc7640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fca50067fc0 0x7fca5006a480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0c9df0 tx=0x7fca68009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:56.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.665+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== osd.0 v2:172.21.15.43:6824/3443768212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fca640c0e70 con 0x7fca5006e460 2024-03-30T13:14:56.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 --> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fca6c0cb930 con 0x7fca5006e460 2024-03-30T13:14:56.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca5e7fc640 1 -- 172.21.15.43:0/833479337 <== osd.0 v2:172.21.15.43:6824/3443768212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fca6c0cb930 con 0x7fca5006e460 2024-03-30T13:14:56.683 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:56.683 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:14:56.683 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:56.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fca5006e460 msgr2=0x7fca500708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:6824/3443768212,v1:172.21.15.43:6825/3443768212] conn(0x7fca5006e460 0x7fca500708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fca50067fc0 msgr2=0x7fca5006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fca50067fc0 0x7fca5006a480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0c9df0 tx=0x7fca68009040 comp rx=0 tx=0).stop 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c1449a0 msgr2=0x7fca6c140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 --2- 172.21.15.43:0/833479337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca6c1449a0 0x7fca6c140ab0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fca640af040 tx=0x7fca64019600 comp rx=0 tx=0).stop 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca724c8640 1 -- 172.21.15.43:0/833479337 reap_dead start 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 shutdown_connections 2024-03-30T13:14:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 >> 172.21.15.43:0/833479337 conn(0x7fca6c0b1b80 msgr2=0x7fca6c0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 shutdown_connections 2024-03-30T13:14:56.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.677+0000 7fca73f2b640 1 -- 172.21.15.43:0/833479337 wait complete. 2024-03-30T13:14:56.699 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-30T13:14:56.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.809+0000 7f32500ad640 1 Processor -- start 2024-03-30T13:14:56.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.809+0000 7f32500ad640 1 -- start start 2024-03-30T13:14:56.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f32500ad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f324813f0a0 0x7f3248160470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f32500ad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324806de90 0x7f32480cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f32500ad640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3248141970 con 0x7f32481609b0 2024-03-30T13:14:56.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f32500ad640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3248141c50 con 0x7f324813f0a0 2024-03-30T13:14:56.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f32500ad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3248141f30 con 0x7f324806de90 2024-03-30T13:14:56.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324de49640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f324813f0a0 0x7f3248160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324d648640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f32481609b0 0x7f324806d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36850/0 (socket says 172.21.15.43:36850) 2024-03-30T13:14:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324d648640 1 -- 172.21.15.43:0/1285302014 learned_addr learned my addr 172.21.15.43:0/1285302014 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324e64a640 1 --2- 172.21.15.43:0/1285302014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324806de90 0x7f32480cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324de49640 1 -- 172.21.15.43:0/1285302014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324806de90 msgr2=0x7f32480cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324de49640 1 --2- 172.21.15.43:0/1285302014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324806de90 0x7f32480cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324de49640 1 -- 172.21.15.43:0/1285302014 >> v1:172.21.15.43:6789/0 conn(0x7f32481609b0 legacy=0x7f324806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324de49640 1 -- 172.21.15.43:0/1285302014 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3248142210 con 0x7f324813f0a0 2024-03-30T13:14:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324ce47640 1 -- 172.21.15.43:0/1285302014 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 890561304 0 0) 0x7f3248141970 con 0x7f32481609b0 2024-03-30T13:14:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324de49640 1 --2- 172.21.15.43:0/1285302014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324813f0a0 0x7f3248160470 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f32440097c0 tx=0x7f324401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7da58020029d411 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324ce47640 1 -- 172.21.15.43:0/1285302014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440aa5f0 con 0x7f324813f0a0 2024-03-30T13:14:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324ce47640 1 -- 172.21.15.43:0/1285302014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32440af070 con 0x7f324813f0a0 2024-03-30T13:14:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.813+0000 7f324ce47640 1 -- 172.21.15.43:0/1285302014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440aa9b0 con 0x7f324813f0a0 2024-03-30T13:14:56.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- 172.21.15.43:0/1285302014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324813f0a0 msgr2=0x7f3248160470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 --2- 172.21.15.43:0/1285302014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324813f0a0 0x7f3248160470 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f32440097c0 tx=0x7f324401c470 comp rx=0 tx=0).stop 2024-03-30T13:14:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- 172.21.15.43:0/1285302014 shutdown_connections 2024-03-30T13:14:56.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 --2- 172.21.15.43:0/1285302014 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f324806de90 0x7f32480cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 --2- 172.21.15.43:0/1285302014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324813f0a0 0x7f3248160470 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- 172.21.15.43:0/1285302014 >> 172.21.15.43:0/1285302014 conn(0x7f3248072500 msgr2=0x7f3248072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- 172.21.15.43:0/1285302014 shutdown_connections 2024-03-30T13:14:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- 172.21.15.43:0/1285302014 wait complete. 2024-03-30T13:14:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 Processor -- start 2024-03-30T13:14:56.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- start start 2024-03-30T13:14:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324806de90 0x7f3248140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324813f0a0 0x7f3248142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32481609b0 0x7f3248144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f32480d08f0 con 0x7f324806de90 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f32480d0770 con 0x7f324813f0a0 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f32480d0a70 con 0x7f32481609b0 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324806de90 0x7f3248140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324806de90 0x7f3248140f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36770/0 (socket says 172.21.15.43:36770) 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 -- 172.21.15.43:0/2447227581 learned_addr learned my addr 172.21.15.43:0/2447227581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324d648640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324813f0a0 0x7f3248142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324e64a640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32481609b0 0x7f3248144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 -- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32481609b0 msgr2=0x7f3248144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32481609b0 0x7f3248144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 -- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324813f0a0 msgr2=0x7f3248142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324813f0a0 0x7f3248142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 -- 172.21.15.43:0/2447227581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324806c930 con 0x7f324806de90 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f324de49640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324806de90 0x7f3248140f90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f32440097c0 tx=0x7f324401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f323e7fc640 1 -- 172.21.15.43:0/2447227581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440be610 con 0x7f324806de90 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f323e7fc640 1 -- 172.21.15.43:0/2447227581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32440be7d0 con 0x7f324806de90 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324815dd40 con 0x7f324806de90 2024-03-30T13:14:56.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f323e7fc640 1 -- 172.21.15.43:0/2447227581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32440bebc0 con 0x7f324806de90 2024-03-30T13:14:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.817+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32481414d0 con 0x7f324806de90 2024-03-30T13:14:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.821+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3248068890 con 0x7f324806de90 2024-03-30T13:14:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.821+0000 7f323e7fc640 1 -- 172.21.15.43:0/2447227581 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f32440bc020 con 0x7f324806de90 2024-03-30T13:14:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.821+0000 7f323e7fc640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f322c067fc0 0x7f322c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.821+0000 7f323e7fc640 1 -- 172.21.15.43:0/2447227581 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f3244120b80 con 0x7f324806de90 2024-03-30T13:14:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.821+0000 7f324d648640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f322c067fc0 0x7f322c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.821+0000 7f324d648640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f322c067fc0 0x7f322c06a480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f32481f8480 tx=0x7f3238007c10 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:56.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.825+0000 7f323e7fc640 1 -- 172.21.15.43:0/2447227581 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3244016090 con 0x7f324806de90 2024-03-30T13:14:56.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.981+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f3248139f30 con 0x7f322c067fc0 2024-03-30T13:14:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f323e7fc640 1 -- 172.21.15.43:0/2447227581 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3238007da0 con 0x7f322c067fc0 2024-03-30T13:14:56.989 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 0 to deep-scrub 2024-03-30T13:14:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f322c067fc0 msgr2=0x7f322c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f322c067fc0 0x7f322c06a480 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f32481f8480 tx=0x7f3238007c10 comp rx=0 tx=0).stop 2024-03-30T13:14:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324806de90 msgr2=0x7f3248140f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324806de90 0x7f3248140f90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f32440097c0 tx=0x7f324401acb0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 shutdown_connections 2024-03-30T13:14:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f322c067fc0 0x7f322c06a480 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f32481609b0 0x7f3248144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f324813f0a0 0x7f3248142910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 --2- 172.21.15.43:0/2447227581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f324806de90 0x7f3248140f90 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 >> 172.21.15.43:0/2447227581 conn(0x7f3248072500 msgr2=0x7f324806b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 shutdown_connections 2024-03-30T13:14:56.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:56.985+0000 7f32500ad640 1 -- 172.21.15.43:0/2447227581 wait complete. 2024-03-30T13:14:57.007 INFO:tasks.ceph:Scrubbing osd.1 2024-03-30T13:14:57.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:14:57.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a80155640 1 Processor -- start 2024-03-30T13:14:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a80155640 1 -- start start 2024-03-30T13:14:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a80155640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a78161af0 0x7f5a780d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a80155640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a78074030 con 0x7f5a780b7cd0 2024-03-30T13:14:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a80155640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a780d2c50 con 0x7f5a780d24e0 2024-03-30T13:14:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a80155640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f5a780d2f30 con 0x7f5a78161af0 2024-03-30T13:14:57.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a7def1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a78161af0 0x7f5a780d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a7def1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5a78161af0 0x7f5a780d3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36784/0 (socket says 172.21.15.43:36784) 2024-03-30T13:14:57.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a7def1640 1 -- 172.21.15.43:0/2564545277 learned_addr learned my addr 172.21.15.43:0/2564545277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:57.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a7def1640 1 -- 172.21.15.43:0/2564545277 >> v1:172.21.15.113:6789/0 conn(0x7f5a780d24e0 legacy=0x7f5a780b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:57.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a7def1640 1 -- 172.21.15.43:0/2564545277 >> v1:172.21.15.43:6789/0 conn(0x7f5a780b7cd0 legacy=0x7f5a7806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a7ceef640 1 -- 172.21.15.43:0/2564545277 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173535514 0 0) 0x7f5a78074030 con 0x7f5a780b7cd0 2024-03-30T13:14:57.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.129+0000 7f5a7def1640 1 -- 172.21.15.43:0/2564545277 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a780d3210 con 0x7f5a78161af0 2024-03-30T13:14:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a7def1640 1 --2- 172.21.15.43:0/2564545277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78161af0 0x7f5a780d3cf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009e50 tx=0x7f5a6c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c8d752343f95cf server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a7ceef640 1 -- 172.21.15.43:0/2564545277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0b4070 con 0x7f5a78161af0 2024-03-30T13:14:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a7ceef640 1 -- 172.21.15.43:0/2564545277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6c0af400 con 0x7f5a78161af0 2024-03-30T13:14:57.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a7ceef640 1 -- 172.21.15.43:0/2564545277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0af7d0 con 0x7f5a78161af0 2024-03-30T13:14:57.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 -- 172.21.15.43:0/2564545277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78161af0 msgr2=0x7f5a780d3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 --2- 172.21.15.43:0/2564545277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78161af0 0x7f5a780d3cf0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c009e50 tx=0x7f5a6c01f1e0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 -- 172.21.15.43:0/2564545277 shutdown_connections 2024-03-30T13:14:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 --2- 172.21.15.43:0/2564545277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a78161af0 0x7f5a780d3cf0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 -- 172.21.15.43:0/2564545277 >> 172.21.15.43:0/2564545277 conn(0x7f5a780d42e0 msgr2=0x7f5a780d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 -- 172.21.15.43:0/2564545277 shutdown_connections 2024-03-30T13:14:57.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 -- 172.21.15.43:0/2564545277 wait complete. 2024-03-30T13:14:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 Processor -- start 2024-03-30T13:14:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.133+0000 7f5a80155640 1 -- start start 2024-03-30T13:14:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a780b7cd0 0x7f5a78152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a780d24e0 0x7f5a78149630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a78161af0 0x7f5a780c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f5a780d2940 con 0x7f5a780b7cd0 2024-03-30T13:14:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f5a780d37d0 con 0x7f5a78161af0 2024-03-30T13:14:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f5a78161ed0 con 0x7f5a780d24e0 2024-03-30T13:14:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a780b7cd0 0x7f5a78152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a780b7cd0 0x7f5a78152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36788/0 (socket says 172.21.15.43:36788) 2024-03-30T13:14:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 -- 172.21.15.43:0/4294274083 learned_addr learned my addr 172.21.15.43:0/4294274083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7d6f0640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a780d24e0 0x7f5a78149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7e6f2640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a78161af0 0x7f5a780c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 -- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a780d24e0 msgr2=0x7f5a78149630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a780d24e0 0x7f5a78149630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 -- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a78161af0 msgr2=0x7f5a780c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a78161af0 0x7f5a780c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 -- 172.21.15.43:0/4294274083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a780c9de0 con 0x7f5a780b7cd0 2024-03-30T13:14:57.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7d6f0640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5a780d24e0 0x7f5a78149630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7e6f2640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f5a78161af0 0x7f5a780c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7def1640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a780b7cd0 0x7f5a78152cc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0b2040 tx=0x7f5a6c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0b4040 con 0x7f5a780b7cd0 2024-03-30T13:14:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6c0afe10 con 0x7f5a780b7cd0 2024-03-30T13:14:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a780ca0c0 con 0x7f5a780b7cd0 2024-03-30T13:14:57.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c009510 con 0x7f5a780b7cd0 2024-03-30T13:14:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a780ca680 con 0x7f5a780b7cd0 2024-03-30T13:14:57.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a780687f0 con 0x7f5a780b7cd0 2024-03-30T13:14:57.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f5a6c0b7070 con 0x7f5a780b7cd0 2024-03-30T13:14:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a5c067fc0 0x7f5a5c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f5a6c123ba0 con 0x7f5a780b7cd0 2024-03-30T13:14:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7f5a5c06e460 0x7f5a5c0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 --> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a5c070f70 con 0x7f5a5c06e460 2024-03-30T13:14:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f5a6c124030 con 0x7f5a780b7cd0 2024-03-30T13:14:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7e6f2640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7f5a5c06e460 0x7f5a5c0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.137+0000 7f5a7d6f0640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a5c067fc0 0x7f5a5c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.141+0000 7f5a7e6f2640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7f5a5c06e460 0x7f5a5c0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.141+0000 7f5a7d6f0640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a5c067fc0 0x7f5a5c06a480 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a780b9a50 tx=0x7f5a64009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.141+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== osd.1 v2:172.21.15.43:6816/2193216203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a5c070f70 con 0x7f5a5c06e460 2024-03-30T13:14:57.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 --> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5a780b8f00 con 0x7f5a5c06e460 2024-03-30T13:14:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a767fc640 1 -- 172.21.15.43:0/4294274083 <== osd.1 v2:172.21.15.43:6816/2193216203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5a780b8f00 con 0x7f5a5c06e460 2024-03-30T13:14:57.159 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:57.159 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:14:57.159 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7f5a5c06e460 msgr2=0x7f5a5c0708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:6816/2193216203,v1:172.21.15.43:6817/2193216203] conn(0x7f5a5c06e460 0x7f5a5c0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a5c067fc0 msgr2=0x7f5a5c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5a5c067fc0 0x7f5a5c06a480 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5a780b9a50 tx=0x7f5a64009040 comp rx=0 tx=0).stop 2024-03-30T13:14:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a780b7cd0 msgr2=0x7f5a78152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 --2- 172.21.15.43:0/4294274083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a780b7cd0 0x7f5a78152cc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0b2040 tx=0x7f5a6c01bf80 comp rx=0 tx=0).stop 2024-03-30T13:14:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a7e6f2640 1 -- 172.21.15.43:0/4294274083 reap_dead start 2024-03-30T13:14:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 shutdown_connections 2024-03-30T13:14:57.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 >> 172.21.15.43:0/4294274083 conn(0x7f5a780d42e0 msgr2=0x7f5a7815ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 shutdown_connections 2024-03-30T13:14:57.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.153+0000 7f5a80155640 1 -- 172.21.15.43:0/4294274083 wait complete. 2024-03-30T13:14:57.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-30T13:14:57.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:57.226 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:14:57.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 Processor -- start 2024-03-30T13:14:57.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 -- start start 2024-03-30T13:14:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5534142440 0x7f55340cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f55340cf5e0 0x7f55340cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f55340cd680 0x7f553415fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f55340cfb60 con 0x7f55340cf5e0 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f55340cfe40 con 0x7f55340cd680 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553ab73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f55340d0120 con 0x7f5534142440 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f55340cf5e0 0x7f55340cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f55340cf5e0 0x7f55340cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36790/0 (socket says 172.21.15.43:36790) 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 -- 172.21.15.43:0/2802381731 learned_addr learned my addr 172.21.15.43:0/2802381731 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5539110640 1 --2- 172.21.15.43:0/2802381731 >> v2:172.21.15.113:3300/0 conn(0x7f55340cd680 0x7f553415fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f553890f640 1 --2- 172.21.15.43:0/2802381731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5534142440 0x7f55340cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 -- 172.21.15.43:0/2802381731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5534142440 msgr2=0x7f55340cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:57.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 --2- 172.21.15.43:0/2802381731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5534142440 0x7f55340cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 -- 172.21.15.43:0/2802381731 >> v2:172.21.15.113:3300/0 conn(0x7f55340cd680 msgr2=0x7f553415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:57.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 --2- 172.21.15.43:0/2802381731 >> v2:172.21.15.113:3300/0 conn(0x7f55340cd680 0x7f553415fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.289+0000 7f5533fff640 1 -- 172.21.15.43:0/2802381731 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55340d0400 con 0x7f55340cf5e0 2024-03-30T13:14:57.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f5533fff640 1 --2- 172.21.15.43:0/2802381731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55340cf5e0 0x7f55340cd140 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5528009870 tx=0x7f552801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f3f4df64d484d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f55337fe640 1 -- 172.21.15.43:0/2802381731 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280b0070 con 0x7f55340cf5e0 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f55337fe640 1 -- 172.21.15.43:0/2802381731 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55280ab600 con 0x7f55340cf5e0 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f55337fe640 1 -- 172.21.15.43:0/2802381731 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55280ab9d0 con 0x7f55340cf5e0 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- 172.21.15.43:0/2802381731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55340cf5e0 msgr2=0x7f55340cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 --2- 172.21.15.43:0/2802381731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55340cf5e0 0x7f55340cd140 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5528009870 tx=0x7f552801c260 comp rx=0 tx=0).stop 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- 172.21.15.43:0/2802381731 shutdown_connections 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 --2- 172.21.15.43:0/2802381731 >> v2:172.21.15.113:3300/0 conn(0x7f55340cd680 0x7f553415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 --2- 172.21.15.43:0/2802381731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f55340cf5e0 0x7f55340cd140 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 --2- 172.21.15.43:0/2802381731 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f5534142440 0x7f55340cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- 172.21.15.43:0/2802381731 >> 172.21.15.43:0/2802381731 conn(0x7f55340b2b30 msgr2=0x7f55340d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- 172.21.15.43:0/2802381731 shutdown_connections 2024-03-30T13:14:57.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- 172.21.15.43:0/2802381731 wait complete. 2024-03-30T13:14:57.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 Processor -- start 2024-03-30T13:14:57.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- start start 2024-03-30T13:14:57.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55340cd680 0x7f55340afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55340cf5e0 0x7f5534155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5534142440 0x7f553415b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f553406bc20 con 0x7f5534142440 2024-03-30T13:14:57.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f55340ccc70 con 0x7f55340cd680 2024-03-30T13:14:57.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f553ab73640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f553406bda0 con 0x7f55340cf5e0 2024-03-30T13:14:57.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.293+0000 7f5539110640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5534142440 0x7f553415b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5534142440 0x7f553415b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36794/0 (socket says 172.21.15.43:36794) 2024-03-30T13:14:57.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 -- 172.21.15.43:0/3074445193 learned_addr learned my addr 172.21.15.43:0/3074445193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:57.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f553890f640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55340cd680 0x7f55340afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5533fff640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55340cf5e0 0x7f5534155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 -- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55340cf5e0 msgr2=0x7f5534155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55340cf5e0 0x7f5534155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 -- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55340cd680 msgr2=0x7f55340afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55340cd680 0x7f55340afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 -- 172.21.15.43:0/3074445193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5534155f90 con 0x7f5534142440 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f5539110640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5534142440 0x7f553415b7b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f552c00ef00 tx=0x7f552c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f55317fa640 1 -- 172.21.15.43:0/3074445193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c018020 con 0x7f5534142440 2024-03-30T13:14:57.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f55317fa640 1 -- 172.21.15.43:0/3074445193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f552c014400 con 0x7f5534142440 2024-03-30T13:14:57.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55341541b0 con 0x7f5534142440 2024-03-30T13:14:57.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f55317fa640 1 -- 172.21.15.43:0/3074445193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0147f0 con 0x7f5534142440 2024-03-30T13:14:57.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55340b54c0 con 0x7f5534142440 2024-03-30T13:14:57.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55340b01c0 con 0x7f5534142440 2024-03-30T13:14:57.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f55317fa640 1 -- 172.21.15.43:0/3074445193 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f552c01b020 con 0x7f5534142440 2024-03-30T13:14:57.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f55317fa640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5520067ff0 0x7f552006a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f55317fa640 1 -- 172.21.15.43:0/3074445193 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f552c0869b0 con 0x7f5534142440 2024-03-30T13:14:57.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f553890f640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5520067ff0 0x7f552006a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.297+0000 7f553890f640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5520067ff0 0x7f552006a4b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f55240079c0 tx=0x7f5524008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.301+0000 7f55317fa640 1 -- 172.21.15.43:0/3074445193 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f552c022070 con 0x7f5534142440 2024-03-30T13:14:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.461+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f5534068890 con 0x7f5520067ff0 2024-03-30T13:14:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f55317fa640 1 -- 172.21.15.43:0/3074445193 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5534068890 con 0x7f5520067ff0 2024-03-30T13:14:57.469 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 1 to deep-scrub 2024-03-30T13:14:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5520067ff0 msgr2=0x7f552006a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5520067ff0 0x7f552006a4b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f55240079c0 tx=0x7f5524008040 comp rx=0 tx=0).stop 2024-03-30T13:14:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5534142440 msgr2=0x7f553415b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5534142440 0x7f553415b7b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f552c00ef00 tx=0x7f552c00b480 comp rx=0 tx=0).stop 2024-03-30T13:14:57.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 shutdown_connections 2024-03-30T13:14:57.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f5520067ff0 0x7f552006a4b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5534142440 0x7f553415b7b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f55340cf5e0 0x7f5534155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 --2- 172.21.15.43:0/3074445193 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f55340cd680 0x7f55340afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 >> 172.21.15.43:0/3074445193 conn(0x7f55340b2b30 msgr2=0x7f553406cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 shutdown_connections 2024-03-30T13:14:57.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.465+0000 7f553ab73640 1 -- 172.21.15.43:0/3074445193 wait complete. 2024-03-30T13:14:57.488 INFO:tasks.ceph:Scrubbing osd.2 2024-03-30T13:14:57.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:14:57.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a20f55640 1 Processor -- start 2024-03-30T13:14:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a20f55640 1 -- start start 2024-03-30T13:14:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a20f55640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a1c1449a0 0x7f2a1c15ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a20f55640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15f220 0x7f2a1c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a20f55640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a1c144d80 con 0x7f2a1c0ce860 2024-03-30T13:14:57.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a20f55640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1c15fc80 con 0x7f2a1c1449a0 2024-03-30T13:14:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a20f55640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1c15f6c0 con 0x7f2a1c15f220 2024-03-30T13:14:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a1c1449a0 0x7f2a1c15ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1ad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2a1c0ce860 0x7f2a1c072200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36868/0 (socket says 172.21.15.43:36868) 2024-03-30T13:14:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1ad9d640 1 -- 172.21.15.43:0/1687174132 learned_addr learned my addr 172.21.15.43:0/1687174132 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1a59c640 1 -- 172.21.15.43:0/1687174132 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15f220 msgr2=0x7f2a1c0ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a19d9b640 1 --2- 172.21.15.43:0/1687174132 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15f220 0x7f2a1c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1959a640 1 -- 172.21.15.43:0/1687174132 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2187929547 0 0) 0x7f2a1c144d80 con 0x7f2a1c0ce860 2024-03-30T13:14:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1a59c640 1 --2- 172.21.15.43:0/1687174132 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15f220 0x7f2a1c0ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1a59c640 1 -- 172.21.15.43:0/1687174132 >> v1:172.21.15.43:6789/0 conn(0x7f2a1c0ce860 legacy=0x7f2a1c072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a1a59c640 1 -- 172.21.15.43:0/1687174132 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c15f9a0 con 0x7f2a1c1449a0 2024-03-30T13:14:57.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.649+0000 7f2a19d9b640 1 --2- 172.21.15.43:0/1687174132 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15f220 0x7f2a1c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a1a59c640 1 --2- 172.21.15.43:0/1687174132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c15ece0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c15fe00 tx=0x7f2a0c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=550765e75d4a493c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a1959a640 1 -- 172.21.15.43:0/1687174132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0b1070 con 0x7f2a1c1449a0 2024-03-30T13:14:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a1959a640 1 -- 172.21.15.43:0/1687174132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0c0ac640 con 0x7f2a1c1449a0 2024-03-30T13:14:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a1959a640 1 -- 172.21.15.43:0/1687174132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0aca10 con 0x7f2a1c1449a0 2024-03-30T13:14:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 -- 172.21.15.43:0/1687174132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 msgr2=0x7f2a1c15ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 --2- 172.21.15.43:0/1687174132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c15ece0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c15fe00 tx=0x7f2a0c01c310 comp rx=0 tx=0).stop 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 -- 172.21.15.43:0/1687174132 shutdown_connections 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 --2- 172.21.15.43:0/1687174132 >> v2:172.21.15.113:3300/0 conn(0x7f2a1c15f220 0x7f2a1c0ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 --2- 172.21.15.43:0/1687174132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c15ece0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 -- 172.21.15.43:0/1687174132 >> 172.21.15.43:0/1687174132 conn(0x7f2a1c0b1b80 msgr2=0x7f2a1c0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 -- 172.21.15.43:0/1687174132 shutdown_connections 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 -- 172.21.15.43:0/1687174132 wait complete. 2024-03-30T13:14:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 Processor -- start 2024-03-30T13:14:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.653+0000 7f2a20f55640 1 -- start start 2024-03-30T13:14:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c0ce860 0x7f2a1c14c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c15f220 0x7f2a1c143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c06d670 con 0x7f2a1c1449a0 2024-03-30T13:14:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1c0cf220 con 0x7f2a1c15f220 2024-03-30T13:14:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1c06d7f0 con 0x7f2a1c0ce860 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c0c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36806/0 (socket says 172.21.15.43:36806) 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 -- 172.21.15.43:0/686654179 learned_addr learned my addr 172.21.15.43:0/686654179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a1ad9d640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c15f220 0x7f2a1c143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a1a59c640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c0ce860 0x7f2a1c14c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 -- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c0ce860 msgr2=0x7f2a1c14c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a1c0ce860 0x7f2a1c14c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 -- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c15f220 msgr2=0x7f2a1c143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c15f220 0x7f2a1c143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 -- 172.21.15.43:0/686654179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1c143e10 con 0x7f2a1c1449a0 2024-03-30T13:14:57.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a1ad9d640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a1c15f220 0x7f2a1c143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:57.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a19d9b640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c0c9960 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000c7f0 tx=0x7f2a1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10015070 con 0x7f2a1c1449a0 2024-03-30T13:14:57.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a10004220 con 0x7f2a1c1449a0 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1c1440f0 con 0x7f2a1c1449a0 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a10003620 con 0x7f2a1c1449a0 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1c0b41d0 con 0x7f2a1c1449a0 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a1c0b45b0 con 0x7f2a1c1449a0 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2a100039d0 con 0x7f2a1c1449a0 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f29fc067fc0 0x7f29fc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f2a10086ae0 con 0x7f2a1c1449a0 2024-03-30T13:14:57.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7f29fc06e460 0x7f29fc0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 --> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f29fc070f70 con 0x7f29fc06e460 2024-03-30T13:14:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2a10086e90 con 0x7f2a1c1449a0 2024-03-30T13:14:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a1ad9d640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7f29fc06e460 0x7f29fc0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a1a59c640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f29fc067fc0 0x7f29fc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.657+0000 7f2a1ad9d640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7f29fc06e460 0x7f29fc0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.661+0000 7f2a1a59c640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f29fc067fc0 0x7f29fc06a480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0ae000 tx=0x7f2a0c0a6070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.661+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== osd.2 v2:172.21.15.43:6800/3862342839 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f29fc070f70 con 0x7f29fc06e460 2024-03-30T13:14:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 --> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2a1c0687f0 con 0x7f29fc06e460 2024-03-30T13:14:57.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a0affd640 1 -- 172.21.15.43:0/686654179 <== osd.2 v2:172.21.15.43:6800/3862342839 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2a1c0687f0 con 0x7f29fc06e460 2024-03-30T13:14:57.678 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:57.678 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:14:57.679 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7f29fc06e460 msgr2=0x7f29fc0708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:6800/3862342839,v1:172.21.15.43:6802/3862342839] conn(0x7f29fc06e460 0x7f29fc0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f29fc067fc0 msgr2=0x7f29fc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f29fc067fc0 0x7f29fc06a480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0ae000 tx=0x7f2a0c0a6070 comp rx=0 tx=0).stop 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 msgr2=0x7f2a1c0c9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 --2- 172.21.15.43:0/686654179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a1c1449a0 0x7f2a1c0c9960 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000c7f0 tx=0x7f2a1000ccb0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a1ad9d640 1 -- 172.21.15.43:0/686654179 reap_dead start 2024-03-30T13:14:57.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 shutdown_connections 2024-03-30T13:14:57.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 >> 172.21.15.43:0/686654179 conn(0x7f2a1c0b1b80 msgr2=0x7f2a1c0b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 shutdown_connections 2024-03-30T13:14:57.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.673+0000 7f2a20f55640 1 -- 172.21.15.43:0/686654179 wait complete. 2024-03-30T13:14:57.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-30T13:14:57.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb95a9640 1 Processor -- start 2024-03-30T13:14:57.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb95a9640 1 -- start start 2024-03-30T13:14:57.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb95a9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3fb406c670 0x7f3fb40cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb95a9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fb40ccef0 con 0x7f3fb406ca50 2024-03-30T13:14:57.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb95a9640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fb40b3710 con 0x7f3fb40b41f0 2024-03-30T13:14:57.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb95a9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f3fb40b34c0 con 0x7f3fb406c670 2024-03-30T13:14:57.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3fb406c670 0x7f3fb40cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3fb406ca50 0x7f3fb40ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36870/0 (socket says 172.21.15.43:36870) 2024-03-30T13:14:57.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb37fe640 1 -- 172.21.15.43:0/264132848 learned_addr learned my addr 172.21.15.43:0/264132848 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:57.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb2ffd640 1 -- 172.21.15.43:0/264132848 >> v1:172.21.15.113:6789/0 conn(0x7f3fb40b41f0 legacy=0x7f3fb406e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb2ffd640 1 -- 172.21.15.43:0/264132848 >> v1:172.21.15.43:6789/0 conn(0x7f3fb406ca50 legacy=0x7f3fb40ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb1ffb640 1 -- 172.21.15.43:0/264132848 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4023041331 0 0) 0x7f3fb40ccef0 con 0x7f3fb406ca50 2024-03-30T13:14:57.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.805+0000 7f3fb2ffd640 1 -- 172.21.15.43:0/264132848 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb40b39f0 con 0x7f3fb406c670 2024-03-30T13:14:57.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb2ffd640 1 --2- 172.21.15.43:0/264132848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406c670 0x7f3fb40cc910 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40097c0 tx=0x7f3fa401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28de3ea00a4aa778 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb1ffb640 1 -- 172.21.15.43:0/264132848 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40b1070 con 0x7f3fb406c670 2024-03-30T13:14:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb1ffb640 1 -- 172.21.15.43:0/264132848 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa40ac410 con 0x7f3fb406c670 2024-03-30T13:14:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb1ffb640 1 -- 172.21.15.43:0/264132848 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40ac7e0 con 0x7f3fb406c670 2024-03-30T13:14:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb95a9640 1 -- 172.21.15.43:0/264132848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406c670 msgr2=0x7f3fb40cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/264132848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406c670 0x7f3fb40cc910 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40097c0 tx=0x7f3fa401c4a0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb95a9640 1 -- 172.21.15.43:0/264132848 shutdown_connections 2024-03-30T13:14:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/264132848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406c670 0x7f3fb40cc910 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb95a9640 1 -- 172.21.15.43:0/264132848 >> 172.21.15.43:0/264132848 conn(0x7f3fb40cfee0 msgr2=0x7f3fb40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb95a9640 1 -- 172.21.15.43:0/264132848 shutdown_connections 2024-03-30T13:14:57.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.809+0000 7f3fb95a9640 1 -- 172.21.15.43:0/264132848 wait complete. 2024-03-30T13:14:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 Processor -- start 2024-03-30T13:14:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 -- start start 2024-03-30T13:14:57.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb406c670 0x7f3fb4138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406ca50 0x7f3fb415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb40b41f0 0x7f3fb4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb416a080 con 0x7f3fb406ca50 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb4169f00 con 0x7f3fb40b41f0 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb416a200 con 0x7f3fb406c670 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406ca50 0x7f3fb415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406ca50 0x7f3fb415b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36812/0 (socket says 172.21.15.43:36812) 2024-03-30T13:14:57.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 -- 172.21.15.43:0/1830126782 learned_addr learned my addr 172.21.15.43:0/1830126782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb2ffd640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb406c670 0x7f3fb4138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb37fe640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb40b41f0 0x7f3fb4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 -- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb406c670 msgr2=0x7f3fb4138ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb406c670 0x7f3fb4138ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 -- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb40b41f0 msgr2=0x7f3fb4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb40b41f0 0x7f3fb4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 -- 172.21.15.43:0/1830126782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb4139890 con 0x7f3fb406ca50 2024-03-30T13:14:57.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb37fe640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb40b41f0 0x7f3fb4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb27fc640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406ca50 0x7f3fb415b9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800c7d0 tx=0x7f3fa800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3f9b7fe640 1 -- 172.21.15.43:0/1830126782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8003b80 con 0x7f3fb406ca50 2024-03-30T13:14:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3f9b7fe640 1 -- 172.21.15.43:0/1830126782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa8003d20 con 0x7f3fb406ca50 2024-03-30T13:14:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb4158230 con 0x7f3fb406ca50 2024-03-30T13:14:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3f9b7fe640 1 -- 172.21.15.43:0/1830126782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800aa90 con 0x7f3fb406ca50 2024-03-30T13:14:57.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb4157c70 con 0x7f3fb406ca50 2024-03-30T13:14:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb4068890 con 0x7f3fb406ca50 2024-03-30T13:14:57.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.813+0000 7f3f9b7fe640 1 -- 172.21.15.43:0/1830126782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f3fa800ac30 con 0x7f3fb406ca50 2024-03-30T13:14:57.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.817+0000 7f3f9b7fe640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fa0067fc0 0x7f3fa006a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:57.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.817+0000 7f3f9b7fe640 1 -- 172.21.15.43:0/1830126782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f3fa8086e30 con 0x7f3fb406ca50 2024-03-30T13:14:57.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.817+0000 7f3fb2ffd640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fa0067fc0 0x7f3fa006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:57.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.817+0000 7f3fb2ffd640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fa0067fc0 0x7f3fa006a480 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40ae000 tx=0x7f3fa4008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:57.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.821+0000 7f3f9b7fe640 1 -- 172.21.15.43:0/1830126782 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f3fb4139890 con 0x7f3fb406ca50 2024-03-30T13:14:57.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f3fb40b32a0 con 0x7f3fa0067fc0 2024-03-30T13:14:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3f9b7fe640 1 -- 172.21.15.43:0/1830126782 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3fb40b32a0 con 0x7f3fa0067fc0 2024-03-30T13:14:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 2 to deep-scrub 2024-03-30T13:14:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fa0067fc0 msgr2=0x7f3fa006a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fa0067fc0 0x7f3fa006a480 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3fa40ae000 tx=0x7f3fa4008040 comp rx=0 tx=0).stop 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406ca50 msgr2=0x7f3fb415b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406ca50 0x7f3fb415b9c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800c7d0 tx=0x7f3fa800cc90 comp rx=0 tx=0).stop 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 shutdown_connections 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3fa0067fc0 0x7f3fa006a480 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f3fb40b41f0 0x7f3fb4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fb406ca50 0x7f3fb415b9c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 --2- 172.21.15.43:0/1830126782 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f3fb406c670 0x7f3fb4138ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 >> 172.21.15.43:0/1830126782 conn(0x7f3fb40cfee0 msgr2=0x7f3fb415de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 shutdown_connections 2024-03-30T13:14:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:57.977+0000 7f3fb95a9640 1 -- 172.21.15.43:0/1830126782 wait complete. 2024-03-30T13:14:57.999 INFO:tasks.ceph:Scrubbing osd.3 2024-03-30T13:14:57.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:14:58.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.117+0000 7f7abc127640 1 Processor -- start 2024-03-30T13:14:58.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.117+0000 7f7abc127640 1 -- start start 2024-03-30T13:14:58.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ab4161af0 0x7f7ab40d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7ab40d24e0 0x7f7ab40b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab4074030 con 0x7f7ab40b7b00 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab40d2c50 con 0x7f7ab4161af0 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab40d2f30 con 0x7f7ab40d24e0 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ab4161af0 0x7f7ab40d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ab4161af0 0x7f7ab40d3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36822/0 (socket says 172.21.15.43:36822) 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 -- 172.21.15.43:0/3565636732 learned_addr learned my addr 172.21.15.43:0/3565636732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab96c2640 1 --2- 172.21.15.43:0/3565636732 >> v2:172.21.15.113:3300/0 conn(0x7f7ab40d24e0 0x7f7ab40b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 -- 172.21.15.43:0/3565636732 >> v2:172.21.15.113:3300/0 conn(0x7f7ab40d24e0 msgr2=0x7f7ab40b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:58.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 --2- 172.21.15.43:0/3565636732 >> v2:172.21.15.113:3300/0 conn(0x7f7ab40d24e0 0x7f7ab40b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 -- 172.21.15.43:0/3565636732 >> v1:172.21.15.113:6789/0 conn(0x7f7ab40b7b00 legacy=0x7f7ab40cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:58.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 -- 172.21.15.43:0/3565636732 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab40d3210 con 0x7f7ab4161af0 2024-03-30T13:14:58.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab9ec3640 1 --2- 172.21.15.43:0/3565636732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab40d3cf0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7ab406d820 tx=0x7f7ab001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff3b27af0c7f193 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab8ec1640 1 -- 172.21.15.43:0/3565636732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00b5070 con 0x7f7ab4161af0 2024-03-30T13:14:58.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab8ec1640 1 -- 172.21.15.43:0/3565636732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ab00a89d0 con 0x7f7ab4161af0 2024-03-30T13:14:58.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7ab8ec1640 1 -- 172.21.15.43:0/3565636732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00a8da0 con 0x7f7ab4161af0 2024-03-30T13:14:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 -- 172.21.15.43:0/3565636732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 msgr2=0x7f7ab40d3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 --2- 172.21.15.43:0/3565636732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab40d3cf0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7ab406d820 tx=0x7f7ab001ecd0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 -- 172.21.15.43:0/3565636732 shutdown_connections 2024-03-30T13:14:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 --2- 172.21.15.43:0/3565636732 >> v2:172.21.15.113:3300/0 conn(0x7f7ab40d24e0 0x7f7ab40b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 --2- 172.21.15.43:0/3565636732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab40d3cf0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.121+0000 7f7abc127640 1 -- 172.21.15.43:0/3565636732 >> 172.21.15.43:0/3565636732 conn(0x7f7ab40d42e0 msgr2=0x7f7ab40d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- 172.21.15.43:0/3565636732 shutdown_connections 2024-03-30T13:14:58.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- 172.21.15.43:0/3565636732 wait complete. 2024-03-30T13:14:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 Processor -- start 2024-03-30T13:14:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- start start 2024-03-30T13:14:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ab40b7b00 0x7f7ab4156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ab40d24e0 0x7f7ab415cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab415d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40d2940 con 0x7f7ab4161af0 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab40d37d0 con 0x7f7ab40b7b00 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab4161ed0 con 0x7f7ab40d24e0 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab415d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7ab96c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ab40d24e0 0x7f7ab415cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab415d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36824/0 (socket says 172.21.15.43:36824) 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7ab96c2640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ab40d24e0 0x7f7ab415cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:42058/0 (socket says 172.21.15.43:42058) 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 -- 172.21.15.43:0/3058162940 learned_addr learned my addr 172.21.15.43:0/3058162940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7ab9ec3640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ab40b7b00 0x7f7ab4156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 -- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ab40d24e0 msgr2=0x7f7ab415cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7ab40d24e0 0x7f7ab415cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 -- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ab40b7b00 msgr2=0x7f7ab4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7ab40b7b00 0x7f7ab4156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 -- 172.21.15.43:0/3058162940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab41542f0 con 0x7f7ab4161af0 2024-03-30T13:14:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aba6c4640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab415d1e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c780 tx=0x7f7aac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac004b70 con 0x7f7ab4161af0 2024-03-30T13:14:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aac004d10 con 0x7f7ab4161af0 2024-03-30T13:14:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab41545d0 con 0x7f7ab4161af0 2024-03-30T13:14:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00aa90 con 0x7f7ab4161af0 2024-03-30T13:14:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab4154b90 con 0x7f7ab4161af0 2024-03-30T13:14:58.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.125+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ab40687f0 con 0x7f7ab4161af0 2024-03-30T13:14:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7aac00ac30 con 0x7f7ab4161af0 2024-03-30T13:14:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7aaa7fc640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a98067fc0 0x7f7a9806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f7aac086c60 con 0x7f7ab4161af0 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7aaa7fc640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7f7a9806e460 0x7f7a980708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 --> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a98070f70 con 0x7f7a9806e460 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7aac087010 con 0x7f7ab4161af0 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7ab96c2640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7f7a9806e460 0x7f7a980708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7ab9ec3640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a98067fc0 0x7f7a9806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7ab96c2640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7f7a9806e460 0x7f7a980708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7ab9ec3640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a98067fc0 0x7f7a9806a480 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00b2000 tx=0x7f7ab0009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.129+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== osd.3 v2:172.21.15.43:6801/367000764 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a98070f70 con 0x7f7a9806e460 2024-03-30T13:14:58.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.141+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 --> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7ab4155150 con 0x7f7a9806e460 2024-03-30T13:14:58.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.141+0000 7f7aaa7fc640 1 -- 172.21.15.43:0/3058162940 <== osd.3 v2:172.21.15.43:6801/367000764 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7ab4155150 con 0x7f7a9806e460 2024-03-30T13:14:58.148 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:58.149 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:14:58.149 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7f7a9806e460 msgr2=0x7f7a980708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:6801/367000764,v1:172.21.15.43:6803/367000764] conn(0x7f7a9806e460 0x7f7a980708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a98067fc0 msgr2=0x7f7a9806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7a98067fc0 0x7f7a9806a480 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00b2000 tx=0x7f7ab0009040 comp rx=0 tx=0).stop 2024-03-30T13:14:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 msgr2=0x7f7ab415d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 --2- 172.21.15.43:0/3058162940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab4161af0 0x7f7ab415d1e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00c780 tx=0x7f7aac00cc40 comp rx=0 tx=0).stop 2024-03-30T13:14:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7aba6c4640 1 -- 172.21.15.43:0/3058162940 reap_dead start 2024-03-30T13:14:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 shutdown_connections 2024-03-30T13:14:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 >> 172.21.15.43:0/3058162940 conn(0x7f7ab40d42e0 msgr2=0x7f7ab415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 shutdown_connections 2024-03-30T13:14:58.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.145+0000 7f7abc127640 1 -- 172.21.15.43:0/3058162940 wait complete. 2024-03-30T13:14:58.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-30T13:14:58.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f6461e31640 1 Processor -- start 2024-03-30T13:14:58.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f6461e31640 1 -- start start 2024-03-30T13:14:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f6461e31640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f645c06c670 0x7f645c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f6461e31640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c06e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f6461e31640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c06e5c0 con 0x7f645c06ca50 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f6461e31640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f645c0b34c0 con 0x7f645c06c670 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f6461e31640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f645c0ccef0 con 0x7f645c0b41f0 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f645c06c670 0x7f645c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f645c06c670 0x7f645c0cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36828/0 (socket says 172.21.15.43:36828) 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645b7fe640 1 -- 172.21.15.43:0/1877058596 learned_addr learned my addr 172.21.15.43:0/1877058596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645affd640 1 --2- 172.21.15.43:0/1877058596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c06e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645b7fe640 1 -- 172.21.15.43:0/1877058596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 msgr2=0x7f645c06e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645b7fe640 1 --2- 172.21.15.43:0/1877058596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c06e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645b7fe640 1 -- 172.21.15.43:0/1877058596 >> v1:172.21.15.113:6789/0 conn(0x7f645c06ca50 legacy=0x7f645c15fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:58.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.269+0000 7f645b7fe640 1 -- 172.21.15.43:0/1877058596 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c0b3710 con 0x7f645c06c670 2024-03-30T13:14:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f645b7fe640 1 --2- 172.21.15.43:0/1877058596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06c670 0x7f645c0cc910 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f644c009870 tx=0x7f644c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edbbbb87c9566ca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f645a7fc640 1 -- 172.21.15.43:0/1877058596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0b0070 con 0x7f645c06c670 2024-03-30T13:14:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f645a7fc640 1 -- 172.21.15.43:0/1877058596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644c0ab600 con 0x7f645c06c670 2024-03-30T13:14:58.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f645a7fc640 1 -- 172.21.15.43:0/1877058596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c0ab9d0 con 0x7f645c06c670 2024-03-30T13:14:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 -- 172.21.15.43:0/1877058596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06c670 msgr2=0x7f645c0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 --2- 172.21.15.43:0/1877058596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06c670 0x7f645c0cc910 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f644c009870 tx=0x7f644c01c520 comp rx=0 tx=0).stop 2024-03-30T13:14:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 -- 172.21.15.43:0/1877058596 shutdown_connections 2024-03-30T13:14:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 --2- 172.21.15.43:0/1877058596 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c06e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 --2- 172.21.15.43:0/1877058596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06c670 0x7f645c0cc910 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 -- 172.21.15.43:0/1877058596 >> 172.21.15.43:0/1877058596 conn(0x7f645c0cfee0 msgr2=0x7f645c0d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 -- 172.21.15.43:0/1877058596 shutdown_connections 2024-03-30T13:14:58.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 -- 172.21.15.43:0/1877058596 wait complete. 2024-03-30T13:14:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 Processor -- start 2024-03-30T13:14:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.273+0000 7f6461e31640 1 -- start start 2024-03-30T13:14:58.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f645c06c670 0x7f645c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06ca50 0x7f645c15d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f645c146000 con 0x7f645c06ca50 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f645c145e80 con 0x7f645c06c670 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f645c168ef0 con 0x7f645c0b41f0 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06ca50 0x7f645c15d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06ca50 0x7f645c15d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36840/0 (socket says 172.21.15.43:36840) 2024-03-30T13:14:58.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 -- 172.21.15.43:0/1880029336 learned_addr learned my addr 172.21.15.43:0/1880029336 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645b7fe640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f645c06c670 0x7f645c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645bfff640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 -- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 msgr2=0x7f645c154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 -- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f645c06c670 msgr2=0x7f645c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f645c06c670 0x7f645c15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 -- 172.21.15.43:0/1880029336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c156940 con 0x7f645c06ca50 2024-03-30T13:14:58.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645b7fe640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f645c06c670 0x7f645c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645bfff640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645affd640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06ca50 0x7f645c15d0e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f645000d800 tx=0x7f645000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6443fff640 1 -- 172.21.15.43:0/1880029336 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645000b6c0 con 0x7f645c06ca50 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6443fff640 1 -- 172.21.15.43:0/1880029336 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6450019070 con 0x7f645c06ca50 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c13a260 con 0x7f645c06ca50 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6443fff640 1 -- 172.21.15.43:0/1880029336 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645000ba60 con 0x7f645c06ca50 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c13a820 con 0x7f645c06ca50 2024-03-30T13:14:58.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c068890 con 0x7f645c06ca50 2024-03-30T13:14:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6443fff640 1 -- 172.21.15.43:0/1880029336 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f645000bc00 con 0x7f645c06ca50 2024-03-30T13:14:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6443fff640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6448067ff0 0x7f644806a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f6443fff640 1 -- 172.21.15.43:0/1880029336 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f64500868c0 con 0x7f645c06ca50 2024-03-30T13:14:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645b7fe640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6448067ff0 0x7f644806a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.277+0000 7f645b7fe640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6448067ff0 0x7f644806a4b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f644c0ad000 tx=0x7f644c008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.281+0000 7f6443fff640 1 -- 172.21.15.43:0/1880029336 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f6450057150 con 0x7f645c06ca50 2024-03-30T13:14:58.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.441+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f645c0d0350 con 0x7f6448067ff0 2024-03-30T13:14:58.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6443fff640 1 -- 172.21.15.43:0/1880029336 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f645c0d0350 con 0x7f6448067ff0 2024-03-30T13:14:58.449 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 3 to deep-scrub 2024-03-30T13:14:58.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6448067ff0 msgr2=0x7f644806a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6448067ff0 0x7f644806a4b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f644c0ad000 tx=0x7f644c008040 comp rx=0 tx=0).stop 2024-03-30T13:14:58.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06ca50 msgr2=0x7f645c15d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06ca50 0x7f645c15d0e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f645000d800 tx=0x7f645000dcc0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 shutdown_connections 2024-03-30T13:14:58.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6448067ff0 0x7f644806a4b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f645c0b41f0 0x7f645c154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f645c06ca50 0x7f645c15d0e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 --2- 172.21.15.43:0/1880029336 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f645c06c670 0x7f645c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 >> 172.21.15.43:0/1880029336 conn(0x7f645c0cfee0 msgr2=0x7f645c15e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 shutdown_connections 2024-03-30T13:14:58.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.445+0000 7f6461e31640 1 -- 172.21.15.43:0/1880029336 wait complete. 2024-03-30T13:14:58.469 INFO:tasks.ceph:Scrubbing osd.4 2024-03-30T13:14:58.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:14:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a41cb6640 1 Processor -- start 2024-03-30T13:14:58.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a41cb6640 1 -- start start 2024-03-30T13:14:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a41cb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c0d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a41cb6640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a3c0b7cd0 0x7f2a3c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a41cb6640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3c074030 con 0x7f2a3c161af0 2024-03-30T13:14:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a41cb6640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2a3c0d4150 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a41cb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3c0d2a20 con 0x7f2a3c0d3cf0 2024-03-30T13:14:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a3c0b7cd0 0x7f2a3c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3affd640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c0d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3bfff640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2a3c0b7cd0 0x7f2a3c06b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:42878/0 (socket says 172.21.15.43:42878) 2024-03-30T13:14:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3bfff640 1 -- 172.21.15.43:0/2102470370 learned_addr learned my addr 172.21.15.43:0/2102470370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3affd640 1 -- 172.21.15.43:0/2102470370 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c0b7cd0 msgr2=0x7f2a3c06b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3affd640 1 --2- 172.21.15.43:0/2102470370 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c0b7cd0 0x7f2a3c06b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3affd640 1 -- 172.21.15.43:0/2102470370 >> v1:172.21.15.113:6789/0 conn(0x7f2a3c161af0 legacy=0x7f2a3c0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3affd640 1 -- 172.21.15.43:0/2102470370 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c0d2c50 con 0x7f2a3c0d3cf0 2024-03-30T13:14:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.593+0000 7f2a3bfff640 1 --2- 172.21.15.43:0/2102470370 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c0b7cd0 0x7f2a3c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-30T13:14:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a3affd640 1 --2- 172.21.15.43:0/2102470370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c0d24e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a240097c0 tx=0x7f2a2401ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ebb6c228a83eec4 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a3a7fc640 1 -- 172.21.15.43:0/2102470370 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240b5070 con 0x7f2a3c0d3cf0 2024-03-30T13:14:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a3a7fc640 1 -- 172.21.15.43:0/2102470370 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a240a8990 con 0x7f2a3c0d3cf0 2024-03-30T13:14:58.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a3a7fc640 1 -- 172.21.15.43:0/2102470370 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240a8d60 con 0x7f2a3c0d3cf0 2024-03-30T13:14:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2102470370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 msgr2=0x7f2a3c0d24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 --2- 172.21.15.43:0/2102470370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c0d24e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a240097c0 tx=0x7f2a2401ec90 comp rx=0 tx=0).stop 2024-03-30T13:14:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2102470370 shutdown_connections 2024-03-30T13:14:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 --2- 172.21.15.43:0/2102470370 >> v2:172.21.15.113:3300/0 conn(0x7f2a3c0b7cd0 0x7f2a3c06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 --2- 172.21.15.43:0/2102470370 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c0d24e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2102470370 >> 172.21.15.43:0/2102470370 conn(0x7f2a3c0d42e0 msgr2=0x7f2a3c0d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2102470370 shutdown_connections 2024-03-30T13:14:58.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.597+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2102470370 wait complete. 2024-03-30T13:14:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 Processor -- start 2024-03-30T13:14:58.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 -- start start 2024-03-30T13:14:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c0b7cd0 0x7f2a3c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c06bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c161af0 0x7f2a3c06c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c0d34f0 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2a3c0d3210 con 0x7f2a3c161af0 2024-03-30T13:14:58.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2a3c0d37d0 con 0x7f2a3c0d3cf0 2024-03-30T13:14:58.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c0b7cd0 0x7f2a3c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c0b7cd0 0x7f2a3c1550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36848/0 (socket says 172.21.15.43:36848) 2024-03-30T13:14:58.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 -- 172.21.15.43:0/2137349442 learned_addr learned my addr 172.21.15.43:0/2137349442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3bfff640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c161af0 0x7f2a3c06c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3affd640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c06bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 -- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 msgr2=0x7f2a3c06bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2a3c0d3cf0 0x7f2a3c06bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 -- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c161af0 msgr2=0x7f2a3c06c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2a3c161af0 0x7f2a3c06c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 -- 172.21.15.43:0/2137349442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3c0b5410 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a3b7fe640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c0b7cd0 0x7f2a3c1550e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400c420 tx=0x7f2a3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a340052d0 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a34005470 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34007260 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3c0b56f0 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a3c0b5cb0 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.601+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a3c0687f0 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2a34007400 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a33fff640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a1c067fc0 0x7f2a1c06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f2a34086410 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a33fff640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7f2a1c06e460 0x7f2a1c0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 --> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a34015ec0 con 0x7f2a1c06e460 2024-03-30T13:14:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2a340868a0 con 0x7f2a3c0b7cd0 2024-03-30T13:14:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a3affd640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a1c067fc0 0x7f2a1c06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a3bfff640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7f2a1c06e460 0x7f2a1c0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a3affd640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a1c067fc0 0x7f2a1c06a480 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2a240b2000 tx=0x7f2a24008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a3bfff640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7f2a1c06e460 0x7f2a1c0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.605+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== osd.4 v2:172.21.15.113:6824/840052469 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a34015ec0 con 0x7f2a1c06e460 2024-03-30T13:14:58.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 --> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2a3c0b7790 con 0x7f2a1c06e460 2024-03-30T13:14:58.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a33fff640 1 -- 172.21.15.43:0/2137349442 <== osd.4 v2:172.21.15.113:6824/840052469 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2a3c0b7790 con 0x7f2a1c06e460 2024-03-30T13:14:58.627 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:58.627 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:14:58.627 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:58.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7f2a1c06e460 msgr2=0x7f2a1c0708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6824/840052469,v1:172.21.15.113:6825/840052469] conn(0x7f2a1c06e460 0x7f2a1c0708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a1c067fc0 msgr2=0x7f2a1c06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2a1c067fc0 0x7f2a1c06a480 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2a240b2000 tx=0x7f2a24008040 comp rx=0 tx=0).stop 2024-03-30T13:14:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c0b7cd0 msgr2=0x7f2a3c1550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 --2- 172.21.15.43:0/2137349442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a3c0b7cd0 0x7f2a3c1550e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400c420 tx=0x7f2a3400c8e0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a3bfff640 1 -- 172.21.15.43:0/2137349442 reap_dead start 2024-03-30T13:14:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 shutdown_connections 2024-03-30T13:14:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 >> 172.21.15.43:0/2137349442 conn(0x7f2a3c0d42e0 msgr2=0x7f2a3c0d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 shutdown_connections 2024-03-30T13:14:58.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.621+0000 7f2a41cb6640 1 -- 172.21.15.43:0/2137349442 wait complete. 2024-03-30T13:14:58.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-30T13:14:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 Processor -- start 2024-03-30T13:14:58.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 -- start start 2024-03-30T13:14:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f620815e400 0x7f62080b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f62080ad8e0 0x7f620806d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620806b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f620806f190 con 0x7f620815e400 2024-03-30T13:14:58.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f62080b08a0 con 0x7f62080ad8e0 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620f54f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f620806f3e0 con 0x7f620806dcc0 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f620815e400 0x7f62080b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f620815e400 0x7f62080b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36852/0 (socket says 172.21.15.43:36852) 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 -- 172.21.15.43:0/765316171 learned_addr learned my addr 172.21.15.43:0/765316171 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620daec640 1 --2- 172.21.15.43:0/765316171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620caea640 1 --2- 172.21.15.43:0/765316171 >> v2:172.21.15.113:3300/0 conn(0x7f62080ad8e0 0x7f620806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 -- 172.21.15.43:0/765316171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 msgr2=0x7f620806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:58.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 --2- 172.21.15.43:0/765316171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 -- 172.21.15.43:0/765316171 >> v2:172.21.15.113:3300/0 conn(0x7f62080ad8e0 msgr2=0x7f620806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 --2- 172.21.15.43:0/765316171 >> v2:172.21.15.113:3300/0 conn(0x7f62080ad8e0 0x7f620806d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.757+0000 7f620d2eb640 1 -- 172.21.15.43:0/765316171 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620806f6c0 con 0x7f620815e400 2024-03-30T13:14:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620d2eb640 1 --2- 172.21.15.43:0/765316171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620815e400 0x7f62080b0360 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f620806c6c0 tx=0x7f620401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e981087a8a15f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f61fffff640 1 -- 172.21.15.43:0/765316171 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040b0070 con 0x7f620815e400 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f61fffff640 1 -- 172.21.15.43:0/765316171 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62040ab600 con 0x7f620815e400 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f61fffff640 1 -- 172.21.15.43:0/765316171 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62040ab9d0 con 0x7f620815e400 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 -- 172.21.15.43:0/765316171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620815e400 msgr2=0x7f62080b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 --2- 172.21.15.43:0/765316171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620815e400 0x7f62080b0360 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f620806c6c0 tx=0x7f620401c260 comp rx=0 tx=0).stop 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 -- 172.21.15.43:0/765316171 shutdown_connections 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 --2- 172.21.15.43:0/765316171 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 --2- 172.21.15.43:0/765316171 >> v2:172.21.15.113:3300/0 conn(0x7f62080ad8e0 0x7f620806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 --2- 172.21.15.43:0/765316171 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f620815e400 0x7f62080b0360 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 -- 172.21.15.43:0/765316171 >> 172.21.15.43:0/765316171 conn(0x7f6208072500 msgr2=0x7f6208072910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 -- 172.21.15.43:0/765316171 shutdown_connections 2024-03-30T13:14:58.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 -- 172.21.15.43:0/765316171 wait complete. 2024-03-30T13:14:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 Processor -- start 2024-03-30T13:14:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.761+0000 7f620f54f640 1 -- start start 2024-03-30T13:14:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620813d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62080ad8e0 0x7f6208153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f620815e400 0x7f620814b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f62080affe0 con 0x7f62080ad8e0 2024-03-30T13:14:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f62080afe60 con 0x7f620815e400 2024-03-30T13:14:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f62080b54e0 con 0x7f620806dcc0 2024-03-30T13:14:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62080ad8e0 0x7f6208153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62080ad8e0 0x7f6208153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36856/0 (socket says 172.21.15.43:36856) 2024-03-30T13:14:58.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 -- 172.21.15.43:0/1509054323 learned_addr learned my addr 172.21.15.43:0/1509054323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620d2eb640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620813d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620daec640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f620815e400 0x7f620814b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 -- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 msgr2=0x7f620813d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620813d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 -- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f620815e400 msgr2=0x7f620814b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f620815e400 0x7f620814b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 -- 172.21.15.43:0/1509054323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620806b660 con 0x7f62080ad8e0 2024-03-30T13:14:58.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620caea640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62080ad8e0 0x7f6208153a90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f61f800d7d0 tx=0x7f61f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f61fdffb640 1 -- 172.21.15.43:0/1509054323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f800b6c0 con 0x7f62080ad8e0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f61fdffb640 1 -- 172.21.15.43:0/1509054323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f8019070 con 0x7f62080ad8e0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62081529f0 con 0x7f62080ad8e0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f61fdffb640 1 -- 172.21.15.43:0/1509054323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f800baa0 con 0x7f62080ad8e0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6208152fb0 con 0x7f62080ad8e0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6208153780 con 0x7f62080ad8e0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f61fdffb640 1 -- 172.21.15.43:0/1509054323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f61f800bc40 con 0x7f62080ad8e0 2024-03-30T13:14:58.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f61fdffb640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f61ec067ff0 0x7f61ec06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f61fdffb640 1 -- 172.21.15.43:0/1509054323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f61f8086780 con 0x7f62080ad8e0 2024-03-30T13:14:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.765+0000 7f620d2eb640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f61ec067ff0 0x7f61ec06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.769+0000 7f620d2eb640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f61ec067ff0 0x7f61ec06a4b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f62040ad000 tx=0x7f6204008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:58.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.769+0000 7f61fdffb640 1 -- 172.21.15.43:0/1509054323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f61f80570a0 con 0x7f62080ad8e0 2024-03-30T13:14:58.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.933+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f6208068890 con 0x7f61ec067ff0 2024-03-30T13:14:58.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.933+0000 7f61fdffb640 1 -- 172.21.15.43:0/1509054323 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6208068890 con 0x7f61ec067ff0 2024-03-30T13:14:58.939 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 4 to deep-scrub 2024-03-30T13:14:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f61ec067ff0 msgr2=0x7f61ec06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f61ec067ff0 0x7f61ec06a4b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f62040ad000 tx=0x7f6204008040 comp rx=0 tx=0).stop 2024-03-30T13:14:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62080ad8e0 msgr2=0x7f6208153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62080ad8e0 0x7f6208153a90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f61f800d7d0 tx=0x7f61f800dc90 comp rx=0 tx=0).stop 2024-03-30T13:14:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 shutdown_connections 2024-03-30T13:14:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f61ec067ff0 0x7f61ec06a4b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f620815e400 0x7f620814b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62080ad8e0 0x7f6208153a90 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 --2- 172.21.15.43:0/1509054323 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f620806dcc0 0x7f620813d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:58.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 >> 172.21.15.43:0/1509054323 conn(0x7f6208072500 msgr2=0x7f62080b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:58.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 shutdown_connections 2024-03-30T13:14:58.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:58.937+0000 7f620f54f640 1 -- 172.21.15.43:0/1509054323 wait complete. 2024-03-30T13:14:58.959 INFO:tasks.ceph:Scrubbing osd.5 2024-03-30T13:14:58.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:14:59.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c204e1640 1 Processor -- start 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c204e1640 1 -- start start 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c204e1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c180b5550 0x7f2c18160610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c204e1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c18073490 0x7f2c1806c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c204e1640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c1815d090 con 0x7f2c180b5930 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c204e1640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f2c1815ce30 con 0x7f2c180b5550 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c204e1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2c1815d370 con 0x7f2c18073490 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1ea7e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c18073490 0x7f2c1806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1e27d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c180b5550 0x7f2c18160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1e27d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f2c180b5550 0x7f2c18160610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:42916/0 (socket says 172.21.15.43:42916) 2024-03-30T13:14:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1ea7e640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c18073490 0x7f2c1806c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3301/0 says I am v2:172.21.15.43:42142/0 (socket says 172.21.15.43:42142) 2024-03-30T13:14:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1e27d640 1 -- 172.21.15.43:0/1695110434 learned_addr learned my addr 172.21.15.43:0/1695110434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1ea7e640 1 -- 172.21.15.43:0/1695110434 >> v2:172.21.15.113:3300/0 conn(0x7f2c180b5550 msgr2=0x7f2c18160610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1ea7e640 1 --2- 172.21.15.43:0/1695110434 >> v2:172.21.15.113:3300/0 conn(0x7f2c180b5550 0x7f2c18160610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1ea7e640 1 -- 172.21.15.43:0/1695110434 >> v1:172.21.15.113:6789/0 conn(0x7f2c180b5930 legacy=0x7f2c1815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.081+0000 7f2c1ea7e640 1 -- 172.21.15.43:0/1695110434 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1815fe10 con 0x7f2c18073490 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c1ea7e640 1 --2- 172.21.15.43:0/1695110434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c18073490 0x7f2c1806c670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009d90 tx=0x7f2c1001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c74a7ff4bb161bb server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c1d27b640 1 -- 172.21.15.43:0/1695110434 <== mon.2 v2:172.21.15.113:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c100b2070 con 0x7f2c18073490 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c1d27b640 1 -- 172.21.15.43:0/1695110434 <== mon.2 v2:172.21.15.113:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c100a5910 con 0x7f2c18073490 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c1d27b640 1 -- 172.21.15.43:0/1695110434 <== mon.2 v2:172.21.15.113:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c100a5ce0 con 0x7f2c18073490 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 -- 172.21.15.43:0/1695110434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c18073490 msgr2=0x7f2c1806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 --2- 172.21.15.43:0/1695110434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c18073490 0x7f2c1806c670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009d90 tx=0x7f2c1001c940 comp rx=0 tx=0).stop 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 -- 172.21.15.43:0/1695110434 shutdown_connections 2024-03-30T13:14:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 --2- 172.21.15.43:0/1695110434 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c18073490 0x7f2c1806c670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 --2- 172.21.15.43:0/1695110434 >> v2:172.21.15.113:3300/0 conn(0x7f2c180b5550 0x7f2c18160610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 -- 172.21.15.43:0/1695110434 >> 172.21.15.43:0/1695110434 conn(0x7f2c180b1b80 msgr2=0x7f2c180b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 -- 172.21.15.43:0/1695110434 shutdown_connections 2024-03-30T13:14:59.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 -- 172.21.15.43:0/1695110434 wait complete. 2024-03-30T13:14:59.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 Processor -- start 2024-03-30T13:14:59.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.085+0000 7f2c204e1640 1 -- start start 2024-03-30T13:14:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c18073490 0x7f2c18149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5550 0x7f2c180b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5930 0x7f2c1814de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18169290 con 0x7f2c18073490 2024-03-30T13:14:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f2c18169110 con 0x7f2c180b5930 2024-03-30T13:14:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f2c18169410 con 0x7f2c180b5550 2024-03-30T13:14:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c18073490 0x7f2c18149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c18073490 0x7f2c18149580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36864/0 (socket says 172.21.15.43:36864) 2024-03-30T13:14:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 -- 172.21.15.43:0/2421334985 learned_addr learned my addr 172.21.15.43:0/2421334985 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1da7c640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5550 0x7f2c180b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1ea7e640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5930 0x7f2c1814de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 -- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5550 msgr2=0x7f2c180b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f2c180b5550 0x7f2c180b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 -- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5930 msgr2=0x7f2c1814de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5930 0x7f2c1814de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 -- 172.21.15.43:0/2421334985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c180b3a20 con 0x7f2c18073490 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1ea7e640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f2c180b5930 0x7f2c1814de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c1e27d640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c18073490 0x7f2c18149580 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e8c0 tx=0x7f2c0800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08017070 con 0x7f2c18073490 2024-03-30T13:14:59.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0800c940 con 0x7f2c18073490 2024-03-30T13:14:59.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800cd30 con 0x7f2c18073490 2024-03-30T13:14:59.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c180b3d00 con 0x7f2c18073490 2024-03-30T13:14:59.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c180b42c0 con 0x7f2c18073490 2024-03-30T13:14:59.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c180b46a0 con 0x7f2c18073490 2024-03-30T13:14:59.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f2c0801a070 con 0x7f2c18073490 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2bfc067fc0 0x7f2bfc06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f2c08086ad0 con 0x7f2c18073490 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7f2bfc06e460 0x7f2bfc0708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 --> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bfc070f70 con 0x7f2bfc06e460 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.089+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2c08086f60 con 0x7f2c18073490 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.093+0000 7f2c1da7c640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2bfc067fc0 0x7f2bfc06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.093+0000 7f2c1ea7e640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7f2bfc06e460 0x7f2bfc0708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.093+0000 7f2c1da7c640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2bfc067fc0 0x7f2bfc06a480 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c181f8cb0 tx=0x7f2c14009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.093+0000 7f2c1ea7e640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7f2bfc06e460 0x7f2bfc0708a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.093+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== osd.5 v2:172.21.15.113:6816/2302616840 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2bfc070f70 con 0x7f2bfc06e460 2024-03-30T13:14:59.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.105+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 --> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f2c180687f0 con 0x7f2bfc06e460 2024-03-30T13:14:59.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c0effd640 1 -- 172.21.15.43:0/2421334985 <== osd.5 v2:172.21.15.113:6816/2302616840 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f2c180687f0 con 0x7f2bfc06e460 2024-03-30T13:14:59.113 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:59.113 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:14:59.113 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:59.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7f2bfc06e460 msgr2=0x7f2bfc0708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6816/2302616840,v1:172.21.15.113:6817/2302616840] conn(0x7f2bfc06e460 0x7f2bfc0708a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2bfc067fc0 msgr2=0x7f2bfc06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f2bfc067fc0 0x7f2bfc06a480 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2c181f8cb0 tx=0x7f2c14009040 comp rx=0 tx=0).stop 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c18073490 msgr2=0x7f2c18149580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 --2- 172.21.15.43:0/2421334985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c18073490 0x7f2c18149580 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e8c0 tx=0x7f2c0800ed80 comp rx=0 tx=0).stop 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c1ea7e640 1 -- 172.21.15.43:0/2421334985 reap_dead start 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 shutdown_connections 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 >> 172.21.15.43:0/2421334985 conn(0x7f2c180b1b80 msgr2=0x7f2c180afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 shutdown_connections 2024-03-30T13:14:59.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.109+0000 7f2c204e1640 1 -- 172.21.15.43:0/2421334985 wait complete. 2024-03-30T13:14:59.129 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-30T13:14:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d23c97640 1 Processor -- start 2024-03-30T13:14:59.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d23c97640 1 -- start start 2024-03-30T13:14:59.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d23c97640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d1c162890 0x7f4d1c06b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d23c97640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f4d1c0709f0 0x7f4d1c06f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d23c97640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d1c06bba0 con 0x7f4d1c06ff20 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d23c97640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f4d1c0b4c20 con 0x7f4d1c162890 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d23c97640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f4d1c0b4010 con 0x7f4d1c0709f0 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d21a33640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d1c162890 0x7f4d1c06b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d22234640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4d1c06ff20 0x7f4d1c1662b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36882/0 (socket says 172.21.15.43:36882) 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d22234640 1 -- 172.21.15.43:0/88778721 learned_addr learned my addr 172.21.15.43:0/88778721 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d21232640 1 --2- 172.21.15.43:0/88778721 >> v2:172.21.15.113:3300/0 conn(0x7f4d1c0709f0 0x7f4d1c06f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d21a33640 1 -- 172.21.15.43:0/88778721 >> v2:172.21.15.113:3300/0 conn(0x7f4d1c0709f0 msgr2=0x7f4d1c06f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d21a33640 1 --2- 172.21.15.43:0/88778721 >> v2:172.21.15.113:3300/0 conn(0x7f4d1c0709f0 0x7f4d1c06f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d21a33640 1 -- 172.21.15.43:0/88778721 >> v1:172.21.15.43:6789/0 conn(0x7f4d1c06ff20 legacy=0x7f4d1c1662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.245+0000 7f4d21a33640 1 -- 172.21.15.43:0/88778721 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c0b42f0 con 0x7f4d1c162890 2024-03-30T13:14:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d21a33640 1 --2- 172.21.15.43:0/88778721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c162890 0x7f4d1c06b660 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c166210 tx=0x7f4d1401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33c5f663447cc2c server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d20a31640 1 -- 172.21.15.43:0/88778721 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140b2070 con 0x7f4d1c162890 2024-03-30T13:14:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d20a31640 1 -- 172.21.15.43:0/88778721 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140a58e0 con 0x7f4d1c162890 2024-03-30T13:14:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d20a31640 1 -- 172.21.15.43:0/88778721 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140a5cb0 con 0x7f4d1c162890 2024-03-30T13:14:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 -- 172.21.15.43:0/88778721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c162890 msgr2=0x7f4d1c06b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 --2- 172.21.15.43:0/88778721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c162890 0x7f4d1c06b660 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c166210 tx=0x7f4d1401c930 comp rx=0 tx=0).stop 2024-03-30T13:14:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 -- 172.21.15.43:0/88778721 shutdown_connections 2024-03-30T13:14:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 --2- 172.21.15.43:0/88778721 >> v2:172.21.15.113:3300/0 conn(0x7f4d1c0709f0 0x7f4d1c06f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 --2- 172.21.15.43:0/88778721 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c162890 0x7f4d1c06b660 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 -- 172.21.15.43:0/88778721 >> 172.21.15.43:0/88778721 conn(0x7f4d1c06e410 msgr2=0x7f4d1c06e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 -- 172.21.15.43:0/88778721 shutdown_connections 2024-03-30T13:14:59.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 -- 172.21.15.43:0/88778721 wait complete. 2024-03-30T13:14:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.249+0000 7f4d23c97640 1 Processor -- start 2024-03-30T13:14:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 -- start start 2024-03-30T13:14:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c06ff20 0x7f4d1c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d1c0709f0 0x7f4d1c14fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d1c162890 0x7f4d1c15a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c16f700 con 0x7f4d1c06ff20 2024-03-30T13:14:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c16f580 con 0x7f4d1c0709f0 2024-03-30T13:14:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c16f880 con 0x7f4d1c162890 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c06ff20 0x7f4d1c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c06ff20 0x7f4d1c1512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36872/0 (socket says 172.21.15.43:36872) 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 -- 172.21.15.43:0/1289528200 learned_addr learned my addr 172.21.15.43:0/1289528200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d22234640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d1c162890 0x7f4d1c15a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21232640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d1c0709f0 0x7f4d1c14fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 -- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d1c162890 msgr2=0x7f4d1c15a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d1c162890 0x7f4d1c15a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 -- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d1c0709f0 msgr2=0x7f4d1c14fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d1c0709f0 0x7f4d1c14fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 -- 172.21.15.43:0/1289528200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c153520 con 0x7f4d1c06ff20 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d22234640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d1c162890 0x7f4d1c15a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d21a33640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c06ff20 0x7f4d1c1512d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4d14019540 tx=0x7f4d140196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d1a7fc640 1 -- 172.21.15.43:0/1289528200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140b2050 con 0x7f4d1c06ff20 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d1a7fc640 1 -- 172.21.15.43:0/1289528200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d140be070 con 0x7f4d1c06ff20 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c15b930 con 0x7f4d1c06ff20 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d1a7fc640 1 -- 172.21.15.43:0/1289528200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d140ada70 con 0x7f4d1c06ff20 2024-03-30T13:14:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1537a0 con 0x7f4d1c06ff20 2024-03-30T13:14:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1c068890 con 0x7f4d1c06ff20 2024-03-30T13:14:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d1a7fc640 1 -- 172.21.15.43:0/1289528200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f4d140adc10 con 0x7f4d1c06ff20 2024-03-30T13:14:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d1a7fc640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d08067fc0 0x7f4d0806a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.253+0000 7f4d1a7fc640 1 -- 172.21.15.43:0/1289528200 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f4d14121660 con 0x7f4d1c06ff20 2024-03-30T13:14:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.257+0000 7f4d21232640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d08067fc0 0x7f4d0806a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.257+0000 7f4d21232640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d08067fc0 0x7f4d0806a480 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c1521b0 tx=0x7f4d10008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.261+0000 7f4d1a7fc640 1 -- 172.21.15.43:0/1289528200 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f4d1c153520 con 0x7f4d1c06ff20 2024-03-30T13:14:59.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.413+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f4d1c0b28c0 con 0x7f4d08067fc0 2024-03-30T13:14:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.413+0000 7f4d1a7fc640 1 -- 172.21.15.43:0/1289528200 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4d1c0b28c0 con 0x7f4d08067fc0 2024-03-30T13:14:59.420 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 5 to deep-scrub 2024-03-30T13:14:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d08067fc0 msgr2=0x7f4d0806a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d08067fc0 0x7f4d0806a480 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c1521b0 tx=0x7f4d10008040 comp rx=0 tx=0).stop 2024-03-30T13:14:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c06ff20 msgr2=0x7f4d1c1512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c06ff20 0x7f4d1c1512d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4d14019540 tx=0x7f4d140196d0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 shutdown_connections 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f4d08067fc0 0x7f4d0806a480 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f4d1c162890 0x7f4d1c15a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f4d1c0709f0 0x7f4d1c14fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 --2- 172.21.15.43:0/1289528200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c06ff20 0x7f4d1c1512d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 >> 172.21.15.43:0/1289528200 conn(0x7f4d1c06e410 msgr2=0x7f4d1c157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 shutdown_connections 2024-03-30T13:14:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.417+0000 7f4d23c97640 1 -- 172.21.15.43:0/1289528200 wait complete. 2024-03-30T13:14:59.438 INFO:tasks.ceph:Scrubbing osd.6 2024-03-30T13:14:59.438 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:14:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ec568a640 1 Processor -- start 2024-03-30T13:14:59.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ec568a640 1 -- start start 2024-03-30T13:14:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ec568a640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f0ec01449a0 0x7f0ec015ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ec568a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ec0144d80 con 0x7f0ec0072200 2024-03-30T13:14:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ec568a640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ec015f6c0 con 0x7f0ec015f220 2024-03-30T13:14:59.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ec568a640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f0ec015fc80 con 0x7f0ec01449a0 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0ec0072200 0x7f0ec006c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36892/0 (socket says 172.21.15.43:36892) 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebf7fe640 1 -- 172.21.15.43:0/2224449430 learned_addr learned my addr 172.21.15.43:0/2224449430 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebeffd640 1 --2- 172.21.15.43:0/2224449430 >> v2:172.21.15.113:3300/0 conn(0x7f0ec01449a0 0x7f0ec015ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894660766 0 0) 0x7f0ec0144d80 con 0x7f0ec0072200 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea8005740 con 0x7f0ec0072200 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2060325933 0 0) 0x7f0ea8005740 con 0x7f0ec0072200 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 >> v2:172.21.15.113:3300/0 conn(0x7f0ec01449a0 msgr2=0x7f0ec015ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:59.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 --2- 172.21.15.43:0/2224449430 >> v2:172.21.15.113:3300/0 conn(0x7f0ec01449a0 0x7f0ec015ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 >> v1:172.21.15.113:6789/0 conn(0x7f0ec015f220 legacy=0x7f0ec00ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec015f9a0 con 0x7f0ec0072200 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2025366159 0 0) 0x7f0eb8001a00 con 0x7f0ec0072200 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 >> v1:172.21.15.43:6789/0 conn(0x7f0ec0072200 legacy=0x7f0ec006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea8005c20 0x7f0ea8006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ea8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea8007500 0x7f0ea800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec015fc80 con 0x7f0ea8006a20 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea800e400 con 0x7f0ea8005c20 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea800e600 con 0x7f0ea8007500 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.553+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ea8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebeffd640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea8005c20 0x7f0ea8006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebf7fe640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea8007500 0x7f0ea800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebe7fc640 1 -- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea8007500 msgr2=0x7f0ea800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ea8007500 0x7f0ea800dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebe7fc640 1 -- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea8005c20 msgr2=0x7f0ea8006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:59.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ea8005c20 0x7f0ea8006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebf7fe640 1 -- 172.21.15.43:0/2224449430 reap_dead start 2024-03-30T13:14:59.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebe7fc640 1 -- 172.21.15.43:0/2224449430 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea8005740 con 0x7f0ea8006a20 2024-03-30T13:14:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ea8006ea0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4001750 tx=0x7f0eb401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbe3d85067a81e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb40013a0 con 0x7f0ea8006a20 2024-03-30T13:14:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb40052f0 con 0x7f0ea8006a20 2024-03-30T13:14:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ebdffb640 1 -- 172.21.15.43:0/2224449430 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4003840 con 0x7f0ea8006a20 2024-03-30T13:14:59.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ec568a640 1 -- 172.21.15.43:0/2224449430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 msgr2=0x7f0ea8006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ec568a640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ea8006ea0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4001750 tx=0x7f0eb401c3b0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ec568a640 1 -- 172.21.15.43:0/2224449430 shutdown_connections 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ec568a640 1 --2- 172.21.15.43:0/2224449430 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ea8006ea0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ec568a640 1 -- 172.21.15.43:0/2224449430 >> 172.21.15.43:0/2224449430 conn(0x7f0ec00b1b80 msgr2=0x7f0ec00b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ec568a640 1 -- 172.21.15.43:0/2224449430 shutdown_connections 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.557+0000 7f0ec568a640 1 -- 172.21.15.43:0/2224449430 wait complete. 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 Processor -- start 2024-03-30T13:14:59.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 -- start start 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ec00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec015a6e0 0x7f0ec015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ec015d060 0x7f0ec015d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec0072b70 con 0x7f0ea8006a20 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f0ec00729f0 con 0x7f0ec015a6e0 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f0ec00cf010 con 0x7f0ec015d060 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ec00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ec00b3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36876/0 (socket says 172.21.15.43:36876) 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 -- 172.21.15.43:0/3759164481 learned_addr learned my addr 172.21.15.43:0/3759164481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebf7fe640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ec015d060 0x7f0ec015d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec015a6e0 0x7f0ec015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 -- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ec015d060 msgr2=0x7f0ec015d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ec015d060 0x7f0ec015d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 -- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec015a6e0 msgr2=0x7f0ec015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec015a6e0 0x7f0ec015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 -- 172.21.15.43:0/3759164481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ec015dd90 con 0x7f0ea8006a20 2024-03-30T13:14:59.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebf7fe640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f0ec015d060 0x7f0ec015d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f0ec015a6e0 0x7f0ec015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ebeffd640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ec00b3750 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0011500 tx=0x7f0eb00119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0015870 con 0x7f0ea8006a20 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eb0015a10 con 0x7f0ea8006a20 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ec015e070 con 0x7f0ea8006a20 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0015de0 con 0x7f0ea8006a20 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ec015e630 con 0x7f0ea8006a20 2024-03-30T13:14:59.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ec00687f0 con 0x7f0ea8006a20 2024-03-30T13:14:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.561+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f0eb001d400 con 0x7f0ea8006a20 2024-03-30T13:14:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ea77fe640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0eac067ff0 0x7f0eac06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f0eb00871d0 con 0x7f0ea8006a20 2024-03-30T13:14:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ea77fe640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f0eac06e490 0x7f0eac0708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 --> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0eb0012e50 con 0x7f0eac06e490 2024-03-30T13:14:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0eb0087660 con 0x7f0ea8006a20 2024-03-30T13:14:59.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0eac067ff0 0x7f0eac06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ebf7fe640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f0eac06e490 0x7f0eac0708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ebe7fc640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0eac067ff0 0x7f0eac06a4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0071210 tx=0x7f0eb40011a0 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ebf7fe640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f0eac06e490 0x7f0eac0708d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.565+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== osd.6 v2:172.21.15.113:6800/594900508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0eb0012e50 con 0x7f0eac06e490 2024-03-30T13:14:59.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 --> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0ec006f260 con 0x7f0eac06e490 2024-03-30T13:14:59.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ea77fe640 1 -- 172.21.15.43:0/3759164481 <== osd.6 v2:172.21.15.113:6800/594900508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0ec006f260 con 0x7f0eac06e490 2024-03-30T13:14:59.586 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:14:59.586 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:14:59.586 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:14:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f0eac06e490 msgr2=0x7f0eac0708d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6800/594900508,v1:172.21.15.113:6801/594900508] conn(0x7f0eac06e490 0x7f0eac0708d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0eac067ff0 msgr2=0x7f0eac06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f0eac067ff0 0x7f0eac06a4b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0071210 tx=0x7f0eb40011a0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 msgr2=0x7f0ec00b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 --2- 172.21.15.43:0/3759164481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ea8006a20 0x7f0ec00b3750 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0011500 tx=0x7f0eb00119c0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ebf7fe640 1 -- 172.21.15.43:0/3759164481 reap_dead start 2024-03-30T13:14:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 shutdown_connections 2024-03-30T13:14:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 >> 172.21.15.43:0/3759164481 conn(0x7f0ec00b1b80 msgr2=0x7f0ec00afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 shutdown_connections 2024-03-30T13:14:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.581+0000 7f0ec568a640 1 -- 172.21.15.43:0/3759164481 wait complete. 2024-03-30T13:14:59.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-30T13:14:59.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.709+0000 7fb9bd183640 1 Processor -- start 2024-03-30T13:14:59.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.709+0000 7fb9bd183640 1 -- start start 2024-03-30T13:14:59.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.709+0000 7fb9bd183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815eb10 0x7fb9b80b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.709+0000 7fb9bd183640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9b806ea60 0x7fb9b80b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.709+0000 7fb9bd183640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9b8072cb0 con 0x7fb9b815a840 2024-03-30T13:14:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fb9b8072f90 con 0x7fb9b806ea60 2024-03-30T13:14:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b8073270 con 0x7fb9b815eb10 2024-03-30T13:14:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb9b815a840 0x7fb9b815e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36906/0 (socket says 172.21.15.43:36906) 2024-03-30T13:14:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b6d9d640 1 -- 172.21.15.43:0/2287131974 learned_addr learned my addr 172.21.15.43:0/2287131974 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:14:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b759e640 1 --2- 172.21.15.43:0/2287131974 >> v2:172.21.15.43:3300/0 conn(0x7fb9b806ea60 0x7fb9b80b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b659c640 1 --2- 172.21.15.43:0/2287131974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815eb10 0x7fb9b80b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b5d9b640 1 -- 172.21.15.43:0/2287131974 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738957409 0 0) 0x7fb9b8072cb0 con 0x7fb9b815a840 2024-03-30T13:14:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b5d9b640 1 -- 172.21.15.43:0/2287131974 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9a0003530 con 0x7fb9b815a840 2024-03-30T13:14:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b759e640 1 -- 172.21.15.43:0/2287131974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815eb10 msgr2=0x7fb9b80b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:14:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b759e640 1 --2- 172.21.15.43:0/2287131974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815eb10 0x7fb9b80b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b759e640 1 -- 172.21.15.43:0/2287131974 >> v1:172.21.15.43:6789/0 conn(0x7fb9b815a840 legacy=0x7fb9b815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b759e640 1 -- 172.21.15.43:0/2287131974 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b8073550 con 0x7fb9b806ea60 2024-03-30T13:14:59.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b759e640 1 --2- 172.21.15.43:0/2287131974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80b3430 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009d90 tx=0x7fb9b001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e9df7ca2f7f9ad server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b5d9b640 1 -- 172.21.15.43:0/2287131974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00b2070 con 0x7fb9b806ea60 2024-03-30T13:14:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b5d9b640 1 -- 172.21.15.43:0/2287131974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9b00a57f0 con 0x7fb9b806ea60 2024-03-30T13:14:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9b5d9b640 1 -- 172.21.15.43:0/2287131974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b00a5bc0 con 0x7fb9b806ea60 2024-03-30T13:14:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 -- 172.21.15.43:0/2287131974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 msgr2=0x7fb9b80b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 --2- 172.21.15.43:0/2287131974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80b3430 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009d90 tx=0x7fb9b001c8e0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 -- 172.21.15.43:0/2287131974 shutdown_connections 2024-03-30T13:14:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 --2- 172.21.15.43:0/2287131974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80b3430 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 --2- 172.21.15.43:0/2287131974 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815eb10 0x7fb9b80b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 -- 172.21.15.43:0/2287131974 >> 172.21.15.43:0/2287131974 conn(0x7fb9b80cfee0 msgr2=0x7fb9b80d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 -- 172.21.15.43:0/2287131974 shutdown_connections 2024-03-30T13:14:59.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.713+0000 7fb9bd183640 1 -- 172.21.15.43:0/2287131974 wait complete. 2024-03-30T13:14:59.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 Processor -- start 2024-03-30T13:14:59.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 -- start start 2024-03-30T13:14:59.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815a840 0x7fb9b80ce450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b815eb10 0x7fb9b814c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b80b3af0 con 0x7fb9b806ea60 2024-03-30T13:14:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fb9b80b3970 con 0x7fb9b815eb10 2024-03-30T13:14:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fb9b80b3c70 con 0x7fb9b815a840 2024-03-30T13:14:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36896/0 (socket says 172.21.15.43:36896) 2024-03-30T13:14:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 -- 172.21.15.43:0/1680948249 learned_addr learned my addr 172.21.15.43:0/1680948249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:14:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b659c640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815a840 0x7fb9b80ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b759e640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b815eb10 0x7fb9b814c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 -- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815a840 msgr2=0x7fb9b80ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815a840 0x7fb9b80ce450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 -- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b815eb10 msgr2=0x7fb9b814c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b815eb10 0x7fb9b814c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 -- 172.21.15.43:0/1680948249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9b80cd3b0 con 0x7fb9b806ea60 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9b6d9d640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80ccdd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c450 tx=0x7fb9a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb99f7fe640 1 -- 172.21.15.43:0/1680948249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8017070 con 0x7fb9b806ea60 2024-03-30T13:14:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb99f7fe640 1 -- 172.21.15.43:0/1680948249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9a8005cf0 con 0x7fb9b806ea60 2024-03-30T13:14:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb99f7fe640 1 -- 172.21.15.43:0/1680948249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8007290 con 0x7fb9b806ea60 2024-03-30T13:14:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9b80cd690 con 0x7fb9b806ea60 2024-03-30T13:14:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9b80cdc50 con 0x7fb9b806ea60 2024-03-30T13:14:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.717+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9b8068890 con 0x7fb9b806ea60 2024-03-30T13:14:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.721+0000 7fb99f7fe640 1 -- 172.21.15.43:0/1680948249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fb9a801a070 con 0x7fb9b806ea60 2024-03-30T13:14:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.721+0000 7fb99f7fe640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9a4067fc0 0x7fb9a406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:14:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.721+0000 7fb99f7fe640 1 -- 172.21.15.43:0/1680948249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7fb9a8085710 con 0x7fb9b806ea60 2024-03-30T13:14:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.721+0000 7fb9b659c640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9a4067fc0 0x7fb9a406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:14:59.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.721+0000 7fb9b659c640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9a4067fc0 0x7fb9a406a480 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb9b815ddf0 tx=0x7fb9ac008040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:14:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.725+0000 7fb99f7fe640 1 -- 172.21.15.43:0/1680948249 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fb9a8056030 con 0x7fb9b806ea60 2024-03-30T13:14:59.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fb9b80719d0 con 0x7fb9a4067fc0 2024-03-30T13:14:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb99f7fe640 1 -- 172.21.15.43:0/1680948249 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb9b80719d0 con 0x7fb9a4067fc0 2024-03-30T13:14:59.890 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 6 to deep-scrub 2024-03-30T13:14:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9a4067fc0 msgr2=0x7fb9a406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9a4067fc0 0x7fb9a406a480 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb9b815ddf0 tx=0x7fb9ac008040 comp rx=0 tx=0).stop 2024-03-30T13:14:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 msgr2=0x7fb9b80ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:14:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80ccdd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb9a800c450 tx=0x7fb9a800c910 comp rx=0 tx=0).stop 2024-03-30T13:14:59.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 shutdown_connections 2024-03-30T13:14:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fb9a4067fc0 0x7fb9a406a480 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fb9b815eb10 0x7fb9b814c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fb9b815a840 0x7fb9b80ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 --2- 172.21.15.43:0/1680948249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b806ea60 0x7fb9b80ccdd0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:14:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 >> 172.21.15.43:0/1680948249 conn(0x7fb9b80cfee0 msgr2=0x7fb9b8071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:14:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 shutdown_connections 2024-03-30T13:14:59.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:14:59.885+0000 7fb9bd183640 1 -- 172.21.15.43:0/1680948249 wait complete. 2024-03-30T13:14:59.908 INFO:tasks.ceph:Scrubbing osd.7 2024-03-30T13:14:59.909 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-30T13:15:00.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1fb06640 1 Processor -- start 2024-03-30T13:15:00.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1fb06640 1 -- start start 2024-03-30T13:15:00.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1fb06640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f1815a920 0x7f6f1806d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1fb06640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f180ca470 0x7f6f180cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1fb06640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f18072c40 con 0x7f6f180719d0 2024-03-30T13:15:00.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1fb06640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f6f1806cd80 con 0x7f6f180ca470 2024-03-30T13:15:00.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1fb06640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1815ad00 con 0x7f6f1815a920 2024-03-30T13:15:00.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1d0a1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f180ca470 0x7f6f180cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1d0a1640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f6f180ca470 0x7f6f180cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.113:3300/0 says I am v2:172.21.15.43:42994/0 (socket says 172.21.15.43:42994) 2024-03-30T13:15:00.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/1061636438 learned_addr learned my addr 172.21.15.43:0/1061636438 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:00.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.033+0000 7f6f1d8a2640 1 --2- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f1815a920 0x7f6f1806d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f1815a920 msgr2=0x7f6f1806d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:15:00.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1d0a1640 1 --2- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f1815a920 0x7f6f1806d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/1061636438 >> v1:172.21.15.113:6789/0 conn(0x7f6f180719d0 legacy=0x7f6f1806c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/1061636438 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f180ce7a0 con 0x7f6f180ca470 2024-03-30T13:15:00.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1d0a1640 1 --2- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180ca470 0x7f6f180cad80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009870 tx=0x7f6f0c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9592896738e2e70 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1c8a0640 1 -- 172.21.15.43:0/1061636438 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0b0070 con 0x7f6f180ca470 2024-03-30T13:15:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1c8a0640 1 -- 172.21.15.43:0/1061636438 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0c0ab600 con 0x7f6f180ca470 2024-03-30T13:15:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1c8a0640 1 -- 172.21.15.43:0/1061636438 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0ab9d0 con 0x7f6f180ca470 2024-03-30T13:15:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 -- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180ca470 msgr2=0x7f6f180cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 --2- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180ca470 0x7f6f180cad80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009870 tx=0x7f6f0c01c4d0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 -- 172.21.15.43:0/1061636438 shutdown_connections 2024-03-30T13:15:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 --2- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f180ca470 0x7f6f180cad80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 --2- 172.21.15.43:0/1061636438 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f1815a920 0x7f6f1806d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 -- 172.21.15.43:0/1061636438 >> 172.21.15.43:0/1061636438 conn(0x7f6f180b1b80 msgr2=0x7f6f180b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 -- 172.21.15.43:0/1061636438 shutdown_connections 2024-03-30T13:15:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.037+0000 7f6f1fb06640 1 -- 172.21.15.43:0/1061636438 wait complete. 2024-03-30T13:15:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 Processor -- start 2024-03-30T13:15:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 -- start start 2024-03-30T13:15:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180719d0 0x7f6f1815c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f180ca470 0x7f6f180c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f1815a920 0x7f6f180c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f6f180cfb20 con 0x7f6f180ca470 2024-03-30T13:15:00.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f6f180cf9a0 con 0x7f6f1815a920 2024-03-30T13:15:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f6f180cfca0 con 0x7f6f180719d0 2024-03-30T13:15:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f180ca470 0x7f6f180c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f180ca470 0x7f6f180c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36900/0 (socket says 172.21.15.43:36900) 2024-03-30T13:15:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/2743983259 learned_addr learned my addr 172.21.15.43:0/2743983259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1e0a3640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f1815a920 0x7f6f180c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d8a2640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180719d0 0x7f6f1815c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180719d0 msgr2=0x7f6f1815c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f6f180719d0 0x7f6f1815c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f1815a920 msgr2=0x7f6f180c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f6f1815a920 0x7f6f180c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 -- 172.21.15.43:0/2743983259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f180b5ba0 con 0x7f6f180ca470 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1d0a1640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f180ca470 0x7f6f180c9200 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0ae040 tx=0x7f6f0c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0b0040 con 0x7f6f180ca470 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f0c0bc070 con 0x7f6f180ca470 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1815cd70 con 0x7f6f180ca470 2024-03-30T13:15:00.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0aa9f0 con 0x7f6f180ca470 2024-03-30T13:15:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1815d330 con 0x7f6f180ca470 2024-03-30T13:15:00.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.041+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f1815cb20 con 0x7f6f180ca470 2024-03-30T13:15:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f6f0c0aab90 con 0x7f6f180ca470 2024-03-30T13:15:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f167fc640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f04067fc0 0x7f6f0406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f6f0c121420 con 0x7f6f180ca470 2024-03-30T13:15:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f167fc640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7f6f0406e460 0x7f6f040708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 --> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f0c0b8ec0 con 0x7f6f0406e460 2024-03-30T13:15:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f6f0c1218b0 con 0x7f6f180ca470 2024-03-30T13:15:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f1d8a2640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f04067fc0 0x7f6f0406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f1e0a3640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7f6f0406e460 0x7f6f040708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f1e0a3640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7f6f0406e460 0x7f6f040708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f1d8a2640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f04067fc0 0x7f6f0406a480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f1814c680 tx=0x7f6f0800b040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.045+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== osd.7 v2:172.21.15.113:6808/3435220755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f0c0b8ec0 con 0x7f6f0406e460 2024-03-30T13:15:00.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 --> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6f1815d750 con 0x7f6f0406e460 2024-03-30T13:15:00.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f167fc640 1 -- 172.21.15.43:0/2743983259 <== osd.7 v2:172.21.15.113:6808/3435220755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6f1815d750 con 0x7f6f0406e460 2024-03-30T13:15:00.066 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-30T13:15:00.066 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-30T13:15:00.067 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-30T13:15:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7f6f0406e460 msgr2=0x7f6f040708a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6808/3435220755,v1:172.21.15.113:6809/3435220755] conn(0x7f6f0406e460 0x7f6f040708a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f04067fc0 msgr2=0x7f6f0406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f6f04067fc0 0x7f6f0406a480 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f1814c680 tx=0x7f6f0800b040 comp rx=0 tx=0).stop 2024-03-30T13:15:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f180ca470 msgr2=0x7f6f180c9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 --2- 172.21.15.43:0/2743983259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f180ca470 0x7f6f180c9200 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0ae040 tx=0x7f6f0c01ad10 comp rx=0 tx=0).stop 2024-03-30T13:15:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1e0a3640 1 -- 172.21.15.43:0/2743983259 reap_dead start 2024-03-30T13:15:00.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 shutdown_connections 2024-03-30T13:15:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 >> 172.21.15.43:0/2743983259 conn(0x7f6f180b1b80 msgr2=0x7f6f1806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 shutdown_connections 2024-03-30T13:15:00.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.061+0000 7f6f1fb06640 1 -- 172.21.15.43:0/2743983259 wait complete. 2024-03-30T13:15:00.082 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-30T13:15:00.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f364a19b640 1 Processor -- start 2024-03-30T13:15:00.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f364a19b640 1 -- start start 2024-03-30T13:15:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f364a19b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36440b3720 0x7f36440b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f364a19b640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f364406bf00 0x7f364406c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f364a19b640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36440ad8e0 con 0x7f364415e370 2024-03-30T13:15:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f364a19b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f36440b4550 con 0x7f36440b3720 2024-03-30T13:15:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f364a19b640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f36440b40c0 con 0x7f364406bf00 2024-03-30T13:15:00.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3642ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36440b3720 0x7f36440b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3642ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f36440b3720 0x7f36440b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36908/0 (socket says 172.21.15.43:36908) 2024-03-30T13:15:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3642ffd640 1 -- 172.21.15.43:0/2620078555 learned_addr learned my addr 172.21.15.43:0/2620078555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3643fff640 1 --2- 172.21.15.43:0/2620078555 >> v2:172.21.15.113:3300/0 conn(0x7f364406bf00 0x7f364406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3642ffd640 1 -- 172.21.15.43:0/2620078555 >> v2:172.21.15.113:3300/0 conn(0x7f364406bf00 msgr2=0x7f364406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:15:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3642ffd640 1 --2- 172.21.15.43:0/2620078555 >> v2:172.21.15.113:3300/0 conn(0x7f364406bf00 0x7f364406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3642ffd640 1 -- 172.21.15.43:0/2620078555 >> v1:172.21.15.113:6789/0 conn(0x7f364415e370 legacy=0x7f36440afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.189+0000 7f3642ffd640 1 -- 172.21.15.43:0/2620078555 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364415e750 con 0x7f36440b3720 2024-03-30T13:15:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f3642ffd640 1 --2- 172.21.15.43:0/2620078555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f36440b3b80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f36440b23b0 tx=0x7f362c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c10626b5f1bd0ca server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f36427fc640 1 -- 172.21.15.43:0/2620078555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0ab610 con 0x7f36440b3720 2024-03-30T13:15:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f36427fc640 1 -- 172.21.15.43:0/2620078555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c0b0070 con 0x7f36440b3720 2024-03-30T13:15:00.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f36427fc640 1 -- 172.21.15.43:0/2620078555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0ab9d0 con 0x7f36440b3720 2024-03-30T13:15:00.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 -- 172.21.15.43:0/2620078555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 msgr2=0x7f36440b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 --2- 172.21.15.43:0/2620078555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f36440b3b80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f36440b23b0 tx=0x7f362c01c520 comp rx=0 tx=0).stop 2024-03-30T13:15:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 -- 172.21.15.43:0/2620078555 shutdown_connections 2024-03-30T13:15:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 --2- 172.21.15.43:0/2620078555 >> v2:172.21.15.113:3300/0 conn(0x7f364406bf00 0x7f364406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 --2- 172.21.15.43:0/2620078555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f36440b3b80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 -- 172.21.15.43:0/2620078555 >> 172.21.15.43:0/2620078555 conn(0x7f36440b2b30 msgr2=0x7f36440d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 -- 172.21.15.43:0/2620078555 shutdown_connections 2024-03-30T13:15:00.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.193+0000 7f364a19b640 1 -- 172.21.15.43:0/2620078555 wait complete. 2024-03-30T13:15:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 Processor -- start 2024-03-30T13:15:00.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 -- start start 2024-03-30T13:15:00.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364406bf00 0x7f36440ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f3644144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364415e370 0x7f3644152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f364406e4e0 con 0x7f36440b3720 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f364406b700 con 0x7f364415e370 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f364406e660 con 0x7f364406bf00 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f3644144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f3644144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36910/0 (socket says 172.21.15.43:36910) 2024-03-30T13:15:00.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 -- 172.21.15.43:0/463558921 learned_addr learned my addr 172.21.15.43:0/463558921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f36437fe640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364406bf00 0x7f36440ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3643fff640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364415e370 0x7f3644152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 -- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364406bf00 msgr2=0x7f36440ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364406bf00 0x7f36440ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 -- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364415e370 msgr2=0x7f3644152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364415e370 0x7f3644152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 -- 172.21.15.43:0/463558921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364415cdf0 con 0x7f36440b3720 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f36437fe640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364406bf00 0x7f36440ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f3642ffd640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f3644144fd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f362c01abd0 tx=0x7f362c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f363bfff640 1 -- 172.21.15.43:0/463558921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0be5f0 con 0x7f36440b3720 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f363bfff640 1 -- 172.21.15.43:0/463558921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f362c0be790 con 0x7f36440b3720 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36440cd140 con 0x7f36440b3720 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f363bfff640 1 -- 172.21.15.43:0/463558921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0beb60 con 0x7f36440b3720 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36440cd700 con 0x7f36440b3720 2024-03-30T13:15:00.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3644068890 con 0x7f36440b3720 2024-03-30T13:15:00.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.197+0000 7f363bfff640 1 -- 172.21.15.43:0/463558921 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f362c0b43f0 con 0x7f36440b3720 2024-03-30T13:15:00.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.201+0000 7f363bfff640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3624067fc0 0x7f362406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.201+0000 7f363bfff640 1 -- 172.21.15.43:0/463558921 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f362c120a10 con 0x7f36440b3720 2024-03-30T13:15:00.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.201+0000 7f36437fe640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3624067fc0 0x7f362406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.201+0000 7f36437fe640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3624067fc0 0x7f362406a480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f363c0012f0 tx=0x7f363c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.205+0000 7f363bfff640 1 -- 172.21.15.43:0/463558921 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f362c0c4060 con 0x7f36440b3720 2024-03-30T13:15:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.361+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f36440d0060 con 0x7f3624067fc0 2024-03-30T13:15:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.361+0000 7f363bfff640 1 -- 172.21.15.43:0/463558921 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f36440d0060 con 0x7f3624067fc0 2024-03-30T13:15:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 7 to deep-scrub 2024-03-30T13:15:00.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3624067fc0 msgr2=0x7f362406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3624067fc0 0x7f362406a480 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f363c0012f0 tx=0x7f363c009040 comp rx=0 tx=0).stop 2024-03-30T13:15:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 msgr2=0x7f3644144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f3644144fd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f362c01abd0 tx=0x7f362c01ad60 comp rx=0 tx=0).stop 2024-03-30T13:15:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 shutdown_connections 2024-03-30T13:15:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f3624067fc0 0x7f362406a480 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f364415e370 0x7f3644152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36440b3720 0x7f3644144fd0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 --2- 172.21.15.43:0/463558921 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f364406bf00 0x7f36440ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 >> 172.21.15.43:0/463558921 conn(0x7f36440b2b30 msgr2=0x7f364406dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 shutdown_connections 2024-03-30T13:15:00.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.365+0000 7f364a19b640 1 -- 172.21.15.43:0/463558921 wait complete. 2024-03-30T13:15:00.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-30T13:15:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb9a7d640 1 Processor -- start 2024-03-30T13:15:00.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb9a7d640 1 -- start start 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb9a7d640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7f7cb406cd80 0x7f7cb4144650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb9a7d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb40b4ba0 con 0x7f7cb40b2ef0 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb9a7d640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cb40b33f0 con 0x7f7cb4144270 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb9a7d640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7f7cb4144fa0 con 0x7f7cb406cd80 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cabfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7cb40b2ef0 0x7f7cb406c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36920/0 (socket says 172.21.15.43:36920) 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cabfff640 1 -- 172.21.15.43:0/2371828132 learned_addr learned my addr 172.21.15.43:0/2371828132 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb3fff640 1 --2- 172.21.15.43:0/2371828132 >> v2:172.21.15.113:3300/0 conn(0x7f7cb406cd80 0x7f7cb4144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/2371828132 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3046532955 0 0) 0x7f7cb40b4ba0 con 0x7f7cb40b2ef0 2024-03-30T13:15:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/2371828132 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ca0003530 con 0x7f7cb40b2ef0 2024-03-30T13:15:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/2371828132 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2469676496 0 0) 0x7f7ca0003530 con 0x7f7cb40b2ef0 2024-03-30T13:15:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb3fff640 1 -- 172.21.15.43:0/2371828132 >> v1:172.21.15.113:6789/0 conn(0x7f7cb4144270 legacy=0x7f7cb40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb3fff640 1 -- 172.21.15.43:0/2371828132 >> v1:172.21.15.43:6789/0 conn(0x7f7cb40b2ef0 legacy=0x7f7cb406c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.505+0000 7f7cb3fff640 1 -- 172.21.15.43:0/2371828132 --> v2:172.21.15.113:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb40b36d0 con 0x7f7cb406cd80 2024-03-30T13:15:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb3fff640 1 --2- 172.21.15.43:0/2371828132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb406cd80 0x7f7cb4144650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009d90 tx=0x7f7cac01c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fabc4508d1c948e server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/2371828132 <== mon.1 v2:172.21.15.113:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cac0b2070 con 0x7f7cb406cd80 2024-03-30T13:15:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/2371828132 <== mon.1 v2:172.21.15.113:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cac0a5610 con 0x7f7cb406cd80 2024-03-30T13:15:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb2ffd640 1 -- 172.21.15.43:0/2371828132 <== mon.1 v2:172.21.15.113:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0016a0 con 0x7f7cb406cd80 2024-03-30T13:15:00.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/2371828132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb406cd80 msgr2=0x7f7cb4144650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/2371828132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb406cd80 0x7f7cb4144650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7cac009d90 tx=0x7f7cac01c7d0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/2371828132 shutdown_connections 2024-03-30T13:15:00.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/2371828132 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb406cd80 0x7f7cb4144650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/2371828132 >> 172.21.15.43:0/2371828132 conn(0x7f7cb40cfee0 msgr2=0x7f7cb40d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:00.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/2371828132 shutdown_connections 2024-03-30T13:15:00.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/2371828132 wait complete. 2024-03-30T13:15:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 Processor -- start 2024-03-30T13:15:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- start start 2024-03-30T13:15:00.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb406cd80 0x7f7cb415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb40b2ef0 0x7f7cb413de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cb4144270 0x7f7cb40cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb40cde90 con 0x7f7cb406cd80 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f7cb40cdd10 con 0x7f7cb40b2ef0 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb9a7d640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f7cb413e380 con 0x7f7cb4144270 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb406cd80 0x7f7cb415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb406cd80 0x7f7cb415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36914/0 (socket says 172.21.15.43:36914) 2024-03-30T13:15:00.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb37fe640 1 -- 172.21.15.43:0/1659436156 learned_addr learned my addr 172.21.15.43:0/1659436156 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cabfff640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb40b2ef0 0x7f7cb413de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.509+0000 7f7cb3fff640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cb4144270 0x7f7cb40cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb37fe640 1 -- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cb4144270 msgr2=0x7f7cb40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb37fe640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cb4144270 0x7f7cb40cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb37fe640 1 -- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb40b2ef0 msgr2=0x7f7cb413de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb37fe640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb40b2ef0 0x7f7cb413de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb37fe640 1 -- 172.21.15.43:0/1659436156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cb415a8f0 con 0x7f7cb406cd80 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb37fe640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb406cd80 0x7f7cb415a030 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c500 tx=0x7f7ca400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb0ff9640 1 -- 172.21.15.43:0/1659436156 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4016020 con 0x7f7cb406cd80 2024-03-30T13:15:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb0ff9640 1 -- 172.21.15.43:0/1659436156 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ca4005220 con 0x7f7cb406cd80 2024-03-30T13:15:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cb415a610 con 0x7f7cb406cd80 2024-03-30T13:15:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb0ff9640 1 -- 172.21.15.43:0/1659436156 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4007030 con 0x7f7cb406cd80 2024-03-30T13:15:00.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cb413d640 con 0x7f7cb406cd80 2024-03-30T13:15:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cb4068890 con 0x7f7cb406cd80 2024-03-30T13:15:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb0ff9640 1 -- 172.21.15.43:0/1659436156 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f7ca40071f0 con 0x7f7cb406cd80 2024-03-30T13:15:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb0ff9640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7c94067fc0 0x7f7c9406a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cb0ff9640 1 -- 172.21.15.43:0/1659436156 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f7ca4085d60 con 0x7f7cb406cd80 2024-03-30T13:15:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cabfff640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7c94067fc0 0x7f7c9406a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.513+0000 7f7cabfff640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7c94067fc0 0x7f7c9406a480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7cb413e6f0 tx=0x7f7c9c009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.517+0000 7f7cb0ff9640 1 -- 172.21.15.43:0/1659436156 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f7ca4020020 con 0x7f7cb406cd80 2024-03-30T13:15:00.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.677+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 --> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7cb40722d0 con 0x7f7c94067fc0 2024-03-30T13:15:00.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.677+0000 7f7cb0ff9640 1 -- 172.21.15.43:0/1659436156 <== mgr.4106 v2:172.21.15.113:6832/153501405 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13450 (secure 0 0 0) 0x7f7cb40722d0 con 0x7f7c94067fc0 2024-03-30T13:15:00.684 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:15:00.684 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-30T13:15:00.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7c94067fc0 msgr2=0x7f7c9406a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7c94067fc0 0x7f7c9406a480 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7cb413e6f0 tx=0x7f7c9c009040 comp rx=0 tx=0).stop 2024-03-30T13:15:00.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb406cd80 msgr2=0x7f7cb415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb406cd80 0x7f7cb415a030 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400c500 tx=0x7f7ca400c9c0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 shutdown_connections 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f7c94067fc0 0x7f7c9406a480 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f7cb4144270 0x7f7cb40cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f7cb40b2ef0 0x7f7cb413de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 --2- 172.21.15.43:0/1659436156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7cb406cd80 0x7f7cb415a030 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 >> 172.21.15.43:0/1659436156 conn(0x7f7cb40cfee0 msgr2=0x7f7cb406b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 shutdown_connections 2024-03-30T13:15:00.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.681+0000 7f7cb9a7d640 1 -- 172.21.15.43:0/1659436156 wait complete. 2024-03-30T13:15:00.703 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-03-30T13:14:59.920825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":271752,"kb_used_data":5640,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754702968,"statfs":{"total":773094113280,"available":772815839232,"internally_reserved":0,"allocated":5775360,"data_stored":3191736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":272354866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-173859201,"num_objects":-93,"num_object_clones":0,"num_object_copies":-186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-93,"num_whiteouts":0,"num_read":-100,"num_read_kb":0,"num_write":-543,"num_write_kb":-172371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.756858"},"pg_stats":[{"pgid":"1.0","version":"49'119","reported_seq":233,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-30T13:14:57.597046+0000","last_change":"2024-03-30T13:14:57.597087+0000","last_active":"2024-03-30T13:14:57.597046+0000","last_peered":"2024-03-30T13:14:57.597046+0000","last_clean":"2024-03-30T13:14:57.597046+0000","last_became_active":"2024-03-30T13:10:45.553184+0000","last_became_peered":"2024-03-30T13:10:45.553184+0000","last_unstale":"2024-03-30T13:14:57.597046+0000","last_undegraded":"2024-03-30T13:14:57.597046+0000","last_fullsized":"2024-03-30T13:14:57.597046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'119","last_scrub_stamp":"2024-03-30T13:14:57.596947+0000","last_deep_scrub":"49'119","last_deep_scrub_stamp":"2024-03-30T13:14:57.596947+0000","last_clean_scrub_stamp":"2024-03-30T13:14:57.596947+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-31T17:04:04.840513+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31740,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31740,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35908,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335932,"statfs":{"total":96636764160,"available":96599994368,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35900,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335940,"statfs":{"total":96636764160,"available":96600002560,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36484,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335356,"statfs":{"total":96636764160,"available":96599404544,"internally_reserved":0,"allocated":1167360,"data_stored":841743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509494,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35908,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335932,"statfs":{"total":96636764160,"available":96599994368,"internally_reserved":0,"allocated":573440,"data_stored":251375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1167360,"data_stored":841743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-30T13:15:00.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-30T13:15:00.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0eed37640 1 Processor -- start 2024-03-30T13:15:00.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0eed37640 1 -- start start 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0eed37640 1 --2- >> v2:172.21.15.113:3300/0 conn(0x7fe0e80ae960 0x7fe0e80b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0eed37640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0e815e6d0 0x7fe0e806c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0eed37640 1 -- --> v1:172.21.15.113:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0e80b58c0 con 0x7fe0e815e2f0 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0eed37640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e80cd060 con 0x7fe0e815e6d0 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0eed37640 1 -- --> v2:172.21.15.113:3300/0 -- mon_getmap magic: 0 -- 0x7fe0e806b660 con 0x7fe0e80ae960 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0ed2d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0e815e6d0 0x7fe0e806c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0ed2d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe0e815e6d0 0x7fe0e806c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36920/0 (socket says 172.21.15.43:36920) 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0ed2d4640 1 -- 172.21.15.43:0/4075176407 learned_addr learned my addr 172.21.15.43:0/4075176407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:00.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0e7fff640 1 --2- 172.21.15.43:0/4075176407 >> v2:172.21.15.113:3300/0 conn(0x7fe0e80ae960 0x7fe0e80b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0ed2d4640 1 -- 172.21.15.43:0/4075176407 >> v2:172.21.15.113:3300/0 conn(0x7fe0e80ae960 msgr2=0x7fe0e80b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:15:00.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0ed2d4640 1 --2- 172.21.15.43:0/4075176407 >> v2:172.21.15.113:3300/0 conn(0x7fe0e80ae960 0x7fe0e80b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0ed2d4640 1 -- 172.21.15.43:0/4075176407 >> v1:172.21.15.113:6789/0 conn(0x7fe0e815e2f0 legacy=0x7fe0e80ae250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:15:00.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.825+0000 7fe0ed2d4640 1 -- 172.21.15.43:0/4075176407 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e80b5ba0 con 0x7fe0e815e6d0 2024-03-30T13:15:00.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0ed2d4640 1 --2- 172.21.15.43:0/4075176407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e6d0 0x7fe0e806c0b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe0e806db60 tx=0x7fe0e001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c91ec2b0e08f672 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0e77fe640 1 -- 172.21.15.43:0/4075176407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00a5720 con 0x7fe0e815e6d0 2024-03-30T13:15:00.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0e77fe640 1 -- 172.21.15.43:0/4075176407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e00a58c0 con 0x7fe0e815e6d0 2024-03-30T13:15:00.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0e77fe640 1 -- 172.21.15.43:0/4075176407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00a5c90 con 0x7fe0e815e6d0 2024-03-30T13:15:00.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- 172.21.15.43:0/4075176407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e6d0 msgr2=0x7fe0e806c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4075176407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e6d0 0x7fe0e806c0b0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe0e806db60 tx=0x7fe0e001c860 comp rx=0 tx=0).stop 2024-03-30T13:15:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- 172.21.15.43:0/4075176407 shutdown_connections 2024-03-30T13:15:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4075176407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e6d0 0x7fe0e806c0b0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4075176407 >> v2:172.21.15.113:3300/0 conn(0x7fe0e80ae960 0x7fe0e80b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- 172.21.15.43:0/4075176407 >> 172.21.15.43:0/4075176407 conn(0x7fe0e80d0c90 msgr2=0x7fe0e80d10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- 172.21.15.43:0/4075176407 shutdown_connections 2024-03-30T13:15:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- 172.21.15.43:0/4075176407 wait complete. 2024-03-30T13:15:00.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 Processor -- start 2024-03-30T13:15:00.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- start start 2024-03-30T13:15:00.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0e80ae960 0x7fe0e814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e2f0 0x7fe0e8153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0e815e6d0 0x7fe0e814ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e816b480 con 0x7fe0e815e2f0 2024-03-30T13:15:00.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e816b600 con 0x7fe0e80ae960 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0eed37640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e816b780 con 0x7fe0e815e6d0 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e2f0 0x7fe0e8153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e2f0 0x7fe0e8153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36926/0 (socket says 172.21.15.43:36926) 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.829+0000 7fe0e7fff640 1 -- 172.21.15.43:0/4226226952 learned_addr learned my addr 172.21.15.43:0/4226226952 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0ecad3640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0e80ae960 0x7fe0e814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0ed2d4640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0e815e6d0 0x7fe0e814ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e7fff640 1 -- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0e815e6d0 msgr2=0x7fe0e814ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e7fff640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0e815e6d0 0x7fe0e814ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e7fff640 1 -- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0e80ae960 msgr2=0x7fe0e814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e7fff640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0e80ae960 0x7fe0e814fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e7fff640 1 -- 172.21.15.43:0/4226226952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e8151080 con 0x7fe0e815e2f0 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0ecad3640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0e80ae960 0x7fe0e814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e7fff640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e2f0 0x7fe0e8153b50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d7d0 tx=0x7fe0d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400b6c0 con 0x7fe0e815e2f0 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0d4019070 con 0x7fe0e815e2f0 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e81579f0 con 0x7fe0e815e2f0 2024-03-30T13:15:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0d400bac0 con 0x7fe0e815e2f0 2024-03-30T13:15:00.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e8150ac0 con 0x7fe0e815e2f0 2024-03-30T13:15:00.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e8068900 con 0x7fe0e815e2f0 2024-03-30T13:15:00.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7fe0d400bc60 con 0x7fe0e815e2f0 2024-03-30T13:15:00.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e57fa640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0cc067ff0 0x7fe0cc06a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:00.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7fe0d4086840 con 0x7fe0e815e2f0 2024-03-30T13:15:00.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0ecad3640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0cc067ff0 0x7fe0cc06a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:00.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.833+0000 7fe0ecad3640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0cc067ff0 0x7fe0cc06a4b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003e40 tx=0x7fe0dc009040 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:00.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.837+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7fe0d4057160 con 0x7fe0e815e2f0 2024-03-30T13:15:01.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:00.997+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fe0e8070680 con 0x7fe0e815e2f0 2024-03-30T13:15:01.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fe0d4056a60 con 0x7fe0e815e2f0 2024-03-30T13:15:01.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0e57fa640 1 -- 172.21.15.43:0/4226226952 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe0d405ae10 con 0x7fe0e815e2f0 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0cc067ff0 msgr2=0x7fe0cc06a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0cc067ff0 0x7fe0cc06a4b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc003e40 tx=0x7fe0dc009040 comp rx=0 tx=0).stop 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e2f0 msgr2=0x7fe0e8153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e2f0 0x7fe0e8153b50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe0d400d7d0 tx=0x7fe0d400dc90 comp rx=0 tx=0).stop 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 shutdown_connections 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7fe0cc067ff0 0x7fe0cc06a4b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7fe0e815e6d0 0x7fe0e814ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe0e815e2f0 0x7fe0e8153b50 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 --2- 172.21.15.43:0/4226226952 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7fe0e80ae960 0x7fe0e814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 >> 172.21.15.43:0/4226226952 conn(0x7fe0e80d0c90 msgr2=0x7fe0e8162610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:01.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 shutdown_connections 2024-03-30T13:15:01.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.001+0000 7fe0eed37640 1 -- 172.21.15.43:0/4226226952 wait complete. 2024-03-30T13:15:01.023 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-30T13:15:01.024 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-30T13:15:01.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8f0604b640 1 Processor -- start 2024-03-30T13:15:01.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8f0604b640 1 -- start start 2024-03-30T13:15:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8f0604b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f0013f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8f0604b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f00141d70 0x7f8f000cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8f0604b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f00070030 con 0x7f8f000cfee0 2024-03-30T13:15:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8f0604b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 -- 0x7f8f00142820 con 0x7f8f00141d70 2024-03-30T13:15:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8f0604b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8f0013f620 con 0x7f8f00141990 2024-03-30T13:15:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f00141d70 0x7f8f000cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:01.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8f000cfee0 0x7f8f0015fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:36932/0 (socket says 172.21.15.43:36932) 2024-03-30T13:15:01.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f00141d70 0x7f8f000cf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36934/0 (socket says 172.21.15.43:36934) 2024-03-30T13:15:01.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efffff640 1 -- 172.21.15.43:0/1887921700 learned_addr learned my addr 172.21.15.43:0/1887921700 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-30T13:15:01.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8eff7fe640 1 --2- 172.21.15.43:0/1887921700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f0013f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:01.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efeffd640 1 -- 172.21.15.43:0/1887921700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 msgr2=0x7f8f0013f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-30T13:15:01.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efeffd640 1 --2- 172.21.15.43:0/1887921700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f0013f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efeffd640 1 -- 172.21.15.43:0/1887921700 >> v1:172.21.15.43:6789/0 conn(0x7f8f000cfee0 legacy=0x7f8f0015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.145+0000 7f8efeffd640 1 -- 172.21.15.43:0/1887921700 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0013f930 con 0x7f8f00141d70 2024-03-30T13:15:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efeffd640 1 --2- 172.21.15.43:0/1887921700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f00141d70 0x7f8f000cf9a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8f0013f800 tx=0x7f8ef401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f817ebb2748a49b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efe7fc640 1 -- 172.21.15.43:0/1887921700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40b2070 con 0x7f8f00141d70 2024-03-30T13:15:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efe7fc640 1 -- 172.21.15.43:0/1887921700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8ef40ad640 con 0x7f8f00141d70 2024-03-30T13:15:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efe7fc640 1 -- 172.21.15.43:0/1887921700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40ada30 con 0x7f8f00141d70 2024-03-30T13:15:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- 172.21.15.43:0/1887921700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f00141d70 msgr2=0x7f8f000cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 --2- 172.21.15.43:0/1887921700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f00141d70 0x7f8f000cf9a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f8f0013f800 tx=0x7f8ef401c470 comp rx=0 tx=0).stop 2024-03-30T13:15:01.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- 172.21.15.43:0/1887921700 shutdown_connections 2024-03-30T13:15:01.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 --2- 172.21.15.43:0/1887921700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f00141d70 0x7f8f000cf9a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 --2- 172.21.15.43:0/1887921700 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f0013f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- 172.21.15.43:0/1887921700 >> 172.21.15.43:0/1887921700 conn(0x7f8f000b2c70 msgr2=0x7f8f000b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:01.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- 172.21.15.43:0/1887921700 shutdown_connections 2024-03-30T13:15:01.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- 172.21.15.43:0/1887921700 wait complete. 2024-03-30T13:15:01.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 Processor -- start 2024-03-30T13:15:01.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- start start 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f000cfee0 0x7f8f0013e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 --2- >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f001f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 --2- >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f00141d70 0x7f8f001443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0006eb00 con 0x7f8f000cfee0 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- --> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0006e820 con 0x7f8f00141d70 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8f0604b640 1 -- --> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] -- mon_getmap magic: 0 -- 0x7f8f001421d0 con 0x7f8f00141990 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f000cfee0 0x7f8f0013e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:01.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f000cfee0 0x7f8f0013e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36946/0 (socket says 172.21.15.43:36946) 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 -- 172.21.15.43:0/3927791298 learned_addr learned my addr 172.21.15.43:0/3927791298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efeffd640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f001f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 -- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 msgr2=0x7f8f001f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f001f52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efffff640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f00141d70 0x7f8f001443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 -- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f00141d70 msgr2=0x7f8f001443a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f00141d70 0x7f8f001443a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 -- 172.21.15.43:0/3927791298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f001f5cb0 con 0x7f8f000cfee0 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efeffd640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f001f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:15:01.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8efffff640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f00141d70 0x7f8f001443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.149+0000 7f8eff7fe640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f000cfee0 0x7f8f0013e770 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007ef0 tx=0x7f8ef000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8ee7fff640 1 -- 172.21.15.43:0/3927791298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0014620 con 0x7f8f000cfee0 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8ee7fff640 1 -- 172.21.15.43:0/3927791298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f8ef0019070 con 0x7f8f000cfee0 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f001f5f90 con 0x7f8f000cfee0 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8ee7fff640 1 -- 172.21.15.43:0/3927791298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0014a20 con 0x7f8f000cfee0 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f00143e80 con 0x7f8f000cfee0 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f00068890 con 0x7f8f000cfee0 2024-03-30T13:15:01.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8ee7fff640 1 -- 172.21.15.43:0/3927791298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) ==== 87803+0+0 (secure 0 0 0) 0x7f8ef0014bc0 con 0x7f8f000cfee0 2024-03-30T13:15:01.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8ee7fff640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8eec067fc0 0x7f8eec06a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-30T13:15:01.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8ee7fff640 1 -- 172.21.15.43:0/3927791298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5335+0+0 (secure 0 0 0) 0x7f8ef0086db0 con 0x7f8f000cfee0 2024-03-30T13:15:01.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8efeffd640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8eec067fc0 0x7f8eec06a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-30T13:15:01.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.153+0000 7f8efeffd640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8eec067fc0 0x7f8eec06a480 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40af000 tx=0x7f8ef40a7070 comp rx=0 tx=0).ready entity=mgr.4106 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-30T13:15:01.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.157+0000 7f8ee7fff640 1 -- 172.21.15.43:0/3927791298 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179117 (secure 0 0 0) 0x7f8ef0057640 con 0x7f8f000cfee0 2024-03-30T13:15:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f00140fb0 con 0x7f8f000cfee0 2024-03-30T13:15:01.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8ee7fff640 1 -- 172.21.15.43:0/3927791298 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+3875 (secure 0 0 0) 0x7f8ef0056f40 con 0x7f8f000cfee0 2024-03-30T13:15:01.358 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-30T13:15:01.358 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4868,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/4271726137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4271726137},{"type":"v1","addr":"172.21.15.43:6834","nonce":4271726137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4871,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1748034672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1748034672},{"type":"v1","addr":"172.21.15.43:6835","nonce":1748034672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4877,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6835/3254700273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6834","nonce":3254700273},{"type":"v1","addr":"172.21.15.113:6835","nonce":3254700273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5696,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.113:6837/2093619648","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.113:6836","nonce":2093619648},{"type":"v1","addr":"172.21.15.113:6837","nonce":2093619648}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[]} 2024-03-30T13:15:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-03-30T13:15:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8eec067fc0 msgr2=0x7f8eec06a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8eec067fc0 0x7f8eec06a480 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40af000 tx=0x7f8ef40a7070 comp rx=0 tx=0).stop 2024-03-30T13:15:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f000cfee0 msgr2=0x7f8f0013e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-30T13:15:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f000cfee0 0x7f8f0013e770 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0007ef0 tx=0x7f8ef000e4b0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 shutdown_connections 2024-03-30T13:15:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:6832/153501405,v1:172.21.15.113:6833/153501405] conn(0x7f8eec067fc0 0x7f8eec06a480 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3300/0,v1:172.21.15.113:6789/0] conn(0x7f8f00141d70 0x7f8f001443a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.113:3301/0,v1:172.21.15.113:6790/0] conn(0x7f8f00141990 0x7f8f001f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 --2- 172.21.15.43:0/3927791298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f000cfee0 0x7f8f0013e770 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-30T13:15:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 >> 172.21.15.43:0/3927791298 conn(0x7f8f000b2c70 msgr2=0x7f8f000b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-30T13:15:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 shutdown_connections 2024-03-30T13:15:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-30T13:15:01.353+0000 7f8f0604b640 1 -- 172.21.15.43:0/3927791298 wait complete. 2024-03-30T13:15:01.375 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-30T13:15:01.375 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-30T13:15:01.375 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-30T13:15:01.376 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:01.387 INFO:tasks.ceph.mds.a:Stopped 2024-03-30T13:15:01.388 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-30T13:15:01.388 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:01.450 INFO:tasks.ceph.mds.c:Stopped 2024-03-30T13:15:01.450 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-30T13:15:01.450 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:01.468 INFO:tasks.ceph.mds.b:Stopped 2024-03-30T13:15:01.468 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-30T13:15:01.468 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:01.527 INFO:tasks.ceph.mds.d:Stopped 2024-03-30T13:15:01.527 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-30T13:15:01.527 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-30T13:15:01.527 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:01.614 INFO:tasks.ceph.osd.0:Stopped 2024-03-30T13:15:01.614 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-30T13:15:01.614 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:03.320 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-03-30T13:15:07.716 INFO:tasks.ceph.osd.1:Stopped 2024-03-30T13:15:07.717 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-30T13:15:07.717 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:07.789 INFO:tasks.ceph.osd.2:Stopped 2024-03-30T13:15:07.789 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-30T13:15:07.790 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:07.879 INFO:tasks.ceph.osd.3:Stopped 2024-03-30T13:15:07.880 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-30T13:15:07.880 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:07.954 INFO:tasks.ceph.osd.4:Stopped 2024-03-30T13:15:07.954 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-30T13:15:07.954 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.054 INFO:tasks.ceph.osd.5:Stopped 2024-03-30T13:15:08.055 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-30T13:15:08.055 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.149 INFO:tasks.ceph.osd.6:Stopped 2024-03-30T13:15:08.149 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-30T13:15:08.149 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.245 INFO:tasks.ceph.osd.7:Stopped 2024-03-30T13:15:08.246 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-30T13:15:08.246 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-30T13:15:08.246 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.275 INFO:tasks.ceph.mgr.y:Stopped 2024-03-30T13:15:08.276 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-30T13:15:08.276 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.304 INFO:tasks.ceph.mgr.x:Stopped 2024-03-30T13:15:08.304 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-30T13:15:08.305 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-30T13:15:08.305 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.337 INFO:tasks.ceph.mon.a:Stopped 2024-03-30T13:15:08.338 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-30T13:15:08.338 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.360 INFO:tasks.ceph.mon.b:Stopped 2024-03-30T13:15:08.361 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-30T13:15:08.361 INFO:teuthology.orchestra.run:waiting for 300 2024-03-30T13:15:08.419 INFO:tasks.ceph.mon.c:Stopped 2024-03-30T13:15:08.419 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-30T13:15:08.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-30T13:15:08.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-03-30T13:15:08.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-30T13:15:08.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-30T13:15:08.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-03-30T13:15:08.439 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:15:08.439 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-30T13:15:08.900 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:15:08.901 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-30T13:15:09.264 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:15:09.264 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-30T13:15:09.499 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:15:09.499 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-30T13:15:09.707 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:15:09.707 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-30T13:15:10.001 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:15:10.002 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-30T13:15:10.214 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:15:10.215 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-30T13:15:10.400 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:15:10.400 DEBUG:teuthology.orchestra.run.smithi113:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-30T13:15:10.592 INFO:tasks.ceph:Archiving mon data... 2024-03-30T13:15:10.593 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/data/mon.a.tgz 2024-03-30T13:15:10.593 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-03-30T13:15:10.599 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.2IoHy0OPdH 2024-03-30T13:15:10.600 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2IoHy0OPdH 2024-03-30T13:15:10.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.2IoHy0OPdH 2024-03-30T13:15:10.789 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.2IoHy0OPdH is 351KB 2024-03-30T13:15:10.851 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.2IoHy0OPdH 2024-03-30T13:15:10.857 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/data/mon.b.tgz 2024-03-30T13:15:10.857 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-03-30T13:15:10.863 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.FkYne4FBqv 2024-03-30T13:15:10.863 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FkYne4FBqv 2024-03-30T13:15:11.012 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.FkYne4FBqv 2024-03-30T13:15:11.078 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.FkYne4FBqv is 461KB 2024-03-30T13:15:11.139 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.FkYne4FBqv 2024-03-30T13:15:11.144 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/data/mon.c.tgz 2024-03-30T13:15:11.144 DEBUG:teuthology.orchestra.run.smithi113:> mktemp 2024-03-30T13:15:11.193 INFO:teuthology.orchestra.run.smithi113.stdout:/tmp/tmp.yTyNlecGCc 2024-03-30T13:15:11.195 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.yTyNlecGCc 2024-03-30T13:15:11.348 DEBUG:teuthology.orchestra.run.smithi113:> sudo chmod 0666 /tmp/tmp.yTyNlecGCc 2024-03-30T13:15:11.413 DEBUG:teuthology.orchestra.remote:smithi113:/tmp/tmp.yTyNlecGCc is 461KB 2024-03-30T13:15:11.475 DEBUG:teuthology.orchestra.run.smithi113:> rm -fr /tmp/tmp.yTyNlecGCc 2024-03-30T13:15:11.481 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-30T13:15:11.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-30T13:15:11.484 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-30T13:15:11.545 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-30T13:15:11.585 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-30T13:15:11.586 INFO:tasks.ceph:Archiving crash dumps... 2024-03-30T13:15:11.587 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/remote/smithi043/crash 2024-03-30T13:15:11.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-30T13:15:11.602 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/remote/smithi113/crash 2024-03-30T13:15:11.604 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-30T13:15:11.641 INFO:tasks.ceph:Shutting down logrotate 2024-03-30T13:15:11.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:11.650 DEBUG:teuthology.orchestra.run.smithi113:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:11.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:11.740 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-30T13:15:11.779 INFO:tasks.ceph:Compressing logs... 2024-03-30T13:15:11.780 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-30T13:15:11.786 DEBUG:teuthology.orchestra.run.smithi113:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-30T13:15:11.816 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36331.log 2024-03-30T13:15:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42880.log 2024-03-30T13:15:11.817 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41391.log 2024-03-30T13:15:11.818 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-03-30T13:15:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40374.log 2024-03-30T13:15:11.818 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42880.log.gz 2024-03-30T13:15:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37102.log 2024-03-30T13:15:11.818 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40513.log 2024-03-30T13:15:11.821 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39674.log 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41391.log.gz 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37102.log.gz 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41291.log 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40374.log.gz 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38343.log 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40513.log.gz 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35680.log 2024-03-30T13:15:11.822 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34323.log 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39674.log.gz 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35680.log: 89.1% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38343.log.gz -- replaced with /var/log/ceph/ceph-client.admin.41291.log.gz 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33965.log 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35680.log.gz 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33595.log 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42802.log 2024-03-30T13:15:11.823 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34323.log.gz 2024-03-30T13:15:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36149.log 2024-03-30T13:15:11.824 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33595.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33965.log.gz 2024-03-30T13:15:11.829 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42802.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33595.log.gz 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34148.log 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42802.log.gz 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36149.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34148.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36149.log.gz 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32869.log 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34148.log.gz 2024-03-30T13:15:11.830 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42415.log 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35810.log 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38687.log 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42415.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32869.log.gz 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35810.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42415.log.gz 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39273.log 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35810.log.gz 2024-03-30T13:15:11.831 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40991.log 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38687.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.39273.log: -- /var/log/ceph/ceph-client.admin.27636.log 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38687.log.gz 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41886.log 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32947.log 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.27636.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.40991.log.gz 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39273.log.gz 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33070.log 2024-03-30T13:15:11.832 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27636.log.gz 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32947.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36549.log 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32947.log.gz 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33569.log 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36549.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33070.log.gz 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33569.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.42180.log -- replaced with /var/log/ceph/ceph-client.admin.36549.log.gz 2024-03-30T13:15:11.833 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:11.834 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.0.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33569.log.gz 2024-03-30T13:15:11.834 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42232.log 2024-03-30T13:15:11.834 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39924.log 2024-03-30T13:15:11.848 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42232.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42180.log.gz 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41191.log 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39924.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42232.log.gz 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35915.log 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37050.log 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39924.log.gz 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34877.log 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37050.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41191.log.gz 2024-03-30T13:15:11.849 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35915.log.gz 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37706.log 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37050.log.gz 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37576.log 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35622.log 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34877.log.gz 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37576.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37706.log.gz 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37888.log 2024-03-30T13:15:11.850 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37576.log.gz 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39200.log 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35622.log: /var/log/ceph/ceph-client.admin.37888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41995.log 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35622.log.gz 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39200.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37888.log.gz 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36496.log 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.41995.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33939.log 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.39200.log.gz 2024-03-30T13:15:11.851 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38765.log 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41995.log.gz 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33939.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36496.log.gz 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40641.log 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33939.log.gz 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35836.log 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34350.log 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40641.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38765.log.gz 2024-03-30T13:15:11.852 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39523.log 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35836.log.gz 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33098.log 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40641.log.gz 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39523.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34350.log.gz 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34579.log 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33098.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.39523.log.gz 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42776.log 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33098.log.gzgzip 2024-03-30T13:15:11.853 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34825.log 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34579.log: /var/log/ceph/ceph-client.admin.42776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36627.log 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34579.log.gz 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42776.log.gz 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37446.log 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34341.log 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34825.log.gz 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37446.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36627.log.gz 2024-03-30T13:15:11.854 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42362.log 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37446.log.gz 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37472.log 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34341.log.gz 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36123.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42362.log.gz 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42493.log 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-03-30T13:15:11.855 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37862.log 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.42493.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34981.log 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37472.log.gz 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37024.log 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42493.log.gz 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37862.log.gz 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38011.log 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37024.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34981.log.gz 2024-03-30T13:15:11.856 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37524.log 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38011.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37024.log.gz 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35137.log 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38011.log.gz 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36918.log 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35137.log: /var/log/ceph/ceph-client.admin.34955.log 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37524.log.gz 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42047.log 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35137.log.gz 2024-03-30T13:15:11.857 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36918.log.gz 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34955.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42047.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34955.log.gz 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42258.log 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42047.log.gz 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37498.log 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36575.log 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37498.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42258.log.gz 2024-03-30T13:15:11.858 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40841.log 2024-03-30T13:15:11.859 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37498.log.gz 2024-03-30T13:15:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2024-03-30T13:15:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.40841.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33254.log 2024-03-30T13:15:11.859 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36575.log.gz 2024-03-30T13:15:11.859 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35862.log 2024-03-30T13:15:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33138.log 2024-03-30T13:15:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27731.log 2024-03-30T13:15:11.893 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.3.33138.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27731.log: 91.7% -- replaced with /var/log/ceph/ceph-client.3.33138.log.gz 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27731.log.gz 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27827.log 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33019.log 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27827.log.gz 2024-03-30T13:15:11.894 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27779.log 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.33019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27707.log 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27683.log 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.1.33019.log.gz 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27779.log.gz 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27707.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27659.log 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27683.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.27707.log.gz 2024-03-30T13:15:11.895 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.27683.log.gz 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27755.log 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27659.log.gz 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.admin.27755.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.c.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27755.log.gz 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-30T13:15:11.896 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-30T13:15:11.897 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-30T13:15:11.897 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-30T13:15:11.897 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mds.b.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-30T13:15:11.897 INFO:teuthology.orchestra.run.smithi113.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-30T13:15:11.897 INFO:teuthology.orchestra.run.smithi113.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-30T13:15:11.897 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33309.log 2024-03-30T13:15:11.907 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27803.log 2024-03-30T13:15:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36788.log 2024-03-30T13:15:11.911 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2024-03-30T13:15:11.911 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40841.log.gz 2024-03-30T13:15:11.911 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37181.log 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35862.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33254.log.gz 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35732.log 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36788.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35862.log.gz 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34549.log 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36788.log.gz 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35732.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37181.log.gz 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39473.log 2024-03-30T13:15:11.912 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34549.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35732.log.gz 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36279.log 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34549.log.gz 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35546.log 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36279.log: /var/log/ceph/ceph-client.2.35546.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39473.log.gz 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39323.log 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36279.log.gz 2024-03-30T13:15:11.913 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.2.35546.log.gz 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33280.log 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41912.log 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41541.log: /var/log/ceph/ceph-client.admin.33280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41912.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39323.log.gz 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38609.log 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41912.log.gz 2024-03-30T13:15:11.914 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.33280.log.gz 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33003.log 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37836.log: gzip/var/log/ceph/ceph-client.admin.38609.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34903.log 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42021.log 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38609.log.gz 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33003.log.gz 2024-03-30T13:15:11.915 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34135.log 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42021.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34903.log.gz 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40568.log 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42021.log.gz 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42310.log 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34135.log: /var/log/ceph/ceph-client.admin.40568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42540.log 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5%/var/log/ceph/ceph-client.admin.42310.log: gzip -- replaced with /var/log/ceph/ceph-client.admin.34135.log.gz 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37420.log 2024-03-30T13:15:11.916 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9%/var/log/ceph/ceph-client.admin.42540.log: -- replaced with /var/log/ceph/ceph-client.admin.40568.log.gz 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42310.log.gz 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41341.log 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42540.log.gz 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36390.log 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41341.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37420.log.gz 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35993.log 2024-03-30T13:15:11.917 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37394.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36390.log.gz 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36762.log 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr: 91.5%/var/log/ceph/ceph-client.admin.35993.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41341.log.gz 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40274.log 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35993.log.gz 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40741.log 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36762.log: /var/log/ceph/ceph-client.admin.40274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35596.log 2024-03-30T13:15:11.918 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36762.log.gz 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38265.log 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40741.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40274.log.gz 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42854.log 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35596.log.gz 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33202.log 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40741.log.gz 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42696.log 2024-03-30T13:15:11.919 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38265.log.gz 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33202.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42854.log.gz 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34343.log 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33202.log.gz 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36970.log 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42696.log: /var/log/ceph/ceph-client.admin.34343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33991.log 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34343.log.gz 2024-03-30T13:15:11.920 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39098.log 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33991.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36970.log.gz 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33124.log 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33991.log.gz 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32790.log 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39098.log: /var/log/ceph/ceph-client.admin.33124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36227.log 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33124.log.gz 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42441.log 2024-03-30T13:15:11.921 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.39098.log.gz 2024-03-30T13:15:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39724.log 2024-03-30T13:15:11.922 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32790.log.gz 2024-03-30T13:15:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42441.log: gzip -5 88.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.36227.log.gz /var/log/ceph/ceph-client.admin.37602.log 2024-03-30T13:15:11.922 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:11.922 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42441.log.gz 2024-03-30T13:15:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43112.log 2024-03-30T13:15:11.922 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32895.log 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37602.log: /var/log/ceph/ceph-client.admin.43112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39824.log 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37602.log.gz 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8%/var/log/ceph/ceph-client.admin.32895.log: -- replaced with /var/log/ceph/ceph-client.admin.39724.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43112.log.gz 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41691.log 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32895.log.gz 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38317.log 2024-03-30T13:15:11.923 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41741.log 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37342.log 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38317.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39824.log.gz 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41691.log.gz 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36653.log 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38317.log.gz 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34581.log 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37342.log: /var/log/ceph/ceph-client.admin.36653.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-30T13:15:11.924 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37342.log.gz 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41741.log.gz 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36653.log.gz 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38843.log 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.3.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34581.log.gz 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38583.log 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34572.log 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38505.log 2024-03-30T13:15:11.925 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.38843.log.gz 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34572.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38583.log.gz 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37758.log 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35215.log 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6%/var/log/ceph/ceph-client.admin.38505.log: -- replaced with /var/log/ceph/ceph-client.admin.34572.log.gz/var/log/ceph/ceph-client.admin.37758.log: 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38395.log 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37758.log.gz 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35215.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38505.log.gz 2024-03-30T13:15:11.926 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33674.log 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38395.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36201.log 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35215.log.gz 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33674.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38395.log.gz 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33621.log 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33674.log.gz 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40024.log 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36201.log: /var/log/ceph/ceph-client.admin.33621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33726.log 2024-03-30T13:15:11.927 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36470.log 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33621.log.gz 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36201.log.gz 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33726.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40024.log.gz 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33726.log.gz 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42958.log 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39573.log 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7%/var/log/ceph/ceph-client.admin.42958.log: -- replaced with /var/log/ceph/ceph-client.admin.36470.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41491.log 2024-03-30T13:15:11.928 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36253.log 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42958.log.gz 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33804.log 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41491.log: 91.5%/var/log/ceph/ceph-client.admin.36253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33913.log 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39573.log.gz 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33804.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36253.log.gz 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33804.log.gz 2024-03-30T13:15:11.929 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41491.log.gz 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36684.log 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33913.log.gz 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35276.log 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35163.log 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37285.log 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36684.log.gz 2024-03-30T13:15:11.930 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35163.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35276.log.gz 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35163.log.gz 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36305.log 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36097.log 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8%/var/log/ceph/ceph-client.admin.36305.log: -- replaced with /var/log/ceph/ceph-client.admin.37285.log.gz 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36305.log.gz 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36097.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36097.log.gz 2024-03-30T13:15:11.931 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32843.log 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35888.log 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33778.log 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32843.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35888.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32843.log.gz 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35888.log.gz 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40425.log 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27526.log 2024-03-30T13:15:11.932 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40425.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33778.log.gz 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39060.log 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7%/var/log/ceph/ceph-client.admin.27526.log: -- replaced with /var/log/ceph/ceph-client.admin.40425.log.gz 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27526.log.gz 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42128.log 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38453.log 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36840.log 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42128.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39060.log.gz 2024-03-30T13:15:11.933 INFO:teuthology.orchestra.run.smithi043.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42128.log.gz 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38453.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38453.log.gz 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34095.log 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37128.log 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36840.log: /var/log/ceph/ceph-client.admin.34095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35111.log 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36840.log.gz 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34095.log.gz 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40324.log 2024-03-30T13:15:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37128.log: /var/log/ceph/ceph-client.admin.35111.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37128.log.gz 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35111.log.gz 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42750.log 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38817.log 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40324.log: /var/log/ceph/ceph-client.admin.42750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34537.log 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38817.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42750.log.gz 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38817.log.gz 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40324.log.gz 2024-03-30T13:15:11.935 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39423.log 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36364.log 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39423.log: 88.3%/var/log/ceph/ceph-client.admin.36364.log: -- replaced with /var/log/ceph/ceph-client.admin.34537.log.gz 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36364.log.gz 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38531.log 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39423.log.gz/var/log/ceph/ceph-client.admin.42618.log: 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34043.log 2024-03-30T13:15:11.936 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38160.log 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42984.log 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34043.log: /var/log/ceph/ceph-client.admin.38160.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34043.log.gz 89.0% 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38531.log.gz 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38291.log 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38160.log.gz 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38791.log 2024-03-30T13:15:11.937 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42984.log: /var/log/ceph/ceph-client.admin.38291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38369.log 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38791.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42984.log.gz 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38291.log.gz 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33543.log 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38791.log.gz 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38949.log 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38369.log: /var/log/ceph/ceph-client.admin.33543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42932.log 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.38949.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33543.log.gz 2024-03-30T13:15:11.938 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38369.log.gz 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32692.log 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42932.log: /var/log/ceph/ceph-client.admin.32692.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42932.log.gz 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32692.log.gz 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42073.log 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33176.log 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.42073.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42073.log.gz 2024-03-30T13:15:11.939 INFO:teuthology.orchestra.run.smithi043.stderr: 85.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38186.log 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42592.log 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33176.log: /var/log/ceph/ceph-client.admin.38186.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33176.log.gz 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38186.log.gz 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42388.log 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34771.log 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42592.log: /var/log/ceph/ceph-client.admin.42388.log: 88.6% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42592.log.gz 2024-03-30T13:15:11.940 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42388.log.gz 2024-03-30T13:15:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35451.log 2024-03-30T13:15:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41441.log 2024-03-30T13:15:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34771.log: /var/log/ceph/ceph-client.admin.35451.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34771.log.gz 2024-03-30T13:15:11.941 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35451.log.gz 2024-03-30T13:15:11.941 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-03-30T13:15:11.942 INFO:teuthology.orchestra.run.smithi113.stderr:/var/log/ceph/ceph-client.1.33309.log: /var/log/ceph/ceph-client.admin.27803.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27803.log.gz 2024-03-30T13:15:11.943 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.1.33309.log.gz 2024-03-30T13:15:11.961 INFO:teuthology.orchestra.run.smithi113.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-30T13:15:11.965 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41441.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41441.log.gz 2024-03-30T13:15:11.965 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38037.log 2024-03-30T13:15:11.965 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34353.log 2024-03-30T13:15:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38037.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz 2024-03-30T13:15:11.966 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40463.log 2024-03-30T13:15:11.966 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38037.log.gz 2024-03-30T13:15:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34353.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34353.log.gz 2024-03-30T13:15:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43086.log 2024-03-30T13:15:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35784.log 2024-03-30T13:15:11.966 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40463.log: /var/log/ceph/ceph-client.admin.43086.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43086.log.gz 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40463.log.gz 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39974.log 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35059.log 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35784.log: /var/log/ceph/ceph-client.admin.39974.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35784.log.gz 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35401.log 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35059.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.39974.log.gz 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35059.log.gz 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32725.log 2024-03-30T13:15:11.967 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41969.log 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.35401.log: /var/log/ceph/ceph-client.admin.32725.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32725.log.gz 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.35401.log.gz 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39373.log 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40174.log 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41969.log: /var/log/ceph/ceph-client.admin.39373.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41969.log.gz 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38213.log 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40174.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39373.log.gz 2024-03-30T13:15:11.968 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40174.log.gz 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32816.log 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43062.log 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38213.log: /var/log/ceph/ceph-client.admin.32816.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38213.log.gz 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32816.log.gz 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34145.log 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43062.log: /var/log/ceph/ceph-client.admin.41834.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43062.log.gz 2024-03-30T13:15:11.969 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42336.log 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34145.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34145.log.gz 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38069.log 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39623.log 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42336.log: /var/log/ceph/ceph-client.admin.38069.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42336.log.gz 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38069.log.gz 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43010.log 2024-03-30T13:15:11.970 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36444.log 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39623.log: /var/log/ceph/ceph-client.admin.43010.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43010.log.gz 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.39623.log.gz 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33150.log 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38479.log 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36444.log: /var/log/ceph/ceph-client.admin.33150.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36444.log.gz 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33150.log.gz 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37914.log 2024-03-30T13:15:11.971 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38479.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38479.log.gz 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42467.log 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37914.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37914.log.gz 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38713.log 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37155.log 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42467.log: /var/log/ceph/ceph-client.admin.38713.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42467.log.gz 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38713.log.gz 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41792.log 2024-03-30T13:15:11.972 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38893.log 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37155.log: /var/log/ceph/ceph-client.admin.41792.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37155.log.gz 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41792.log.gz 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32921.log 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41041.log 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38893.log: /var/log/ceph/ceph-client.admin.32921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38893.log.gz 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32921.log.gz 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33330.log 2024-03-30T13:15:11.973 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41041.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41041.log.gz 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37654.log 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41141.log 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33330.log: /var/log/ceph/ceph-client.admin.37654.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33330.log.gz 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37654.log.gz 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37948.log 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36736.log 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41141.log: /var/log/ceph/ceph-client.admin.37948.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37948.log.gz 2024-03-30T13:15:11.974 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41141.log.gz 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34147.log 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33700.log 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36736.log: /var/log/ceph/ceph-client.admin.34147.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36736.log.gz 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33458.log 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34147.log.gz 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33700.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33700.log.gz 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-03-30T13:15:11.975 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37810.log 2024-03-30T13:15:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33458.log: /var/log/ceph/ceph-client.admin.34069.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gz 2024-03-30T13:15:11.976 INFO:teuthology.orchestra.run.smithi043.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.33458.log.gz 2024-03-30T13:15:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39148.log 2024-03-30T13:15:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38557.log 2024-03-30T13:15:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37810.log: /var/log/ceph/ceph-client.admin.39148.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37810.log.gz 2024-03-30T13:15:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35967.log 2024-03-30T13:15:11.976 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38557.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39148.log.gz 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38557.log.gz 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36071.log 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27417.log 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35967.log: /var/log/ceph/ceph-client.admin.36071.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35967.log.gz 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36071.log.gz 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36998.log 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41641.log 2024-03-30T13:15:11.977 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.27417.log: /var/log/ceph/ceph-client.admin.36998.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36998.log.gz 2024-03-30T13:15:11.978 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27417.log.gz 2024-03-30T13:15:11.978 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36892.log 2024-03-30T13:15:11.978 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41091.log 2024-03-30T13:15:11.978 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41641.log: /var/log/ceph/ceph-client.admin.36892.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36892.log.gz 2024-03-30T13:15:11.978 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41641.log.gz 2024-03-30T13:15:11.978 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37076.log 2024-03-30T13:15:11.978 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41091.log: /var/log/ceph/ceph-client.admin.37076.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37076.log.gz 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.41091.log.gz 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40224.log 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32975.log 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34797.log: /var/log/ceph/ceph-client.admin.40224.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38635.log 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40224.log.gz 2024-03-30T13:15:11.979 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.32975.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32975.log.gz 2024-03-30T13:15:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-30T13:15:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-30T13:15:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38635.log: /var/log/ceph/ceph.audit.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38635.log.gz 2024-03-30T13:15:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34546.log 2024-03-30T13:15:11.980 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.c.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-30T13:15:11.984 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38739.log 2024-03-30T13:15:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34546.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34546.log.gz 2024-03-30T13:15:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35007.log 2024-03-30T13:15:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38739.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38739.log.gz 2024-03-30T13:15:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37207.log 2024-03-30T13:15:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35007.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35007.log.gz 2024-03-30T13:15:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34149.log 2024-03-30T13:15:11.985 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37207.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37207.log.gz 2024-03-30T13:15:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42154.log 2024-03-30T13:15:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34149.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34149.log.gz 2024-03-30T13:15:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-30T13:15:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42154.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42154.log.gz 2024-03-30T13:15:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35941.log 2024-03-30T13:15:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.a.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-30T13:15:11.986 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35706.log 2024-03-30T13:15:11.987 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35941.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35941.log.gz 2024-03-30T13:15:11.988 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38661.log 2024-03-30T13:15:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35706.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35706.log.gz 2024-03-30T13:15:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38239.log 2024-03-30T13:15:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38661.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38661.log.gz 2024-03-30T13:15:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39003.log 2024-03-30T13:15:11.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38239.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38239.log.gz 2024-03-30T13:15:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37259.log 2024-03-30T13:15:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39003.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39003.log.gz 2024-03-30T13:15:11.990 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-03-30T13:15:11.991 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37259.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37259.log.gz 2024-03-30T13:15:11.992 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35033.log 2024-03-30T13:15:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33887.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-03-30T13:15:11.993 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34142.log 2024-03-30T13:15:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35033.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35033.log.gz 2024-03-30T13:15:11.994 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34017.log 2024-03-30T13:15:11.995 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34142.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34142.log.gz 2024-03-30T13:15:11.996 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42906.log 2024-03-30T13:15:11.997 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34017.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34017.log.gz 2024-03-30T13:15:11.998 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39874.log 2024-03-30T13:15:11.998 INFO:teuthology.orchestra.run.smithi043.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-30T13:15:11.998 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42906.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42906.log.gz 2024-03-30T13:15:11.998 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37732.log 2024-03-30T13:15:11.999 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40124.log 2024-03-30T13:15:12.000 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37732.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37732.log.gz 2024-03-30T13:15:12.001 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.39874.log.gz 2024-03-30T13:15:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34400.log 2024-03-30T13:15:12.001 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-03-30T13:15:12.002 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40124.log: /var/log/ceph/ceph-client.admin.34400.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34400.log.gz 91.3% 2024-03-30T13:15:12.002 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40124.log.gz 2024-03-30T13:15:12.002 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36418.log 2024-03-30T13:15:12.002 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42102.log 2024-03-30T13:15:12.003 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34605.log: /var/log/ceph/ceph-client.admin.36418.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-03-30T13:15:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35085.log 2024-03-30T13:15:12.004 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36418.log.gz 2024-03-30T13:15:12.004 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42102.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42102.log.gz 2024-03-30T13:15:12.005 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42206.log 2024-03-30T13:15:12.005 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33038.log 2024-03-30T13:15:12.006 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35085.log: /var/log/ceph/ceph-client.admin.42206.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35085.log.gz 2024-03-30T13:15:12.006 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42722.log 2024-03-30T13:15:12.006 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42206.log.gz 2024-03-30T13:15:12.007 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.33038.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33038.log.gz 2024-03-30T13:15:12.007 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36814.log 2024-03-30T13:15:12.007 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34929.log 2024-03-30T13:15:12.008 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42722.log: /var/log/ceph/ceph-client.admin.36814.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42722.log.gz 2024-03-30T13:15:12.008 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34851.log 2024-03-30T13:15:12.008 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36814.log.gz 2024-03-30T13:15:12.009 INFO:teuthology.orchestra.run.smithi113.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-30T13:15:12.010 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34929.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34929.log.gz 2024-03-30T13:15:12.010 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-03-30T13:15:12.010 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34851.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34851.log.gz 2024-03-30T13:15:12.010 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40074.log 2024-03-30T13:15:12.011 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38423.log 2024-03-30T13:15:12.011 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40074.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz 2024-03-30T13:15:12.012 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40074.log.gz 2024-03-30T13:15:12.012 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41938.log 2024-03-30T13:15:12.013 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39774.log 2024-03-30T13:15:12.013 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.38423.log: /var/log/ceph/ceph-client.admin.41938.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38423.log.gz 2024-03-30T13:15:12.013 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41938.log.gz 2024-03-30T13:15:12.013 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36866.log 2024-03-30T13:15:12.014 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36601.log 2024-03-30T13:15:12.015 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39774.log: /var/log/ceph/ceph-client.admin.36866.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36866.log.gz 2024-03-30T13:15:12.015 INFO:teuthology.orchestra.run.smithi043.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.39774.log.gz 2024-03-30T13:15:12.015 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41591.log 2024-03-30T13:15:12.016 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33517.log 2024-03-30T13:15:12.016 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36601.log: /var/log/ceph/ceph-client.admin.41591.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36601.log.gz 2024-03-30T13:15:12.016 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40791.log 2024-03-30T13:15:12.017 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2%/var/log/ceph/ceph-client.admin.33517.log: -- replaced with /var/log/ceph/ceph-client.admin.41591.log.gz 2024-03-30T13:15:12.018 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33517.log.gz 2024-03-30T13:15:12.018 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2024-03-30T13:15:12.018 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22609.log 2024-03-30T13:15:12.019 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40791.log: /var/log/ceph/ceph-client.admin.33835.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2024-03-30T13:15:12.019 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40791.log.gz 2024-03-30T13:15:12.019 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33647.log 2024-03-30T13:15:12.020 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41241.log 2024-03-30T13:15:12.020 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.tmp-client.admin.22609.log: /var/log/ceph/ceph-client.admin.33647.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22609.log.gz 2024-03-30T13:15:12.021 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37368.log 2024-03-30T13:15:12.021 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33647.log.gz 2024-03-30T13:15:12.021 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41241.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41241.log.gz 2024-03-30T13:15:12.022 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33752.log 2024-03-30T13:15:12.023 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37233.log 2024-03-30T13:15:12.023 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37368.log: /var/log/ceph/ceph-client.admin.33752.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37368.log.gz 2024-03-30T13:15:12.023 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33752.log.gz 2024-03-30T13:15:12.023 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-03-30T13:15:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39029.log 2024-03-30T13:15:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37233.log: /var/log/ceph/ceph-client.admin.41860.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-03-30T13:15:12.024 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37233.log.gz 2024-03-30T13:15:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33861.log 2024-03-30T13:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2024-03-30T13:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39029.log: /var/log/ceph/ceph-client.admin.33861.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39029.log.gz 2024-03-30T13:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33861.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36019.log 2024-03-30T13:15:12.026 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37628.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2024-03-30T13:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40941.log 2024-03-30T13:15:12.027 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35758.log 2024-03-30T13:15:12.028 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36019.log: /var/log/ceph/ceph-client.admin.40941.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36019.log.gz 2024-03-30T13:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-03-30T13:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40941.log.gz 2024-03-30T13:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35758.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35758.log.gz 2024-03-30T13:15:12.029 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32635.log 2024-03-30T13:15:12.030 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42284.log 2024-03-30T13:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40891.log: /var/log/ceph/ceph-client.admin.32635.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32635.log.gz 2024-03-30T13:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-03-30T13:15:12.031 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36045.log 2024-03-30T13:15:12.032 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-03-30T13:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42284.log: /var/log/ceph/ceph-client.admin.36045.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42284.log.gz 2024-03-30T13:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36045.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36175.log 2024-03-30T13:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:12.033 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.35189.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz 2024-03-30T13:15:12.034 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34146.log 2024-03-30T13:15:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34322.log 2024-03-30T13:15:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36175.log: /var/log/ceph/ceph-client.admin.34146.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36175.log.gz 2024-03-30T13:15:12.035 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34146.log.gz 2024-03-30T13:15:12.035 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43036.log 2024-03-30T13:15:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34322.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34322.log.gz 2024-03-30T13:15:12.036 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42644.log 2024-03-30T13:15:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37680.log 2024-03-30T13:15:12.037 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42644.log: /var/log/ceph/ceph-client.admin.43036.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43036.log.gz 2024-03-30T13:15:12.038 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42644.log.gz 2024-03-30T13:15:12.038 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36710.log 2024-03-30T13:15:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37784.log 2024-03-30T13:15:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37680.log: /var/log/ceph/ceph-client.admin.36710.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37680.log.gz 2024-03-30T13:15:12.039 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36710.log.gz 2024-03-30T13:15:12.039 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42670.log 2024-03-30T13:15:12.040 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37784.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37784.log.gz 2024-03-30T13:15:12.040 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38134.log 2024-03-30T13:15:12.041 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42566.log 2024-03-30T13:15:12.041 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42670.log: /var/log/ceph/ceph-client.admin.38134.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42670.log.gz 2024-03-30T13:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38134.log.gz 2024-03-30T13:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34315.log 2024-03-30T13:15:12.042 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42566.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42566.log.gz 2024-03-30T13:15:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34541.log 2024-03-30T13:15:12.043 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37550.log 2024-03-30T13:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34315.log: /var/log/ceph/ceph-client.admin.34541.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34315.log.gz 2024-03-30T13:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34541.log.gz 2024-03-30T13:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35654.log 2024-03-30T13:15:12.044 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42828.log 2024-03-30T13:15:12.045 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.37550.log: /var/log/ceph/ceph-client.admin.35654.log: 88.8% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35654.log.gz 2024-03-30T13:15:12.046 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37550.log.gz 2024-03-30T13:15:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33228.log 2024-03-30T13:15:12.046 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36944.log 2024-03-30T13:15:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42828.log: /var/log/ceph/ceph-client.admin.33228.log: 89.2% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42828.log.gz 2024-03-30T13:15:12.048 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33228.log.gz 2024-03-30T13:15:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-03-30T13:15:12.048 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.36944.log: /var/log/ceph/ceph-client.admin.34745.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36944.log.gz 2024-03-30T13:15:12.048 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-03-30T13:15:12.064 INFO:teuthology.orchestra.run.smithi113.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-30T13:15:12.193 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-30T13:15:12.661 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-30T13:15:12.765 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-30T13:15:12.912 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-30T13:15:12.923 INFO:teuthology.orchestra.run.smithi043.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-30T13:15:12.930 INFO:teuthology.orchestra.run.smithi113.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-30T13:15:12.955 INFO:teuthology.orchestra.run.smithi113.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-30T13:15:13.009 INFO:teuthology.orchestra.run.smithi113.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-30T13:15:13.017 INFO:teuthology.orchestra.run.smithi043.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-30T13:15:13.133 INFO:teuthology.orchestra.run.smithi113.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-30T13:15:13.135 INFO:teuthology.orchestra.run.smithi113.stderr: 2024-03-30T13:15:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:real 0m1.307s 2024-03-30T13:15:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:user 0m5.499s 2024-03-30T13:15:13.135 INFO:teuthology.orchestra.run.smithi113.stderr:sys 0m0.212s 2024-03-30T13:15:13.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-03-30T13:15:13.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-03-30T13:15:13.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-03-30T13:15:13.824 INFO:teuthology.orchestra.run.smithi043.stderr: 96.9% -- replaced with /var/log/ceph/ceph-client.0.38949.log.gz 2024-03-30T13:15:13.826 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-30T13:15:13.826 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m2.038s 2024-03-30T13:15:13.826 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m6.620s 2024-03-30T13:15:13.826 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m0.347s 2024-03-30T13:15:13.827 INFO:tasks.ceph:Archiving logs... 2024-03-30T13:15:13.828 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/remote/smithi043/log 2024-03-30T13:15:13.829 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T13:15:14.736 DEBUG:teuthology.misc:Transferring archived files from smithi113:/var/log/ceph to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/remote/smithi113/log 2024-03-30T13:15:14.737 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-30T13:15:15.292 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-30T13:15:15.302 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-30T13:15:15.302 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T13:15:15.305 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-30T13:15:15.384 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-30T13:15:15.393 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-30T13:15:15.394 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:15:15.396 DEBUG:teuthology.orchestra.run.smithi113:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-30T13:15:15.408 INFO:teuthology.orchestra.run.smithi043.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T13:15:15.408 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================== 2024-03-30T13:15:15.408 INFO:teuthology.orchestra.run.smithi043.stdout:+hv01.front.sepi 44.4.53.2 2 u 43 64 377 0.083 +0.120 0.914 2024-03-30T13:15:15.408 INFO:teuthology.orchestra.run.smithi043.stdout:*hv02.front.sepi 63.231.80.2 3 u 45 64 377 0.061 -0.995 0.862 2024-03-30T13:15:15.408 INFO:teuthology.orchestra.run.smithi043.stdout:+hv03.front.sepi 74.6.168.72 3 u 45 64 377 0.093 -0.999 0.564 2024-03-30T13:15:15.408 INFO:teuthology.orchestra.run.smithi043.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:15:15.410 INFO:teuthology.orchestra.run.smithi113.stdout: remote refid st t when poll reach delay offset jitter 2024-03-30T13:15:15.410 INFO:teuthology.orchestra.run.smithi113.stdout:============================================================================== 2024-03-30T13:15:15.410 INFO:teuthology.orchestra.run.smithi113.stdout:*hv01.front.sepi 44.4.53.2 2 u 42 64 377 0.070 -1.037 0.667 2024-03-30T13:15:15.410 INFO:teuthology.orchestra.run.smithi113.stdout:+hv02.front.sepi 63.231.80.2 3 u 44 64 377 0.119 -0.747 0.664 2024-03-30T13:15:15.410 INFO:teuthology.orchestra.run.smithi113.stdout:+hv03.front.sepi 74.6.168.72 3 u 35 64 377 0.083 -1.102 0.604 2024-03-30T13:15:15.411 INFO:teuthology.orchestra.run.smithi113.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-30T13:15:15.411 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-30T13:15:15.481 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-30T13:15:15.482 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-30T13:15:15.648 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-30T13:15:15.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-30T13:15:15.907 INFO:teuthology.task.internal:Duration was 935.721398 seconds 2024-03-30T13:15:15.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-30T13:15:16.005 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-30T13:15:16.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T13:15:16.008 DEBUG:teuthology.orchestra.run.smithi113:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-30T13:15:16.046 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-30T13:15:16.046 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-03-30T13:15:16.046 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T13:15:16.098 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi113.front.sepia.ceph.com 2024-03-30T13:15:16.098 DEBUG:teuthology.orchestra.run.smithi113:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-30T13:15:16.108 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-30T13:15:16.109 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T13:15:16.142 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-30T13:15:16.182 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-30T13:15:16.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T13:15:16.218 DEBUG:teuthology.orchestra.run.smithi113:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-30T13:15:16.283 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-30T13:15:16.293 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-30T13:15:16.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T13:15:16.306 DEBUG:teuthology.orchestra.run.smithi113:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-30T13:15:16.317 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-30T13:15:16.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:15:16.351 DEBUG:teuthology.orchestra.run.smithi113:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:15:16.360 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-03-30T13:15:16.369 INFO:teuthology.orchestra.run.smithi113.stdout:kernel.core_pattern = core 2024-03-30T13:15:16.383 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:15:16.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:15:16.423 DEBUG:teuthology.orchestra.run.smithi113:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-30T13:15:16.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-30T13:15:16.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-30T13:15:16.527 INFO:teuthology.task.internal:Transferring archived files... 2024-03-30T13:15:16.528 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/remote/smithi043 2024-03-30T13:15:16.528 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T13:15:16.551 DEBUG:teuthology.misc:Transferring archived files from smithi113:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150/remote/smithi113 2024-03-30T13:15:16.551 DEBUG:teuthology.orchestra.run.smithi113:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-30T13:15:16.572 INFO:teuthology.task.internal:Removing archive directory... 2024-03-30T13:15:16.572 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T13:15:16.590 DEBUG:teuthology.orchestra.run.smithi113:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-30T13:15:16.617 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-30T13:15:16.659 INFO:teuthology.task.internal:Not uploading archives. 2024-03-30T13:15:16.659 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-30T13:15:16.806 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-30T13:15:16.806 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T13:15:16.809 DEBUG:teuthology.orchestra.run.smithi113:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-30T13:15:16.813 INFO:teuthology.orchestra.run.smithi043.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 13:15 /home/ubuntu/cephtest 2024-03-30T13:15:16.816 INFO:teuthology.orchestra.run.smithi113.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 30 13:15 /home/ubuntu/cephtest 2024-03-30T13:15:16.818 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-30T13:15:16.829 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-30T13:15:17.169 INFO:teuthology.nuke:Checking targets against current locks 2024-03-30T13:15:17.188 DEBUG:teuthology.nuke:shortname: smithi043 2024-03-30T13:15:17.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T13:15:17.207 DEBUG:teuthology.nuke:shortname: smithi113 2024-03-30T13:15:17.207 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-30T13:15:17.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:21.705614', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNaqCXgku5wm8rAduSfSpUMrux+P1koXaV2BHr2lfdroQgKNRMUBxi1nIm84HEYi0uXdSPkeSvgHbdEy4/3xDJQ='} 2024-03-30T13:15:17.228 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi113.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2024-03-29_13:38:54-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro-default-smithi/7629150', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-30 12:50:21.703701', 'locked_by': 'scheduled_mchangir@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO8cAcGOMjXJOBLP1+rNVUwlOfwTYgbuBqwTwBM8uzVGbh6LQallKPwYP2ellDlBciZriUua4cEwE3jlHVMtDnM='} 2024-03-30T13:15:17.252 INFO:teuthology.orchestra.console.smithi043:Power off 2024-03-30T13:15:17.252 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T13:15:17.277 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-03-30T13:15:17.277 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:15:17.292 INFO:teuthology.orchestra.console.smithi113:Power off 2024-03-30T13:15:17.293 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-30T13:15:17.305 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-03-30T13:15:17.318 DEBUG:teuthology.orchestra.console.smithi113:power off output: Chassis Power Control: Down/Off 2024-03-30T13:15:17.319 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:15:17.343 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-03-30T13:15:21.307 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:15:21.344 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:15:21.419 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-03-30T13:15:21.456 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is on 2024-03-30T13:15:25.422 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:15:25.457 DEBUG:teuthology.orchestra.console.smithi113:pexpect command: ipmitool -H smithi113.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:15:25.534 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-03-30T13:15:25.569 DEBUG:teuthology.orchestra.console.smithi113:check power output: Chassis Power is off 2024-03-30T13:15:25.669 INFO:teuthology.orchestra.console.smithi113:Power off completed 2024-03-30T13:15:29.536 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-30T13:15:29.646 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-03-30T13:15:29.747 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-03-30T13:15:29.851 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/uninlining} duration: 935.7213976383209 failure_reason: 'Test failure: test_data_uninlining (tasks.cephfs.test_uninlining.TestDataUninlining)' flavor: default owner: scheduled_mchangir@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=33bf6c56bd6b4b00811a359c831529f9 status: fail success: false 2024-03-30T13:15:29.852 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-30T13:15:29.936 INFO:teuthology.run:FAIL